2023-09-23T09:08:21.596 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-23T09:08:21.596 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639 --owner scheduled_vshankar@teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} --name vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi -- /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/orig.config.yaml 2023-09-23T09:08:21.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T09:08:21.782 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639 branch: wip-vshankar-testing-20230922.144532 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} email: vshankar@redhat.com first_in_suite: false job_id: '7401639' kernel: client: sha1: distro flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-20230922.144532 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230922.144532 sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8196 sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a sleep_before_teardown: 0 subset: 57/1024 suite: fs suite_branch: wip-vshankar-testing-20230922.144532 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-23_02:50:30 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-23T09:08:21.783 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa; will attempt to use it 2023-09-23T09:08:21.784 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks 2023-09-23T09:08:21.784 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-23T09:08:21.786 INFO:teuthology.task.internal:Checking packages... 2023-09-23T09:08:21.809 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '862663913a74a99e1c52d50e6a7e59dd6705749a' 2023-09-23T09:08:21.809 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-23T09:08:21.809 INFO:teuthology.packaging:ref: None 2023-09-23T09:08:21.809 INFO:teuthology.packaging:tag: None 2023-09-23T09:08:21.809 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230922.144532 2023-09-23T09:08:21.810 INFO:teuthology.packaging:sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:08:21.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20230922.144532 2023-09-23T09:08:22.069 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6336.g86266391 2023-09-23T09:08:22.069 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-23T09:08:22.075 INFO:teuthology.task.internal:no buildpackages task found 2023-09-23T09:08:22.076 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-23T09:08:22.109 INFO:teuthology.task.internal:Saving configuration 2023-09-23T09:08:22.131 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-23T09:08:22.143 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T09:08:22.167 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 09:03:12.678871', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T09:08:22.190 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 09:03:12.676220', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T09:08:22.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 09:03:12.677677', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T09:08:22.212 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-23T09:08:22.218 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-23T09:08:22.218 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-23T09:08:22.219 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-23T09:08:22.219 INFO:teuthology.run_tasks:Running task console_log... 2023-09-23T09:08:22.347 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcfcc2e7b80>, signals=[15]) 2023-09-23T09:08:22.347 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-23T09:08:22.354 INFO:teuthology.task.internal:Opening connections... 2023-09-23T09:08:22.354 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:08:22.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T09:08:22.423 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2023-09-23T09:08:22.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T09:08:22.493 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2023-09-23T09:08:22.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T09:08:22.560 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-23T09:08:22.567 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2023-09-23T09:08:22.588 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-09-23T09:08:22.588 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-09-23T09:08:22.649 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Red Hat Enterprise Linux" 2023-09-23T09:08:22.649 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8.6 (Ootpa)" 2023-09-23T09:08:22.649 INFO:teuthology.orchestra.run.smithi045.stdout:ID="rhel" 2023-09-23T09:08:22.649 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="fedora" 2023-09-23T09:08:22.650 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8.6" 2023-09-23T09:08:22.650 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-09-23T09:08:22.650 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-23T09:08:22.650 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-09-23T09:08:22.650 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-23T09:08:22.650 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.redhat.com/" 2023-09-23T09:08:22.650 INFO:teuthology.orchestra.run.smithi045.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-23T09:08:22.651 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T09:08:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:08:22.651 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T09:08:22.651 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-23T09:08:22.651 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-23T09:08:22.651 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-23T09:08:22.652 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-09-23T09:08:22.679 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2023-09-23T09:08:22.699 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-09-23T09:08:22.699 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-09-23T09:08:22.758 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/" 2023-09-23T09:08:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-23T09:08:22.760 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T09:08:22.760 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:08:22.760 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T09:08:22.760 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-23T09:08:22.760 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-23T09:08:22.760 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-23T09:08:22.760 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-09-23T09:08:22.779 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-09-23T09:08:22.800 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-09-23T09:08:22.800 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-09-23T09:08:22.859 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.6 (Ootpa)" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.6" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-23T09:08:22.860 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T09:08:22.861 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:08:22.861 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T09:08:22.861 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-23T09:08:22.861 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-23T09:08:22.861 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-23T09:08:22.861 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-09-23T09:08:22.878 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-23T09:08:22.888 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-23T09:08:22.934 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-23T09:08:22.935 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2023-09-23T09:08:22.938 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2023-09-23T09:08:22.940 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2023-09-23T09:08:22.959 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-23T09:08:22.967 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-23T09:08:22.967 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2023-09-23T09:08:22.998 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2023-09-23T09:08:23.001 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2023-09-23T09:08:23.031 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-23T09:08:23.102 INFO:teuthology.run_tasks:Running task kernel... 2023-09-23T09:08:23.116 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-09-23T09:08:23.116 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-09-23T09:08:23.116 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-09-23T09:08:23.117 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T09:08:23.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:08:23.136 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-09-23T09:08:23.197 INFO:teuthology.orchestra.run.smithi045.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-23T09:08:23.197 INFO:teuthology.task.kernel:Running kernel on smithi045: 4.18.0-372.9.1.el8.x86_64 2023-09-23T09:08:23.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2023-09-23T09:08:24.456 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-09-23T09:08:24.456 INFO:teuthology.orchestra.run.smithi045.stdout:Unable to read consumer identity 2023-09-23T09:08:25.973 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-09-23T09:08:26.890 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-09-23T09:08:30.590 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 493 kB/s | 24 kB 00:00 2023-09-23T09:08:31.335 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-09-23T09:08:31.396 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-09-23T09:08:31.397 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-09-23T09:08:31.397 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-09-23T09:08:31.507 DEBUG:teuthology.orchestra.run.smithi045:> echo no | sudo yum reinstall kernel || true 2023-09-23T09:08:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-09-23T09:08:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:Unable to read consumer identity 2023-09-23T09:08:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:02 ago on Sat 23 Sep 2023 09:08:30 AM UTC. 2023-09-23T09:08:32.217 INFO:teuthology.orchestra.run.smithi045.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-23T09:08:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:Error: No packages marked for reinstall. 2023-09-23T09:08:32.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum reinstall -y kernel || true 2023-09-23T09:08:32.655 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-09-23T09:08:32.655 INFO:teuthology.orchestra.run.smithi045.stdout:Unable to read consumer identity 2023-09-23T09:08:32.925 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:02 ago on Sat 23 Sep 2023 09:08:30 AM UTC. 2023-09-23T09:08:32.969 INFO:teuthology.orchestra.run.smithi045.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-23T09:08:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:Error: No packages marked for reinstall. 2023-09-23T09:08:33.045 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-09-23T09:08:33.128 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-09-23T09:08:33.128 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-09-23T09:08:33.128 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-09-23T09:08:33.128 DEBUG:teuthology.parallel:result is None 2023-09-23T09:08:33.128 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-23T09:08:33.136 INFO:teuthology.task.internal:Creating test directory... 2023-09-23T09:08:33.136 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T09:08:33.139 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T09:08:33.141 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T09:08:33.161 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-23T09:08:33.183 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-23T09:08:33.216 INFO:teuthology.task.internal:Creating archive directory... 2023-09-23T09:08:33.216 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T09:08:33.219 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T09:08:33.221 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T09:08:33.260 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-23T09:08:33.268 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-23T09:08:33.268 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T09:08:33.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:08:33.303 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T09:08:33.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:08:33.322 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T09:08:33.324 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T09:08:33.346 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T09:08:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T09:08:33.366 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T09:08:33.381 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T09:08:33.402 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T09:08:33.412 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T09:08:33.438 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T09:08:33.440 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-23T09:08:33.447 INFO:teuthology.task.internal:Configuring sudo... 2023-09-23T09:08:33.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T09:08:33.449 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T09:08:33.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T09:08:33.512 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-23T09:08:33.520 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-23T09:08:33.521 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T09:08:33.523 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T09:08:33.525 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T09:08:33.573 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T09:08:33.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T09:08:33.674 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T09:08:33.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T09:08:33.801 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:08:33.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T09:08:33.869 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T09:08:33.916 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T09:08:33.970 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T09:08:34.054 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T09:08:34.099 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:08:34.099 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T09:08:34.169 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T09:08:34.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T09:08:34.269 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T09:08:34.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T09:08:34.398 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:08:34.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T09:08:34.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2023-09-23T09:08:34.469 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2023-09-23T09:08:34.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2023-09-23T09:08:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T09:08:34.560 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T09:08:34.576 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T09:08:34.849 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-23T09:08:34.857 INFO:teuthology.task.internal:Starting timer... 2023-09-23T09:08:34.857 INFO:teuthology.run_tasks:Running task pcp... 2023-09-23T09:08:34.905 INFO:teuthology.run_tasks:Running task selinux... 2023-09-23T09:08:34.935 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-09-23T09:08:34.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2023-09-23T09:08:35.001 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2023-09-23T09:08:35.003 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2023-09-23T09:08:35.068 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2023-09-23T09:08:35.070 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2023-09-23T09:08:35.129 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2023-09-23T09:08:35.130 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-23T09:08:35.131 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2023-09-23T09:08:35.150 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2023-09-23T09:08:35.151 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2023-09-23T09:08:35.175 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2023-09-23T09:08:35.175 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2023-09-23T09:08:35.198 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2023-09-23T09:08:35.199 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2023-09-23T09:08:35.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T09:08:35.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:08:35.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T09:08:35.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:08:35.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T09:08:35.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:08:35.283 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-23T09:08:35.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2023-09-23T09:08:35.313 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2023-09-23T09:08:35.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2023-09-23T09:08:35.367 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-23T09:08:35.376 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-23T09:08:35.381 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-23T09:08:35.494 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-23T09:08:35.532 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-23T09:08:35.533 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi098.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-23T09:17:26.412 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2023-09-23T09:17:26.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-09-23T09:17:26.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T09:17:26.507 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-09-23T09:17:26.576 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-09-23T09:17:26.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-09-23T09:17:26.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T09:17:26.645 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-09-23T09:17:26.713 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-09-23T09:17:26.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-23T09:17:26.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T09:17:26.782 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-09-23T09:17:26.807 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-23T09:17:26.808 INFO:teuthology.run_tasks:Running task clock... 2023-09-23T09:17:26.818 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-23T09:17:26.819 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T09:17:26.819 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T09:17:26.821 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T09:17:26.822 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T09:17:26.824 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T09:17:26.824 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T09:17:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T09:17:26.855 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T09:17:26.869 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T09:17:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T09:17:26.878 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T09:17:26.892 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T09:17:26.933 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2023-09-23T09:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2023-09-23T09:17:26.945 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2023-09-23T09:17:26.952 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2023-09-23T09:17:26.960 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T09:17:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2023-09-23T09:17:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T09:17:26.974 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T09:17:26.975 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2023-09-23T09:17:26.982 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T09:17:26.989 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T09:17:27.002 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T09:17:27.055 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-09-23T09:17:27.059 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T09:17:27.059 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-09-23T09:17:27.059 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.060 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.060 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.060 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.070 INFO:teuthology.orchestra.run.smithi098.stderr:bash: ntpq: command not found 2023-09-23T09:17:27.073 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-09-23T09:17:27.073 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T09:17:27.073 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2023-09-23T09:17:27.073 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.073 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.073 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.074 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.076 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T09:17:27.076 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-09-23T09:17:27.076 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.076 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.076 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.076 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:17:27.077 INFO:teuthology.run_tasks:Running task pexec... 2023-09-23T09:17:27.086 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-23T09:17:27.086 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:17:27.087 DEBUG:teuthology.orchestra.run.smithi045:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-23T09:17:27.087 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi098.front.sepia.ceph.com 2023-09-23T09:17:27.087 DEBUG:teuthology.orchestra.run.smithi098:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-23T09:17:27.088 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi123.front.sepia.ceph.com 2023-09-23T09:17:27.088 DEBUG:teuthology.orchestra.run.smithi123:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-23T09:17:27.484 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:27.487 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:27.491 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:28.288 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:33 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:28.313 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:33 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:28.362 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:26 ago on Sat 23 Sep 2023 09:16:01 AM UTC. 2023-09-23T09:17:30.045 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-09-23T09:17:30.046 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-09-23T09:17:30.046 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2023-09-23T09:17:30.046 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-09-23T09:17:30.046 INFO:teuthology.orchestra.run.smithi123.stdout:Resetting modules: 2023-09-23T09:17:30.046 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools 2023-09-23T09:17:30.046 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:17:30.047 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-09-23T09:17:30.047 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-09-23T09:17:30.047 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:17:30.179 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-09-23T09:17:30.180 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-09-23T09:17:30.180 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2023-09-23T09:17:30.180 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-09-23T09:17:30.181 INFO:teuthology.orchestra.run.smithi098.stdout:Resetting modules: 2023-09-23T09:17:30.181 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools 2023-09-23T09:17:30.181 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:17:30.181 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-09-23T09:17:30.181 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-09-23T09:17:30.181 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:17:30.324 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-09-23T09:17:30.326 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-09-23T09:17:30.326 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-09-23T09:17:30.326 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-09-23T09:17:30.326 INFO:teuthology.orchestra.run.smithi045.stdout:Resetting modules: 2023-09-23T09:17:30.326 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools 2023-09-23T09:17:30.326 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:17:30.327 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-09-23T09:17:30.327 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-09-23T09:17:30.327 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:17:31.007 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:36 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:31.144 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:36 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:31.372 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:29 ago on Sat 23 Sep 2023 09:16:01 AM UTC. 2023-09-23T09:17:33.580 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-09-23T09:17:34.036 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:34.039 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-09-23T09:17:34.290 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-09-23T09:17:34.524 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:34.775 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:34.829 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:40 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:35.323 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:40 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:35.595 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:34 ago on Sat 23 Sep 2023 09:16:01 AM UTC. 2023-09-23T09:17:36.561 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-09-23T09:17:36.652 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================================ 2023-09-23T09:17:36.652 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-09-23T09:17:36.652 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================================ 2023-09-23T09:17:36.652 INFO:teuthology.orchestra.run.smithi123.stdout:Installing group/module packages: 2023-09-23T09:17:36.652 INFO:teuthology.orchestra.run.smithi123.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-23T09:17:36.652 INFO:teuthology.orchestra.run.smithi123.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout:Installing module profiles: 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools/common 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:17:36.653 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling module streams: 2023-09-23T09:17:36.654 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools rhel8 2023-09-23T09:17:36.654 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:17:36.654 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-09-23T09:17:36.654 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================================ 2023-09-23T09:17:36.654 INFO:teuthology.orchestra.run.smithi123.stdout:Install 10 Packages 2023-09-23T09:17:36.654 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:17:36.655 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 18 M 2023-09-23T09:17:36.655 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 65 M 2023-09-23T09:17:36.655 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-09-23T09:17:36.728 INFO:teuthology.orchestra.run.smithi123.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 558 kB/s | 41 kB 00:00 2023-09-23T09:17:36.745 INFO:teuthology.orchestra.run.smithi123.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 275 kB/s | 25 kB 00:00 2023-09-23T09:17:36.770 INFO:teuthology.orchestra.run.smithi123.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 813 kB/s | 94 kB 00:00 2023-09-23T09:17:36.795 INFO:teuthology.orchestra.run.smithi123.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 734 kB/s | 49 kB 00:00 2023-09-23T09:17:36.829 INFO:teuthology.orchestra.run.smithi123.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-23T09:17:36.854 INFO:teuthology.orchestra.run.smithi123.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-23T09:17:36.896 INFO:teuthology.orchestra.run.smithi123.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.1 MB/s | 213 kB 00:00 2023-09-23T09:17:36.988 INFO:teuthology.orchestra.run.smithi123.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-09-23T09:17:37.039 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-09-23T09:17:37.113 INFO:teuthology.orchestra.run.smithi123.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 23 MB/s | 8.0 MB 00:00 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout:Installing group/module packages: 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-23T09:17:37.132 INFO:teuthology.orchestra.run.smithi098.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout:Installing module profiles: 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools/common 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout:Enabling module streams: 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools rhel8 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-09-23T09:17:37.133 INFO:teuthology.orchestra.run.smithi098.stdout:Install 10 Packages 2023-09-23T09:17:37.134 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:17:37.134 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 18 M 2023-09-23T09:17:37.134 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 65 M 2023-09-23T09:17:37.134 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2023-09-23T09:17:37.197 INFO:teuthology.orchestra.run.smithi123.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 21 MB/s | 7.0 MB 00:00 2023-09-23T09:17:37.197 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-09-23T09:17:37.197 INFO:teuthology.orchestra.run.smithi123.stdout:Total 34 MB/s | 18 MB 00:00 2023-09-23T09:17:37.202 INFO:teuthology.orchestra.run.smithi098.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-09-23T09:17:37.219 INFO:teuthology.orchestra.run.smithi098.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 295 kB/s | 25 kB 00:00 2023-09-23T09:17:37.253 INFO:teuthology.orchestra.run.smithi098.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 345 kB/s | 41 kB 00:00 2023-09-23T09:17:37.278 INFO:teuthology.orchestra.run.smithi098.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 652 kB/s | 49 kB 00:00 2023-09-23T09:17:37.312 INFO:teuthology.orchestra.run.smithi098.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.5 MB/s | 524 kB 00:00 2023-09-23T09:17:37.337 INFO:teuthology.orchestra.run.smithi098.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-23T09:17:37.362 INFO:teuthology.orchestra.run.smithi098.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-23T09:17:37.396 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-09-23T09:17:37.415 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-09-23T09:17:37.415 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-09-23T09:17:37.479 INFO:teuthology.orchestra.run.smithi098.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-09-23T09:17:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-09-23T09:17:37.596 INFO:teuthology.orchestra.run.smithi098.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-09-23T09:17:37.597 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-09-23T09:17:37.601 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-09-23T09:17:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================================ 2023-09-23T09:17:37.674 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================================ 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:Installing group/module packages: 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-09-23T09:17:37.675 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:Installing module profiles: 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools/common 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling module streams: 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools rhel8 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================================ 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:Install 10 Packages 2023-09-23T09:17:37.676 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:17:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 18 M 2023-09-23T09:17:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 65 M 2023-09-23T09:17:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-09-23T09:17:37.688 INFO:teuthology.orchestra.run.smithi098.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-23T09:17:37.689 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2023-09-23T09:17:37.689 INFO:teuthology.orchestra.run.smithi098.stdout:Total 33 MB/s | 18 MB 00:00 2023-09-23T09:17:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-09-23T09:17:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 276 kB/s | 25 kB 00:00 2023-09-23T09:17:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 354 kB/s | 41 kB 00:00 2023-09-23T09:17:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-09-23T09:17:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-23T09:17:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-23T09:17:37.889 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2023-09-23T09:17:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-23T09:17:37.908 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2023-09-23T09:17:37.908 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2023-09-23T09:17:38.002 INFO:teuthology.orchestra.run.smithi045.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-09-23T09:17:38.089 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2023-09-23T09:17:38.093 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2023-09-23T09:17:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-09-23T09:17:38.140 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-09-23T09:17:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-09-23T09:17:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-09-23T09:17:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:Total 34 MB/s | 18 MB 00:00 2023-09-23T09:17:38.358 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-23T09:17:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-09-23T09:17:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-09-23T09:17:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-09-23T09:17:38.521 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T09:17:38.600 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2023-09-23T09:17:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-09-23T09:17:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-09-23T09:17:38.698 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-23T09:17:38.801 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-23T09:17:38.984 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-23T09:17:39.001 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T09:17:39.184 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-09-23T09:17:39.215 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-23T09:17:39.332 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-23T09:17:39.379 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-23T09:17:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-23T09:17:39.552 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T09:17:39.757 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-23T09:17:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-23T09:17:39.981 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-23T09:17:40.101 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-23T09:17:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-23T09:17:40.651 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-23T09:17:40.984 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-23T09:17:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-23T09:17:41.225 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-23T09:17:41.411 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-23T09:17:41.550 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-23T09:17:41.605 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T09:17:41.803 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-23T09:17:41.905 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-23T09:17:41.906 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-23T09:17:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-23T09:17:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-23T09:17:42.201 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T09:17:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-23T09:17:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-23T09:17:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-23T09:17:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-23T09:17:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-23T09:17:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T09:17:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T09:17:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-23T09:17:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T09:17:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-23T09:17:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-23T09:17:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-23T09:17:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-23T09:17:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-23T09:17:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-23T09:17:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-23T09:17:43.020 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-23T09:17:43.021 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:48 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:43.648 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-23T09:17:43.648 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:49 ago on Sat 23 Sep 2023 09:15:54 AM UTC. 2023-09-23T09:17:44.081 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-23T09:17:44.082 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:42 ago on Sat 23 Sep 2023 09:16:01 AM UTC. 2023-09-23T09:17:45.557 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-09-23T09:17:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T09:17:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-23T09:17:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:17:45.744 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-09-23T09:17:45.879 DEBUG:teuthology.parallel:result is None 2023-09-23T09:17:46.191 INFO:teuthology.orchestra.run.smithi098.stdout:Installed products updated. 2023-09-23T09:17:46.379 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:17:46.379 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2023-09-23T09:17:46.379 INFO:teuthology.orchestra.run.smithi098.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:46.379 INFO:teuthology.orchestra.run.smithi098.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T09:17:46.379 INFO:teuthology.orchestra.run.smithi098.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:46.379 INFO:teuthology.orchestra.run.smithi098.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:17:46.380 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-09-23T09:17:46.527 DEBUG:teuthology.parallel:result is None 2023-09-23T09:17:46.854 INFO:teuthology.orchestra.run.smithi045.stdout:Installed products updated. 2023-09-23T09:17:47.073 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:17:47.073 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-09-23T09:17:47.073 INFO:teuthology.orchestra.run.smithi045.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:47.073 INFO:teuthology.orchestra.run.smithi045.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T09:17:47.073 INFO:teuthology.orchestra.run.smithi045.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:17:47.074 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-09-23T09:17:47.226 DEBUG:teuthology.parallel:result is None 2023-09-23T09:17:47.227 INFO:teuthology.run_tasks:Running task install... 2023-09-23T09:17:47.235 DEBUG:teuthology.task.install:project ceph 2023-09-23T09:17:47.235 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'}} 2023-09-23T09:17:47.235 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'} 2023-09-23T09:17:47.235 INFO:teuthology.task.install:Using flavor: default 2023-09-23T09:17:47.240 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-23T09:17:47.241 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-23T09:17:47.241 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T09:17:47.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:17:47.243 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T09:17:47.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:17:47.244 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T09:17:47.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:17:47.395 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/ 2023-09-23T09:17:47.396 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T09:17:47.490 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/ 2023-09-23T09:17:47.491 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T09:17:47.495 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/ 2023-09-23T09:17:47.495 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T09:17:47.576 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T09:17:47.576 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:17:47.576 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T09:17:47.600 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T09:17:47.600 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:17:47.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T09:17:47.608 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T09:17:47.609 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T09:17:47.632 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T09:17:47.633 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T09:17:47.665 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T09:17:47.665 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:17:47.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T09:17:47.679 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T09:17:47.693 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T09:17:47.693 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T09:17:47.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T09:17:47.759 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T09:17:47.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T09:17:47.790 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T09:17:47.828 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2023-09-23T09:17:47.829 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2023-09-23T09:17:47.840 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T09:17:47.863 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2023-09-23T09:17:47.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2023-09-23T09:17:47.905 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2023-09-23T09:17:47.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2023-09-23T09:17:48.204 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:48.250 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:48.299 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:48.497 INFO:teuthology.orchestra.run.smithi098.stdout:44 files removed 2023-09-23T09:17:48.514 INFO:teuthology.orchestra.run.smithi123.stdout:44 files removed 2023-09-23T09:17:48.543 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T09:17:48.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T09:17:48.576 INFO:teuthology.orchestra.run.smithi045.stdout:44 files removed 2023-09-23T09:17:48.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T09:17:48.874 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:48.895 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:48.978 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-09-23T09:17:49.533 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 314 kB/s | 81 kB 00:00 2023-09-23T09:17:49.705 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 271 kB/s | 81 kB 00:00 2023-09-23T09:17:49.736 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 170 kB/s | 81 kB 00:00 2023-09-23T09:17:49.866 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 31 kB/s | 9.4 kB 00:00 2023-09-23T09:17:50.045 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 34 kB/s | 9.4 kB 00:00 2023-09-23T09:17:50.097 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-09-23T09:17:50.150 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2023-09-23T09:17:50.318 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-09-23T09:17:50.354 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-09-23T09:17:50.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2023-09-23T09:17:50.488 INFO:teuthology.orchestra.run.smithi098.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-09-23T09:17:50.515 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-09-23T09:17:51.121 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-09-23T09:17:52.143 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 16 MB 00:01 2023-09-23T09:17:52.489 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 16 MB 00:01 2023-09-23T09:17:54.619 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 398 kB/s | 24 kB 00:00 2023-09-23T09:17:55.520 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-09-23T09:17:56.006 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 493 kB/s | 24 kB 00:00 2023-09-23T09:17:56.115 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 310 kB/s | 24 kB 00:00 2023-09-23T09:17:56.924 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-09-23T09:17:57.026 INFO:teuthology.orchestra.run.smithi045.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-09-23T09:18:03.542 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-09-23T09:18:05.034 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-09-23T09:18:05.827 INFO:teuthology.orchestra.run.smithi045.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-09-23T09:18:14.539 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T09:18:14.539 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T09:18:14.554 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-23T09:18:14.555 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-23T09:18:14.556 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-23T09:18:14.558 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-23T09:18:14.560 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-23T09:18:14.563 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-23T09:18:14.564 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-23T09:18:14.698 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.4 k 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el8 ceph 5.4 M 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 912 k 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el8 ceph 167 k 2023-09-23T09:18:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el8 ceph 1.6 M 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 132 k 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 1.7 M 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 7.4 M 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 47 k 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 14 M 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el8 ceph 49 M 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 261 k 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 181 k 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 260 k 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 18 k 2023-09-23T09:18:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 31 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 832 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 139 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-23T09:18:14.713 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el8 ceph 197 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el8 ceph 371 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 364 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 107 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 94 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.6 M 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 180 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-23T09:18:14.714 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 23 M 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 23 k 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el8 ceph 2.3 M 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 242 k 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el8 ceph 4.6 M 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el8 ceph 18 M 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 15 k 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el8 ceph 24 k 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-23T09:18:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el8 ceph 184 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-23T09:18:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 519 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.9 M 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-23T09:18:14.717 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el8 ceph 45 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 125 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-23T09:18:14.718 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-23T09:18:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-23T09:18:14.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-23T09:18:14.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling module streams: 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql 10 2023-09-23T09:18:14.722 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:18:14.723 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-09-23T09:18:14.723 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-09-23T09:18:14.723 INFO:teuthology.orchestra.run.smithi123.stdout:Install 128 Packages 2023-09-23T09:18:14.723 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 3 Packages 2023-09-23T09:18:14.723 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:18:14.723 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 209 M 2023-09-23T09:18:14.723 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-09-23T09:18:14.930 INFO:teuthology.orchestra.run.smithi123.stdout:(1/131): ceph-18.0.0-6336.g86266391.el8.x86_64. 38 kB/s | 6.4 kB 00:00 2023-09-23T09:18:15.488 INFO:teuthology.orchestra.run.smithi123.stdout:(2/131): ceph-base-18.0.0-6336.g86266391.el8.x8 7.4 MB/s | 5.4 MB 00:00 2023-09-23T09:18:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:(3/131): ceph-fuse-18.0.0-6336.g86266391.el8.x8 1.3 MB/s | 912 kB 00:00 2023-09-23T09:18:15.656 INFO:teuthology.orchestra.run.smithi123.stdout:(4/131): ceph-immutable-object-cache-18.0.0-633 992 kB/s | 167 kB 00:00 2023-09-23T09:18:15.831 INFO:teuthology.orchestra.run.smithi123.stdout:(5/131): ceph-mgr-18.0.0-6336.g86266391.el8.x86 9.3 MB/s | 1.6 MB 00:00 2023-09-23T09:18:16.024 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T09:18:16.025 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T09:18:16.039 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-23T09:18:16.041 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-23T09:18:16.042 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-23T09:18:16.043 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-23T09:18:16.045 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-23T09:18:16.048 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-23T09:18:16.050 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-23T09:18:16.183 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-09-23T09:18:16.195 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-09-23T09:18:16.195 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2023-09-23T09:18:16.195 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-09-23T09:18:16.195 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2023-09-23T09:18:16.195 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-23T09:18:16.195 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-23T09:18:16.195 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.4 k 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el8 ceph 5.4 M 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 912 k 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el8 ceph 167 k 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el8 ceph 1.6 M 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 132 k 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 1.7 M 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 7.4 M 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 47 k 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 14 M 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el8 ceph 49 M 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 261 k 2023-09-23T09:18:16.196 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 181 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 260 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 18 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 31 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 832 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 139 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-23T09:18:16.197 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el8 ceph 197 k 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el8 ceph 371 k 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 364 k 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 107 k 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 94 k 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.6 M 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 180 k 2023-09-23T09:18:16.198 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 23 M 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 23 k 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el8 ceph 2.3 M 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 242 k 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el8 ceph 4.6 M 2023-09-23T09:18:16.199 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el8 ceph 18 M 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 15 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el8 ceph 24 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el8 ceph 184 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-23T09:18:16.200 INFO:teuthology.orchestra.run.smithi098.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 519 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.9 M 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-23T09:18:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el8 ceph 45 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 125 k 2023-09-23T09:18:16.202 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T09:18:16.203 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-23T09:18:16.204 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-23T09:18:16.205 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-23T09:18:16.206 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout:Enabling module streams: 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql 10 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout:Install 128 Packages 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 3 Packages 2023-09-23T09:18:16.207 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:18:16.208 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 209 M 2023-09-23T09:18:16.208 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2023-09-23T09:18:16.332 INFO:teuthology.orchestra.run.smithi123.stdout:(6/131): ceph-mon-18.0.0-6336.g86266391.el8.x86 9.2 MB/s | 4.6 MB 00:00 2023-09-23T09:18:16.515 INFO:teuthology.orchestra.run.smithi123.stdout:(7/131): ceph-mds-18.0.0-6336.g86266391.el8.x86 2.5 MB/s | 2.3 MB 00:00 2023-09-23T09:18:16.660 INFO:teuthology.orchestra.run.smithi098.stdout:(1/131): ceph-18.0.0-6336.g86266391.el8.x86_64. 48 kB/s | 6.4 kB 00:00 2023-09-23T09:18:16.724 INFO:teuthology.orchestra.run.smithi123.stdout:(8/131): ceph-common-18.0.0-6336.g86266391.el8. 12 MB/s | 23 MB 00:01 2023-09-23T09:18:16.816 INFO:teuthology.orchestra.run.smithi123.stdout:(9/131): ceph-selinux-18.0.0-6336.g86266391.el8 266 kB/s | 24 kB 00:00 2023-09-23T09:18:17.123 INFO:teuthology.orchestra.run.smithi098.stdout:(2/131): ceph-fuse-18.0.0-6336.g86266391.el8.x8 1.9 MB/s | 912 kB 00:00 2023-09-23T09:18:17.269 INFO:teuthology.orchestra.run.smithi098.stdout:(3/131): ceph-immutable-object-cache-18.0.0-633 1.1 MB/s | 167 kB 00:00 2023-09-23T09:18:17.919 INFO:teuthology.orchestra.run.smithi098.stdout:(4/131): ceph-base-18.0.0-6336.g86266391.el8.x8 3.9 MB/s | 5.4 MB 00:01 2023-09-23T09:18:18.082 INFO:teuthology.orchestra.run.smithi123.stdout:(10/131): ceph-osd-18.0.0-6336.g86266391.el8.x8 10 MB/s | 18 MB 00:01 2023-09-23T09:18:18.111 INFO:teuthology.orchestra.run.smithi098.stdout:(5/131): ceph-mgr-18.0.0-6336.g86266391.el8.x86 8.5 MB/s | 1.6 MB 00:00 2023-09-23T09:18:18.141 INFO:teuthology.orchestra.run.smithi123.stdout:(11/131): cephfs-mirror-18.0.0-6336.g86266391.e 4.4 MB/s | 260 kB 00:00 2023-09-23T09:18:18.156 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T09:18:18.157 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T09:18:18.172 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-23T09:18:18.173 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-23T09:18:18.174 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-23T09:18:18.175 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-23T09:18:18.178 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-23T09:18:18.181 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-23T09:18:18.182 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-23T09:18:18.212 INFO:teuthology.orchestra.run.smithi098.stdout:(6/131): ceph-mds-18.0.0-6336.g86266391.el8.x86 2.5 MB/s | 2.3 MB 00:00 2023-09-23T09:18:18.216 INFO:teuthology.orchestra.run.smithi123.stdout:(12/131): libcephfs-devel-18.0.0-6336.g86266391 416 kB/s | 31 kB 00:00 2023-09-23T09:18:18.326 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-09-23T09:18:18.339 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-09-23T09:18:18.339 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-09-23T09:18:18.340 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-09-23T09:18:18.340 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-09-23T09:18:18.340 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-23T09:18:18.340 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-23T09:18:18.340 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.4 k 2023-09-23T09:18:18.340 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el8 ceph 5.4 M 2023-09-23T09:18:18.340 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 912 k 2023-09-23T09:18:18.341 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el8 ceph 167 k 2023-09-23T09:18:18.341 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el8 ceph 1.6 M 2023-09-23T09:18:18.341 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 132 k 2023-09-23T09:18:18.341 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 1.7 M 2023-09-23T09:18:18.341 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 7.4 M 2023-09-23T09:18:18.341 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 47 k 2023-09-23T09:18:18.341 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 14 M 2023-09-23T09:18:18.342 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el8 ceph 49 M 2023-09-23T09:18:18.342 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 261 k 2023-09-23T09:18:18.342 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 181 k 2023-09-23T09:18:18.342 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 260 k 2023-09-23T09:18:18.342 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 18 k 2023-09-23T09:18:18.342 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-23T09:18:18.342 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-23T09:18:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-23T09:18:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 31 k 2023-09-23T09:18:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 832 k 2023-09-23T09:18:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 139 k 2023-09-23T09:18:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-23T09:18:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-23T09:18:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el8 ceph 197 k 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el8 ceph 371 k 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 364 k 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 107 k 2023-09-23T09:18:18.344 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 94 k 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.6 M 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 180 k 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T09:18:18.345 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-09-23T09:18:18.346 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-23T09:18:18.346 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 23 M 2023-09-23T09:18:18.346 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 23 k 2023-09-23T09:18:18.346 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el8 ceph 2.3 M 2023-09-23T09:18:18.346 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 242 k 2023-09-23T09:18:18.346 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el8 ceph 4.6 M 2023-09-23T09:18:18.346 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el8 ceph 18 M 2023-09-23T09:18:18.347 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 15 k 2023-09-23T09:18:18.347 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el8 ceph 24 k 2023-09-23T09:18:18.347 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-23T09:18:18.347 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-23T09:18:18.347 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-23T09:18:18.347 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-23T09:18:18.347 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-23T09:18:18.348 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el8 ceph 184 k 2023-09-23T09:18:18.348 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-23T09:18:18.348 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-23T09:18:18.348 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-23T09:18:18.348 INFO:teuthology.orchestra.run.smithi045.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-23T09:18:18.348 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-23T09:18:18.348 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-23T09:18:18.349 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 519 k 2023-09-23T09:18:18.349 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-23T09:18:18.349 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.9 M 2023-09-23T09:18:18.349 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-23T09:18:18.349 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-23T09:18:18.349 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-23T09:18:18.350 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-23T09:18:18.350 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-23T09:18:18.350 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-23T09:18:18.350 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-23T09:18:18.350 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-23T09:18:18.350 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-23T09:18:18.351 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-23T09:18:18.351 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-23T09:18:18.351 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-23T09:18:18.351 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-23T09:18:18.351 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-23T09:18:18.351 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-23T09:18:18.351 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-23T09:18:18.352 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el8 ceph 45 k 2023-09-23T09:18:18.352 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 125 k 2023-09-23T09:18:18.352 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-23T09:18:18.352 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-23T09:18:18.352 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-23T09:18:18.352 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-23T09:18:18.352 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-23T09:18:18.353 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-23T09:18:18.353 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-23T09:18:18.353 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-23T09:18:18.353 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-23T09:18:18.353 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-23T09:18:18.353 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-23T09:18:18.354 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T09:18:18.354 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-23T09:18:18.354 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-23T09:18:18.354 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-23T09:18:18.354 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-23T09:18:18.354 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-23T09:18:18.355 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-23T09:18:18.355 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-23T09:18:18.355 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-23T09:18:18.355 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-23T09:18:18.355 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-23T09:18:18.355 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-23T09:18:18.356 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-23T09:18:18.356 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-23T09:18:18.356 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-23T09:18:18.356 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-23T09:18:18.356 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-23T09:18:18.356 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-23T09:18:18.356 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-23T09:18:18.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T09:18:18.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-23T09:18:18.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-23T09:18:18.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-23T09:18:18.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-23T09:18:18.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-23T09:18:18.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-23T09:18:18.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-23T09:18:18.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-23T09:18:18.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-23T09:18:18.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-23T09:18:18.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-23T09:18:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-23T09:18:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-23T09:18:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-23T09:18:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-23T09:18:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-23T09:18:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-23T09:18:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-23T09:18:18.360 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling module streams: 2023-09-23T09:18:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql 10 2023-09-23T09:18:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:18:18.360 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-09-23T09:18:18.360 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-09-23T09:18:18.360 INFO:teuthology.orchestra.run.smithi045.stdout:Install 128 Packages 2023-09-23T09:18:18.361 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 3 Packages 2023-09-23T09:18:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:18:18.361 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 209 M 2023-09-23T09:18:18.362 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-09-23T09:18:18.391 INFO:teuthology.orchestra.run.smithi123.stdout:(13/131): libcephfs2-18.0.0-6336.g86266391.el8. 4.6 MB/s | 832 kB 00:00 2023-09-23T09:18:18.475 INFO:teuthology.orchestra.run.smithi123.stdout:(14/131): libcephsqlite-18.0.0-6336.g86266391.e 2.2 MB/s | 184 kB 00:00 2023-09-23T09:18:18.559 INFO:teuthology.orchestra.run.smithi123.stdout:(15/131): librados-devel-18.0.0-6336.g86266391. 1.6 MB/s | 139 kB 00:00 2023-09-23T09:18:18.651 INFO:teuthology.orchestra.run.smithi123.stdout:(16/131): libradosstriper1-18.0.0-6336.g8626639 5.5 MB/s | 519 kB 00:00 2023-09-23T09:18:18.660 INFO:teuthology.orchestra.run.smithi045.stdout:(1/131): ceph-18.0.0-6336.g86266391.el8.x86_64. 26 kB/s | 6.4 kB 00:00 2023-09-23T09:18:18.670 INFO:teuthology.orchestra.run.smithi098.stdout:(7/131): ceph-mon-18.0.0-6336.g86266391.el8.x86 8.3 MB/s | 4.6 MB 00:00 2023-09-23T09:18:19.151 INFO:teuthology.orchestra.run.smithi123.stdout:(17/131): ceph-radosgw-18.0.0-6336.g86266391.el 5.3 MB/s | 14 MB 00:02 2023-09-23T09:18:19.251 INFO:teuthology.orchestra.run.smithi123.stdout:(18/131): python3-ceph-argparse-18.0.0-6336.g86 448 kB/s | 45 kB 00:00 2023-09-23T09:18:19.352 INFO:teuthology.orchestra.run.smithi123.stdout:(19/131): python3-ceph-common-18.0.0-6336.g8626 1.2 MB/s | 125 kB 00:00 2023-09-23T09:18:19.460 INFO:teuthology.orchestra.run.smithi123.stdout:(20/131): python3-cephfs-18.0.0-6336.g86266391. 1.8 MB/s | 197 kB 00:00 2023-09-23T09:18:19.577 INFO:teuthology.orchestra.run.smithi123.stdout:(21/131): python3-rados-18.0.0-6336.g86266391.e 3.1 MB/s | 371 kB 00:00 2023-09-23T09:18:19.694 INFO:teuthology.orchestra.run.smithi123.stdout:(22/131): python3-rbd-18.0.0-6336.g86266391.el8 3.0 MB/s | 364 kB 00:00 2023-09-23T09:18:19.803 INFO:teuthology.orchestra.run.smithi123.stdout:(23/131): python3-rgw-18.0.0-6336.g86266391.el8 987 kB/s | 107 kB 00:00 2023-09-23T09:18:19.912 INFO:teuthology.orchestra.run.smithi123.stdout:(24/131): rbd-fuse-18.0.0-6336.g86266391.el8.x8 865 kB/s | 94 kB 00:00 2023-09-23T09:18:19.918 INFO:teuthology.orchestra.run.smithi045.stdout:(2/131): ceph-fuse-18.0.0-6336.g86266391.el8.x8 725 kB/s | 912 kB 00:01 2023-09-23T09:18:20.169 INFO:teuthology.orchestra.run.smithi045.stdout:(3/131): ceph-immutable-object-cache-18.0.0-633 668 kB/s | 167 kB 00:00 2023-09-23T09:18:20.462 INFO:teuthology.orchestra.run.smithi123.stdout:(25/131): librgw2-18.0.0-6336.g86266391.el8.x86 3.8 MB/s | 6.9 MB 00:01 2023-09-23T09:18:20.528 INFO:teuthology.orchestra.run.smithi123.stdout:(26/131): rbd-nbd-18.0.0-6336.g86266391.el8.x86 2.6 MB/s | 180 kB 00:00 2023-09-23T09:18:20.595 INFO:teuthology.orchestra.run.smithi098.stdout:(8/131): ceph-radosgw-18.0.0-6336.g86266391.el8 7.2 MB/s | 14 MB 00:01 2023-09-23T09:18:20.596 INFO:teuthology.orchestra.run.smithi123.stdout:(27/131): rbd-mirror-18.0.0-6336.g86266391.el8. 5.2 MB/s | 3.6 MB 00:00 2023-09-23T09:18:20.612 INFO:teuthology.orchestra.run.smithi123.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6336.g 276 kB/s | 23 kB 00:00 2023-09-23T09:18:20.671 INFO:teuthology.orchestra.run.smithi123.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.7 MB/s | 132 kB 00:00 2023-09-23T09:18:20.679 INFO:teuthology.orchestra.run.smithi098.stdout:(9/131): ceph-selinux-18.0.0-6336.g86266391.el8 293 kB/s | 24 kB 00:00 2023-09-23T09:18:21.196 INFO:teuthology.orchestra.run.smithi123.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6336.g86266 3.0 MB/s | 1.7 MB 00:00 2023-09-23T09:18:21.355 INFO:teuthology.orchestra.run.smithi123.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6336.g86 1.5 MB/s | 242 kB 00:00 2023-09-23T09:18:21.447 INFO:teuthology.orchestra.run.smithi123.stdout:(32/131): ceph-mgr-rook-18.0.0-6336.g86266391.e 518 kB/s | 47 kB 00:00 2023-09-23T09:18:21.531 INFO:teuthology.orchestra.run.smithi123.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6336.g8 174 kB/s | 15 kB 00:00 2023-09-23T09:18:21.697 INFO:teuthology.orchestra.run.smithi123.stdout:(34/131): ceph-volume-18.0.0-6336.g86266391.el8 1.5 MB/s | 261 kB 00:00 2023-09-23T09:18:21.798 INFO:teuthology.orchestra.run.smithi123.stdout:(35/131): cephadm-18.0.0-6336.g86266391.el8.noa 1.8 MB/s | 181 kB 00:00 2023-09-23T09:18:21.882 INFO:teuthology.orchestra.run.smithi123.stdout:(36/131): cephfs-top-18.0.0-6336.g86266391.el8. 217 kB/s | 18 kB 00:00 2023-09-23T09:18:21.885 INFO:teuthology.orchestra.run.smithi045.stdout:(4/131): ceph-base-18.0.0-6336.g86266391.el8.x8 1.6 MB/s | 5.4 MB 00:03 2023-09-23T09:18:21.989 INFO:teuthology.orchestra.run.smithi123.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 5.6 MB/s | 7.4 MB 00:01 2023-09-23T09:18:22.181 INFO:teuthology.orchestra.run.smithi045.stdout:(5/131): ceph-mds-18.0.0-6336.g86266391.el8.x86 1.1 MB/s | 2.3 MB 00:02 2023-09-23T09:18:22.321 INFO:teuthology.orchestra.run.smithi098.stdout:(10/131): ceph-common-18.0.0-6336.g86266391.el8 3.9 MB/s | 23 MB 00:05 2023-09-23T09:18:22.461 INFO:teuthology.orchestra.run.smithi045.stdout:(6/131): ceph-mgr-18.0.0-6336.g86266391.el8.x86 2.8 MB/s | 1.6 MB 00:00 2023-09-23T09:18:22.463 INFO:teuthology.orchestra.run.smithi098.stdout:(11/131): cephfs-mirror-18.0.0-6336.g86266391.e 1.8 MB/s | 260 kB 00:00 2023-09-23T09:18:22.515 INFO:teuthology.orchestra.run.smithi123.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 112 kB/s | 59 kB 00:00 2023-09-23T09:18:22.555 INFO:teuthology.orchestra.run.smithi098.stdout:(12/131): libcephfs-devel-18.0.0-6336.g86266391 339 kB/s | 31 kB 00:00 2023-09-23T09:18:22.699 INFO:teuthology.orchestra.run.smithi123.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 410 kB/s | 75 kB 00:00 2023-09-23T09:18:22.724 INFO:teuthology.orchestra.run.smithi123.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 363 kB/s | 306 kB 00:00 2023-09-23T09:18:22.730 INFO:teuthology.orchestra.run.smithi098.stdout:(13/131): libcephfs2-18.0.0-6336.g86266391.el8. 4.6 MB/s | 832 kB 00:00 2023-09-23T09:18:22.830 INFO:teuthology.orchestra.run.smithi098.stdout:(14/131): libcephsqlite-18.0.0-6336.g86266391.e 1.8 MB/s | 184 kB 00:00 2023-09-23T09:18:22.858 INFO:teuthology.orchestra.run.smithi123.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2023-09-23T09:18:22.914 INFO:teuthology.orchestra.run.smithi098.stdout:(15/131): librados-devel-18.0.0-6336.g86266391. 1.6 MB/s | 139 kB 00:00 2023-09-23T09:18:23.016 INFO:teuthology.orchestra.run.smithi123.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-09-23T09:18:23.056 INFO:teuthology.orchestra.run.smithi098.stdout:(16/131): libradosstriper1-18.0.0-6336.g8626639 3.6 MB/s | 519 kB 00:00 2023-09-23T09:18:23.067 INFO:teuthology.orchestra.run.smithi123.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2023-09-23T09:18:23.151 INFO:teuthology.orchestra.run.smithi123.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 246 kB/s | 33 kB 00:00 2023-09-23T09:18:23.193 INFO:teuthology.orchestra.run.smithi123.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-09-23T09:18:23.310 INFO:teuthology.orchestra.run.smithi123.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-09-23T09:18:23.360 INFO:teuthology.orchestra.run.smithi123.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.3 MB/s | 384 kB 00:00 2023-09-23T09:18:23.452 INFO:teuthology.orchestra.run.smithi123.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 573 kB/s | 81 kB 00:00 2023-09-23T09:18:23.477 INFO:teuthology.orchestra.run.smithi123.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-09-23T09:18:23.577 INFO:teuthology.orchestra.run.smithi123.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-09-23T09:18:23.728 INFO:teuthology.orchestra.run.smithi123.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 325 kB/s | 49 kB 00:00 2023-09-23T09:18:23.761 INFO:teuthology.orchestra.run.smithi123.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2023-09-23T09:18:23.862 INFO:teuthology.orchestra.run.smithi123.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 439 kB/s | 59 kB 00:00 2023-09-23T09:18:23.887 INFO:teuthology.orchestra.run.smithi123.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 481 kB/s | 60 kB 00:00 2023-09-23T09:18:24.004 INFO:teuthology.orchestra.run.smithi123.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 134 kB/s | 16 kB 00:00 2023-09-23T09:18:24.046 INFO:teuthology.orchestra.run.smithi123.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2023-09-23T09:18:24.064 INFO:teuthology.orchestra.run.smithi098.stdout:(17/131): ceph-osd-18.0.0-6336.g86266391.el8.x8 3.1 MB/s | 18 MB 00:05 2023-09-23T09:18:24.129 INFO:teuthology.orchestra.run.smithi123.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 268 kB/s | 34 kB 00:00 2023-09-23T09:18:24.157 INFO:teuthology.orchestra.run.smithi098.stdout:(18/131): librgw2-18.0.0-6336.g86266391.el8.x86 6.3 MB/s | 6.9 MB 00:01 2023-09-23T09:18:24.182 INFO:teuthology.orchestra.run.smithi098.stdout:(19/131): python3-ceph-argparse-18.0.0-6336.g86 383 kB/s | 45 kB 00:00 2023-09-23T09:18:24.204 INFO:teuthology.orchestra.run.smithi123.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-09-23T09:18:24.240 INFO:teuthology.orchestra.run.smithi098.stdout:(20/131): python3-ceph-common-18.0.0-6336.g8626 1.5 MB/s | 125 kB 00:00 2023-09-23T09:18:24.244 INFO:teuthology.orchestra.run.smithi045.stdout:(7/131): ceph-mon-18.0.0-6336.g86266391.el8.x86 2.2 MB/s | 4.6 MB 00:02 2023-09-23T09:18:24.246 INFO:teuthology.orchestra.run.smithi123.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 549 kB/s | 64 kB 00:00 2023-09-23T09:18:24.266 INFO:teuthology.orchestra.run.smithi098.stdout:(21/131): python3-cephfs-18.0.0-6336.g86266391. 2.3 MB/s | 197 kB 00:00 2023-09-23T09:18:24.308 INFO:teuthology.orchestra.run.smithi098.stdout:(22/131): python3-rados-18.0.0-6336.g86266391.e 5.4 MB/s | 371 kB 00:00 2023-09-23T09:18:24.330 INFO:teuthology.orchestra.run.smithi123.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 149 kB/s | 19 kB 00:00 2023-09-23T09:18:24.364 INFO:teuthology.orchestra.run.smithi123.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-09-23T09:18:24.384 INFO:teuthology.orchestra.run.smithi098.stdout:(23/131): python3-rbd-18.0.0-6336.g86266391.el8 3.0 MB/s | 364 kB 00:00 2023-09-23T09:18:24.409 INFO:teuthology.orchestra.run.smithi098.stdout:(24/131): python3-rgw-18.0.0-6336.g86266391.el8 1.0 MB/s | 107 kB 00:00 2023-09-23T09:18:24.451 INFO:teuthology.orchestra.run.smithi098.stdout:(25/131): rbd-fuse-18.0.0-6336.g86266391.el8.x8 1.4 MB/s | 94 kB 00:00 2023-09-23T09:18:24.456 INFO:teuthology.orchestra.run.smithi123.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-09-23T09:18:24.481 INFO:teuthology.orchestra.run.smithi123.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-09-23T09:18:24.517 INFO:teuthology.orchestra.run.smithi098.stdout:(26/131): rbd-nbd-18.0.0-6336.g86266391.el8.x86 2.6 MB/s | 180 kB 00:00 2023-09-23T09:18:24.593 INFO:teuthology.orchestra.run.smithi098.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6336.g 307 kB/s | 23 kB 00:00 2023-09-23T09:18:24.614 INFO:teuthology.orchestra.run.smithi123.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2023-09-23T09:18:24.631 INFO:teuthology.orchestra.run.smithi123.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2023-09-23T09:18:24.668 INFO:teuthology.orchestra.run.smithi098.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.7 MB/s | 132 kB 00:00 2023-09-23T09:18:24.732 INFO:teuthology.orchestra.run.smithi123.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 523 kB/s | 61 kB 00:00 2023-09-23T09:18:24.765 INFO:teuthology.orchestra.run.smithi123.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 639 kB/s | 85 kB 00:00 2023-09-23T09:18:24.874 INFO:teuthology.orchestra.run.smithi123.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 162 kB/s | 23 kB 00:00 2023-09-23T09:18:24.899 INFO:teuthology.orchestra.run.smithi123.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-09-23T09:18:24.958 INFO:teuthology.orchestra.run.smithi123.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 807 kB/s | 47 kB 00:00 2023-09-23T09:18:24.960 INFO:teuthology.orchestra.run.smithi098.stdout:(29/131): rbd-mirror-18.0.0-6336.g86266391.el8. 6.5 MB/s | 3.6 MB 00:00 2023-09-23T09:18:25.000 INFO:teuthology.orchestra.run.smithi123.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-09-23T09:18:25.042 INFO:teuthology.orchestra.run.smithi123.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-09-23T09:18:25.092 INFO:teuthology.orchestra.run.smithi123.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-23T09:18:25.134 INFO:teuthology.orchestra.run.smithi123.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-09-23T09:18:25.168 INFO:teuthology.orchestra.run.smithi123.stdout:(75/131): thrift-0.13.0-2.el8.x86_64.rpm 4.3 MB/s | 1.7 MB 00:00 2023-09-23T09:18:25.185 INFO:teuthology.orchestra.run.smithi123.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-23T09:18:25.219 INFO:teuthology.orchestra.run.smithi098.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6336.g86266 3.2 MB/s | 1.7 MB 00:00 2023-09-23T09:18:25.235 INFO:teuthology.orchestra.run.smithi123.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 639 kB/s | 43 kB 00:00 2023-09-23T09:18:25.260 INFO:teuthology.orchestra.run.smithi123.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.5 MB/s | 109 kB 00:00 2023-09-23T09:18:25.277 INFO:teuthology.orchestra.run.smithi123.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-09-23T09:18:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-09-23T09:18:25.345 INFO:teuthology.orchestra.run.smithi123.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 279 kB/s | 19 kB 00:00 2023-09-23T09:18:25.362 INFO:teuthology.orchestra.run.smithi123.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-09-23T09:18:25.378 INFO:teuthology.orchestra.run.smithi098.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6336.g86 1.5 MB/s | 242 kB 00:00 2023-09-23T09:18:25.430 INFO:teuthology.orchestra.run.smithi123.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 6.4 MB/s | 559 kB 00:00 2023-09-23T09:18:25.454 INFO:teuthology.orchestra.run.smithi123.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 195 kB/s | 18 kB 00:00 2023-09-23T09:18:25.461 INFO:teuthology.orchestra.run.smithi098.stdout:(32/131): ceph-mgr-rook-18.0.0-6336.g86266391.e 572 kB/s | 47 kB 00:00 2023-09-23T09:18:25.488 INFO:teuthology.orchestra.run.smithi123.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-09-23T09:18:25.513 INFO:teuthology.orchestra.run.smithi123.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-09-23T09:18:25.536 INFO:teuthology.orchestra.run.smithi098.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6336.g8 194 kB/s | 15 kB 00:00 2023-09-23T09:18:25.580 INFO:teuthology.orchestra.run.smithi123.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 2.3 MB 00:00 2023-09-23T09:18:25.605 INFO:teuthology.orchestra.run.smithi123.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 786 kB/s | 72 kB 00:00 2023-09-23T09:18:25.630 INFO:teuthology.orchestra.run.smithi123.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-23T09:18:25.656 INFO:teuthology.orchestra.run.smithi123.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-09-23T09:18:25.670 INFO:teuthology.orchestra.run.smithi098.stdout:(34/131): ceph-volume-18.0.0-6336.g86266391.el8 1.9 MB/s | 261 kB 00:00 2023-09-23T09:18:25.681 INFO:teuthology.orchestra.run.smithi123.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-23T09:18:25.706 INFO:teuthology.orchestra.run.smithi123.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-23T09:18:25.732 INFO:teuthology.orchestra.run.smithi123.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-09-23T09:18:25.753 INFO:teuthology.orchestra.run.smithi098.stdout:(35/131): cephadm-18.0.0-6336.g86266391.el8.noa 2.1 MB/s | 181 kB 00:00 2023-09-23T09:18:25.765 INFO:teuthology.orchestra.run.smithi123.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-09-23T09:18:25.790 INFO:teuthology.orchestra.run.smithi123.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-09-23T09:18:25.808 INFO:teuthology.orchestra.run.smithi123.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-09-23T09:18:25.858 INFO:teuthology.orchestra.run.smithi123.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-09-23T09:18:25.908 INFO:teuthology.orchestra.run.smithi123.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-09-23T09:18:25.937 INFO:teuthology.orchestra.run.smithi098.stdout:(36/131): cephfs-top-18.0.0-6336.g86266391.el8. 99 kB/s | 18 kB 00:00 2023-09-23T09:18:25.958 INFO:teuthology.orchestra.run.smithi123.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-09-23T09:18:26.025 INFO:teuthology.orchestra.run.smithi123.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-09-23T09:18:26.046 INFO:teuthology.orchestra.run.smithi098.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.8 MB/s | 7.4 MB 00:01 2023-09-23T09:18:26.196 INFO:teuthology.orchestra.run.smithi098.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 392 kB/s | 59 kB 00:00 2023-09-23T09:18:26.238 INFO:teuthology.orchestra.run.smithi098.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2023-09-23T09:18:26.261 INFO:teuthology.orchestra.run.smithi123.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 165 kB/s | 39 kB 00:00 2023-09-23T09:18:26.297 INFO:teuthology.orchestra.run.smithi098.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 749 kB/s | 75 kB 00:00 2023-09-23T09:18:26.356 INFO:teuthology.orchestra.run.smithi098.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2023-09-23T09:18:26.372 INFO:teuthology.orchestra.run.smithi098.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 581 kB/s | 44 kB 00:00 2023-09-23T09:18:26.426 INFO:teuthology.orchestra.run.smithi123.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 22 MB/s | 14 MB 00:00 2023-09-23T09:18:26.460 INFO:teuthology.orchestra.run.smithi123.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.4 MB/s | 688 kB 00:00 2023-09-23T09:18:26.502 INFO:teuthology.orchestra.run.smithi123.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-09-23T09:18:26.527 INFO:teuthology.orchestra.run.smithi123.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-09-23T09:18:26.552 INFO:teuthology.orchestra.run.smithi123.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-09-23T09:18:26.628 INFO:teuthology.orchestra.run.smithi123.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 7.0 MB/s | 713 kB 00:00 2023-09-23T09:18:26.670 INFO:teuthology.orchestra.run.smithi123.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 635 kB/s | 27 kB 00:00 2023-09-23T09:18:26.753 INFO:teuthology.orchestra.run.smithi123.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-09-23T09:18:26.779 INFO:teuthology.orchestra.run.smithi123.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-09-23T09:18:26.797 INFO:teuthology.orchestra.run.smithi123.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-09-23T09:18:26.823 INFO:teuthology.orchestra.run.smithi098.stdout:(43/131): ceph-test-18.0.0-6336.g86266391.el8.x 8.0 MB/s | 49 MB 00:06 2023-09-23T09:18:26.848 INFO:teuthology.orchestra.run.smithi098.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 69 kB/s | 33 kB 00:00 2023-09-23T09:18:26.854 INFO:teuthology.orchestra.run.smithi123.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-09-23T09:18:26.873 INFO:teuthology.orchestra.run.smithi098.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 358 kB/s | 185 kB 00:00 2023-09-23T09:18:26.921 INFO:teuthology.orchestra.run.smithi123.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 26 MB/s | 3.7 MB 00:00 2023-09-23T09:18:26.946 INFO:teuthology.orchestra.run.smithi123.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-09-23T09:18:26.957 INFO:teuthology.orchestra.run.smithi098.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-09-23T09:18:26.972 INFO:teuthology.orchestra.run.smithi123.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-09-23T09:18:26.982 INFO:teuthology.orchestra.run.smithi098.stdout:(47/131): python3-certifi-2018.10.15-7.el8.noar 103 kB/s | 16 kB 00:00 2023-09-23T09:18:26.997 INFO:teuthology.orchestra.run.smithi123.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-23T09:18:27.008 INFO:teuthology.orchestra.run.smithi098.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-09-23T09:18:27.022 INFO:teuthology.orchestra.run.smithi123.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-09-23T09:18:27.037 INFO:teuthology.orchestra.run.smithi045.stdout:(8/131): ceph-osd-18.0.0-6336.g86266391.el8.x86 3.9 MB/s | 18 MB 00:04 2023-09-23T09:18:27.041 INFO:teuthology.orchestra.run.smithi098.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 967 kB/s | 81 kB 00:00 2023-09-23T09:18:27.048 INFO:teuthology.orchestra.run.smithi123.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-23T09:18:27.067 INFO:teuthology.orchestra.run.smithi098.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-09-23T09:18:27.086 INFO:teuthology.orchestra.run.smithi045.stdout:(9/131): ceph-selinux-18.0.0-6336.g86266391.el8 493 kB/s | 24 kB 00:00 2023-09-23T09:18:27.092 INFO:teuthology.orchestra.run.smithi098.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 209 kB/s | 18 kB 00:00 2023-09-23T09:18:27.134 INFO:teuthology.orchestra.run.smithi098.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 729 kB/s | 49 kB 00:00 2023-09-23T09:18:27.168 INFO:teuthology.orchestra.run.smithi098.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 9.3 MB/s | 1.2 MB 00:00 2023-09-23T09:18:27.193 INFO:teuthology.orchestra.run.smithi098.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 584 kB/s | 59 kB 00:00 2023-09-23T09:18:27.218 INFO:teuthology.orchestra.run.smithi098.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 719 kB/s | 60 kB 00:00 2023-09-23T09:18:27.243 INFO:teuthology.orchestra.run.smithi098.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-09-23T09:18:27.269 INFO:teuthology.orchestra.run.smithi098.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 206 kB/s | 16 kB 00:00 2023-09-23T09:18:27.294 INFO:teuthology.orchestra.run.smithi098.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-09-23T09:18:27.320 INFO:teuthology.orchestra.run.smithi098.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-09-23T09:18:27.345 INFO:teuthology.orchestra.run.smithi098.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 852 kB/s | 64 kB 00:00 2023-09-23T09:18:27.370 INFO:teuthology.orchestra.run.smithi098.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-09-23T09:18:27.395 INFO:teuthology.orchestra.run.smithi098.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-09-23T09:18:27.441 INFO:teuthology.orchestra.run.smithi098.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 305 kB/s | 29 kB 00:00 2023-09-23T09:18:27.471 INFO:teuthology.orchestra.run.smithi098.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-09-23T09:18:27.490 INFO:teuthology.orchestra.run.smithi123.stdout:(119/131): ceph-test-18.0.0-6336.g86266391.el8. 4.6 MB/s | 49 MB 00:10 2023-09-23T09:18:27.521 INFO:teuthology.orchestra.run.smithi098.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-09-23T09:18:27.540 INFO:teuthology.orchestra.run.smithi123.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 1.3 MB/s | 709 kB 00:00 2023-09-23T09:18:27.547 INFO:teuthology.orchestra.run.smithi098.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2023-09-23T09:18:27.565 INFO:teuthology.orchestra.run.smithi123.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2023-09-23T09:18:27.572 INFO:teuthology.orchestra.run.smithi098.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.9 MB/s | 251 kB 00:00 2023-09-23T09:18:27.597 INFO:teuthology.orchestra.run.smithi098.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-09-23T09:18:27.615 INFO:teuthology.orchestra.run.smithi123.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-09-23T09:18:27.622 INFO:teuthology.orchestra.run.smithi098.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-09-23T09:18:27.648 INFO:teuthology.orchestra.run.smithi098.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-09-23T09:18:27.654 INFO:teuthology.orchestra.run.smithi045.stdout:(10/131): ceph-radosgw-18.0.0-6336.g86266391.el 4.1 MB/s | 14 MB 00:03 2023-09-23T09:18:27.690 INFO:teuthology.orchestra.run.smithi098.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 702 kB/s | 47 kB 00:00 2023-09-23T09:18:27.708 INFO:teuthology.orchestra.run.smithi123.stdout:(123/131): openblas-threads-0.3.15-3.el8.x86_64 7.4 MB/s | 4.9 MB 00:00 2023-09-23T09:18:27.715 INFO:teuthology.orchestra.run.smithi098.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.2 MB/s | 200 kB 00:00 2023-09-23T09:18:27.729 INFO:teuthology.orchestra.run.smithi045.stdout:(11/131): cephfs-mirror-18.0.0-6336.g86266391.e 3.4 MB/s | 260 kB 00:00 2023-09-23T09:18:27.740 INFO:teuthology.orchestra.run.smithi098.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 846 kB/s | 43 kB 00:00 2023-09-23T09:18:27.749 INFO:teuthology.orchestra.run.smithi123.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-09-23T09:18:27.779 INFO:teuthology.orchestra.run.smithi045.stdout:(12/131): libcephfs-devel-18.0.0-6336.g86266391 624 kB/s | 31 kB 00:00 2023-09-23T09:18:27.782 INFO:teuthology.orchestra.run.smithi098.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 8.3 MB/s | 1.7 MB 00:00 2023-09-23T09:18:27.808 INFO:teuthology.orchestra.run.smithi098.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 718 kB/s | 48 kB 00:00 2023-09-23T09:18:27.824 INFO:teuthology.orchestra.run.smithi123.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-09-23T09:18:27.841 INFO:teuthology.orchestra.run.smithi098.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-09-23T09:18:27.866 INFO:teuthology.orchestra.run.smithi123.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.7 MB/s | 808 kB 00:00 2023-09-23T09:18:27.867 INFO:teuthology.orchestra.run.smithi098.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 800 kB/s | 43 kB 00:00 2023-09-23T09:18:27.884 INFO:teuthology.orchestra.run.smithi098.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 861 kB/s | 87 kB 00:00 2023-09-23T09:18:27.909 INFO:teuthology.orchestra.run.smithi098.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-09-23T09:18:27.925 INFO:teuthology.orchestra.run.smithi123.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-09-23T09:18:27.929 INFO:teuthology.orchestra.run.smithi045.stdout:(13/131): libcephfs2-18.0.0-6336.g86266391.el8. 5.4 MB/s | 832 kB 00:00 2023-09-23T09:18:27.961 INFO:teuthology.orchestra.run.smithi098.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-09-23T09:18:27.979 INFO:teuthology.orchestra.run.smithi045.stdout:(14/131): libcephsqlite-18.0.0-6336.g86266391.e 3.6 MB/s | 184 kB 00:00 2023-09-23T09:18:27.985 INFO:teuthology.orchestra.run.smithi098.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 249 kB/s | 19 kB 00:00 2023-09-23T09:18:28.002 INFO:teuthology.orchestra.run.smithi098.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.7 MB/s | 155 kB 00:00 2023-09-23T09:18:28.009 INFO:teuthology.orchestra.run.smithi123.stdout:(128/131): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-09-23T09:18:28.030 INFO:teuthology.orchestra.run.smithi045.stdout:(15/131): librados-devel-18.0.0-6336.g86266391. 2.7 MB/s | 139 kB 00:00 2023-09-23T09:18:28.035 INFO:teuthology.orchestra.run.smithi098.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.6 MB/s | 560 kB 00:00 2023-09-23T09:18:28.061 INFO:teuthology.orchestra.run.smithi098.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-09-23T09:18:28.076 INFO:teuthology.orchestra.run.smithi123.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-09-23T09:18:28.094 INFO:teuthology.orchestra.run.smithi098.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2023-09-23T09:18:28.111 INFO:teuthology.orchestra.run.smithi098.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-09-23T09:18:28.136 INFO:teuthology.orchestra.run.smithi098.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2023-09-23T09:18:28.138 INFO:teuthology.orchestra.run.smithi045.stdout:(16/131): libradosstriper1-18.0.0-6336.g8626639 4.7 MB/s | 519 kB 00:00 2023-09-23T09:18:28.161 INFO:teuthology.orchestra.run.smithi098.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-09-23T09:18:28.187 INFO:teuthology.orchestra.run.smithi098.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-23T09:18:28.212 INFO:teuthology.orchestra.run.smithi098.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-09-23T09:18:28.237 INFO:teuthology.orchestra.run.smithi098.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-23T09:18:28.254 INFO:teuthology.orchestra.run.smithi098.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-09-23T09:18:28.271 INFO:teuthology.orchestra.run.smithi098.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-09-23T09:18:28.305 INFO:teuthology.orchestra.run.smithi098.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 13 MB/s | 644 kB 00:00 2023-09-23T09:18:28.330 INFO:teuthology.orchestra.run.smithi098.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-09-23T09:18:28.356 INFO:teuthology.orchestra.run.smithi098.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-09-23T09:18:28.414 INFO:teuthology.orchestra.run.smithi098.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 7.3 MB/s | 2.3 MB 00:00 2023-09-23T09:18:28.439 INFO:teuthology.orchestra.run.smithi098.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-09-23T09:18:28.490 INFO:teuthology.orchestra.run.smithi098.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 722 kB/s | 54 kB 00:00 2023-09-23T09:18:28.515 INFO:teuthology.orchestra.run.smithi098.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-09-23T09:18:28.534 INFO:teuthology.orchestra.run.smithi123.stdout:(130/131): librados2-18.0.0-6336.g86266391.el8. 5.6 MB/s | 3.7 MB 00:00 2023-09-23T09:18:28.548 INFO:teuthology.orchestra.run.smithi098.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-09-23T09:18:28.574 INFO:teuthology.orchestra.run.smithi098.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 663 kB/s | 39 kB 00:00 2023-09-23T09:18:28.616 INFO:teuthology.orchestra.run.smithi098.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-09-23T09:18:28.650 INFO:teuthology.orchestra.run.smithi098.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-09-23T09:18:28.675 INFO:teuthology.orchestra.run.smithi098.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-23T09:18:28.684 INFO:teuthology.orchestra.run.smithi123.stdout:(131/131): librbd1-18.0.0-6336.g86266391.el8.x8 4.9 MB/s | 3.7 MB 00:00 2023-09-23T09:18:28.688 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-09-23T09:18:28.688 INFO:teuthology.orchestra.run.smithi123.stdout:Total 15 MB/s | 209 MB 00:13 2023-09-23T09:18:28.700 INFO:teuthology.orchestra.run.smithi098.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-23T09:18:28.751 INFO:teuthology.orchestra.run.smithi098.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-09-23T09:18:28.793 INFO:teuthology.orchestra.run.smithi098.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 639 kB/s | 27 kB 00:00 2023-09-23T09:18:28.835 INFO:teuthology.orchestra.run.smithi098.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-23T09:18:28.868 INFO:teuthology.orchestra.run.smithi098.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-09-23T09:18:28.984 INFO:teuthology.orchestra.run.smithi098.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-09-23T09:18:29.036 INFO:teuthology.orchestra.run.smithi098.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.8 MB/s | 457 kB 00:00 2023-09-23T09:18:29.078 INFO:teuthology.orchestra.run.smithi098.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-09-23T09:18:29.145 INFO:teuthology.orchestra.run.smithi098.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-09-23T09:18:29.155 INFO:teuthology.orchestra.run.smithi045.stdout:(17/131): librgw2-18.0.0-6336.g86266391.el8.x86 6.8 MB/s | 6.9 MB 00:01 2023-09-23T09:18:29.170 INFO:teuthology.orchestra.run.smithi098.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-09-23T09:18:29.195 INFO:teuthology.orchestra.run.smithi098.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-23T09:18:29.197 INFO:teuthology.orchestra.run.smithi045.stdout:(18/131): python3-ceph-argparse-18.0.0-6336.g86 1.1 MB/s | 45 kB 00:00 2023-09-23T09:18:29.220 INFO:teuthology.orchestra.run.smithi098.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-09-23T09:18:29.239 INFO:teuthology.orchestra.run.smithi045.stdout:(19/131): python3-ceph-common-18.0.0-6336.g8626 3.0 MB/s | 125 kB 00:00 2023-09-23T09:18:29.246 INFO:teuthology.orchestra.run.smithi098.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-23T09:18:29.279 INFO:teuthology.orchestra.run.smithi098.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-23T09:18:29.298 INFO:teuthology.orchestra.run.smithi045.stdout:(20/131): python3-cephfs-18.0.0-6336.g86266391. 3.3 MB/s | 197 kB 00:00 2023-09-23T09:18:29.322 INFO:teuthology.orchestra.run.smithi098.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-09-23T09:18:29.373 INFO:teuthology.orchestra.run.smithi045.stdout:(21/131): python3-rados-18.0.0-6336.g86266391.e 4.8 MB/s | 371 kB 00:00 2023-09-23T09:18:29.448 INFO:teuthology.orchestra.run.smithi045.stdout:(22/131): python3-rbd-18.0.0-6336.g86266391.el8 4.8 MB/s | 364 kB 00:00 2023-09-23T09:18:29.464 INFO:teuthology.orchestra.run.smithi098.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 12 MB/s | 14 MB 00:01 2023-09-23T09:18:29.500 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-09-23T09:18:29.507 INFO:teuthology.orchestra.run.smithi045.stdout:(23/131): python3-rgw-18.0.0-6336.g86266391.el8 1.8 MB/s | 107 kB 00:00 2023-09-23T09:18:29.514 INFO:teuthology.orchestra.run.smithi098.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-09-23T09:18:29.556 INFO:teuthology.orchestra.run.smithi098.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-23T09:18:29.566 INFO:teuthology.orchestra.run.smithi045.stdout:(24/131): rbd-fuse-18.0.0-6336.g86266391.el8.x8 1.6 MB/s | 94 kB 00:00 2023-09-23T09:18:29.619 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-09-23T09:18:29.619 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-09-23T09:18:29.657 INFO:teuthology.orchestra.run.smithi098.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-09-23T09:18:29.740 INFO:teuthology.orchestra.run.smithi098.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2023-09-23T09:18:29.781 INFO:teuthology.orchestra.run.smithi098.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-09-23T09:18:29.840 INFO:teuthology.orchestra.run.smithi098.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-09-23T09:18:29.932 INFO:teuthology.orchestra.run.smithi098.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-09-23T09:18:30.015 INFO:teuthology.orchestra.run.smithi098.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 9.5 MB/s | 798 kB 00:00 2023-09-23T09:18:30.024 INFO:teuthology.orchestra.run.smithi045.stdout:(25/131): rbd-mirror-18.0.0-6336.g86266391.el8. 7.8 MB/s | 3.6 MB 00:00 2023-09-23T09:18:30.250 INFO:teuthology.orchestra.run.smithi045.stdout:(26/131): ceph-common-18.0.0-6336.g86266391.el8 1.9 MB/s | 23 MB 00:11 2023-09-23T09:18:30.275 INFO:teuthology.orchestra.run.smithi045.stdout:(27/131): rbd-nbd-18.0.0-6336.g86266391.el8.x86 719 kB/s | 180 kB 00:00 2023-09-23T09:18:30.317 INFO:teuthology.orchestra.run.smithi045.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6336.g 347 kB/s | 23 kB 00:00 2023-09-23T09:18:30.350 INFO:teuthology.orchestra.run.smithi045.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.7 MB/s | 132 kB 00:00 2023-09-23T09:18:30.366 INFO:teuthology.orchestra.run.smithi098.stdout:(130/131): librbd1-18.0.0-6336.g86266391.el8.x8 7.1 MB/s | 3.7 MB 00:00 2023-09-23T09:18:30.474 INFO:teuthology.orchestra.run.smithi098.stdout:(131/131): librados2-18.0.0-6336.g86266391.el8. 5.4 MB/s | 3.7 MB 00:00 2023-09-23T09:18:30.477 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2023-09-23T09:18:30.477 INFO:teuthology.orchestra.run.smithi098.stdout:Total 15 MB/s | 209 MB 00:14 2023-09-23T09:18:30.742 INFO:teuthology.orchestra.run.smithi045.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6336.g86266 4.1 MB/s | 1.7 MB 00:00 2023-09-23T09:18:30.754 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-09-23T09:18:30.758 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-09-23T09:18:30.809 INFO:teuthology.orchestra.run.smithi045.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6336.g86 3.6 MB/s | 242 kB 00:00 2023-09-23T09:18:30.876 INFO:teuthology.orchestra.run.smithi045.stdout:(32/131): ceph-mgr-rook-18.0.0-6336.g86266391.e 713 kB/s | 47 kB 00:00 2023-09-23T09:18:30.943 INFO:teuthology.orchestra.run.smithi045.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6336.g8 218 kB/s | 15 kB 00:00 2023-09-23T09:18:31.027 INFO:teuthology.orchestra.run.smithi045.stdout:(34/131): ceph-volume-18.0.0-6336.g86266391.el8 3.1 MB/s | 261 kB 00:00 2023-09-23T09:18:31.102 INFO:teuthology.orchestra.run.smithi045.stdout:(35/131): cephadm-18.0.0-6336.g86266391.el8.noa 2.4 MB/s | 181 kB 00:00 2023-09-23T09:18:31.169 INFO:teuthology.orchestra.run.smithi045.stdout:(36/131): cephfs-top-18.0.0-6336.g86266391.el8. 271 kB/s | 18 kB 00:00 2023-09-23T09:18:31.297 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2023-09-23T09:18:31.345 INFO:teuthology.orchestra.run.smithi045.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 7.5 MB/s | 7.4 MB 00:00 2023-09-23T09:18:31.416 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2023-09-23T09:18:31.416 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2023-09-23T09:18:31.687 INFO:teuthology.orchestra.run.smithi045.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2023-09-23T09:18:31.787 INFO:teuthology.orchestra.run.smithi045.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 496 kB/s | 306 kB 00:00 2023-09-23T09:18:31.887 INFO:teuthology.orchestra.run.smithi045.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 375 kB/s | 75 kB 00:00 2023-09-23T09:18:31.996 INFO:teuthology.orchestra.run.smithi045.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2023-09-23T09:18:32.020 INFO:teuthology.orchestra.run.smithi045.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2023-09-23T09:18:32.471 INFO:teuthology.orchestra.run.smithi045.stdout:(43/131): ceph-test-18.0.0-6336.g86266391.el8.x 9.1 MB/s | 49 MB 00:05 2023-09-23T09:18:32.496 INFO:teuthology.orchestra.run.smithi045.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 69 kB/s | 33 kB 00:00 2023-09-23T09:18:32.521 INFO:teuthology.orchestra.run.smithi045.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 352 kB/s | 185 kB 00:00 2023-09-23T09:18:32.549 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2023-09-23T09:18:32.553 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2023-09-23T09:18:32.647 INFO:teuthology.orchestra.run.smithi045.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-09-23T09:18:32.672 INFO:teuthology.orchestra.run.smithi045.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 984 kB/s | 173 kB 00:00 2023-09-23T09:18:32.714 INFO:teuthology.orchestra.run.smithi045.stdout:(48/131): python3-certifi-2018.10.15-7.el8.noar 67 kB/s | 16 kB 00:00 2023-09-23T09:18:32.756 INFO:teuthology.orchestra.run.smithi045.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 748 kB/s | 81 kB 00:00 2023-09-23T09:18:32.773 INFO:teuthology.orchestra.run.smithi045.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2023-09-23T09:18:32.823 INFO:teuthology.orchestra.run.smithi045.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 161 kB/s | 18 kB 00:00 2023-09-23T09:18:32.873 INFO:teuthology.orchestra.run.smithi045.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 486 kB/s | 49 kB 00:00 2023-09-23T09:18:32.965 INFO:teuthology.orchestra.run.smithi045.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-09-23T09:18:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-09-23T09:18:33.007 INFO:teuthology.orchestra.run.smithi045.stdout:(55/131): python3-more-itertools-7.2.0-3.el8.no 319 kB/s | 59 kB 00:00 2023-09-23T09:18:33.066 INFO:teuthology.orchestra.run.smithi045.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-09-23T09:18:33.083 INFO:teuthology.orchestra.run.smithi045.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-09-23T09:18:33.116 INFO:teuthology.orchestra.run.smithi045.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2023-09-23T09:18:33.190 INFO:teuthology.orchestra.run.smithi045.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-09-23T09:18:33.217 INFO:teuthology.orchestra.run.smithi045.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 480 kB/s | 64 kB 00:00 2023-09-23T09:18:33.234 INFO:teuthology.orchestra.run.smithi045.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2023-09-23T09:18:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 255 kB/s | 24 kB 00:00 2023-09-23T09:18:33.318 INFO:teuthology.orchestra.run.smithi045.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-09-23T09:18:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-09-23T09:18:33.385 INFO:teuthology.orchestra.run.smithi045.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-09-23T09:18:33.477 INFO:teuthology.orchestra.run.smithi045.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 458 kB/s | 61 kB 00:00 2023-09-23T09:18:33.502 INFO:teuthology.orchestra.run.smithi045.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-09-23T09:18:33.553 INFO:teuthology.orchestra.run.smithi045.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 510 kB/s | 85 kB 00:00 2023-09-23T09:18:33.578 INFO:teuthology.orchestra.run.smithi045.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-23T09:18:33.603 INFO:teuthology.orchestra.run.smithi045.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 182 kB/s | 23 kB 00:00 2023-09-23T09:18:33.670 INFO:teuthology.orchestra.run.smithi045.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-09-23T09:18:33.712 INFO:teuthology.orchestra.run.smithi045.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-23T09:18:33.754 INFO:teuthology.orchestra.run.smithi045.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-23T09:18:33.796 INFO:teuthology.orchestra.run.smithi045.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-09-23T09:18:33.838 INFO:teuthology.orchestra.run.smithi045.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 936 kB/s | 39 kB 00:00 2023-09-23T09:18:33.879 INFO:teuthology.orchestra.run.smithi045.stdout:(76/131): thrift-0.13.0-2.el8.x86_64.rpm 4.6 MB/s | 1.7 MB 00:00 2023-09-23T09:18:33.952 INFO:teuthology.orchestra.run.smithi045.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 5.9 MB/s | 688 kB 00:00 2023-09-23T09:18:33.997 INFO:teuthology.orchestra.run.smithi045.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.5 MB/s | 538 kB 00:00 2023-09-23T09:18:34.022 INFO:teuthology.orchestra.run.smithi045.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.2 MB/s | 157 kB 00:00 2023-09-23T09:18:34.039 INFO:teuthology.orchestra.run.smithi045.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-09-23T09:18:34.089 INFO:teuthology.orchestra.run.smithi045.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-23T09:18:34.131 INFO:teuthology.orchestra.run.smithi045.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 641 kB/s | 27 kB 00:00 2023-09-23T09:18:34.173 INFO:teuthology.orchestra.run.smithi045.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-23T09:18:34.215 INFO:teuthology.orchestra.run.smithi045.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-09-23T09:18:34.315 INFO:teuthology.orchestra.run.smithi045.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-09-23T09:18:34.466 INFO:teuthology.orchestra.run.smithi045.stdout:(86/131): python3-scipy-1.0.0-21.module+el8.5.0 15 MB/s | 14 MB 00:00 2023-09-23T09:18:34.491 INFO:teuthology.orchestra.run.smithi045.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.6 MB/s | 457 kB 00:00 2023-09-23T09:18:34.516 INFO:teuthology.orchestra.run.smithi045.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.4 MB/s | 225 kB 00:00 2023-09-23T09:18:34.541 INFO:teuthology.orchestra.run.smithi045.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-09-23T09:18:34.566 INFO:teuthology.orchestra.run.smithi045.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-09-23T09:18:34.592 INFO:teuthology.orchestra.run.smithi045.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-09-23T09:18:34.617 INFO:teuthology.orchestra.run.smithi045.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-23T09:18:34.684 INFO:teuthology.orchestra.run.smithi045.stdout:(93/131): python3-numpy-1.14.3-10.el8.x86_64.rp 8.0 MB/s | 3.7 MB 00:00 2023-09-23T09:18:34.743 INFO:teuthology.orchestra.run.smithi045.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2023-09-23T09:18:34.768 INFO:teuthology.orchestra.run.smithi045.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-09-23T09:18:34.810 INFO:teuthology.orchestra.run.smithi045.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-23T09:18:34.852 INFO:teuthology.orchestra.run.smithi045.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-23T09:18:34.952 INFO:teuthology.orchestra.run.smithi045.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-09-23T09:18:35.027 INFO:teuthology.orchestra.run.smithi045.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-09-23T09:18:35.078 INFO:teuthology.orchestra.run.smithi045.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-09-23T09:18:35.111 INFO:teuthology.orchestra.run.smithi045.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-09-23T09:18:35.161 INFO:teuthology.orchestra.run.smithi045.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-09-23T09:18:35.187 INFO:teuthology.orchestra.run.smithi045.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-09-23T09:18:35.212 INFO:teuthology.orchestra.run.smithi045.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-09-23T09:18:35.237 INFO:teuthology.orchestra.run.smithi045.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-23T09:18:35.263 INFO:teuthology.orchestra.run.smithi045.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 955 kB/s | 48 kB 00:00 2023-09-23T09:18:35.288 INFO:teuthology.orchestra.run.smithi045.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-23T09:18:35.313 INFO:teuthology.orchestra.run.smithi045.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-09-23T09:18:35.338 INFO:teuthology.orchestra.run.smithi045.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-23T09:18:35.364 INFO:teuthology.orchestra.run.smithi045.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-09-23T09:18:35.431 INFO:teuthology.orchestra.run.smithi045.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.7 MB/s | 5.1 MB 00:00 2023-09-23T09:18:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2023-09-23T09:18:35.506 INFO:teuthology.orchestra.run.smithi045.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-09-23T09:18:35.540 INFO:teuthology.orchestra.run.smithi045.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.7 MB/s | 560 kB 00:00 2023-09-23T09:18:35.557 INFO:teuthology.orchestra.run.smithi045.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 357 kB/s | 18 kB 00:00 2023-09-23T09:18:35.582 INFO:teuthology.orchestra.run.smithi045.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.3 MB/s | 559 kB 00:00 2023-09-23T09:18:35.607 INFO:teuthology.orchestra.run.smithi045.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-09-23T09:18:35.649 INFO:teuthology.orchestra.run.smithi045.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-09-23T09:18:35.667 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-09-23T09:18:35.855 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-23T09:18:35.880 INFO:teuthology.orchestra.run.smithi045.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 265 kB/s | 72 kB 00:00 2023-09-23T09:18:36.025 INFO:teuthology.orchestra.run.smithi045.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 220 kB/s | 83 kB 00:00 2023-09-23T09:18:36.042 INFO:teuthology.orchestra.run.smithi045.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 372 kB/s | 60 kB 00:00 2023-09-23T09:18:36.046 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-23T09:18:36.067 INFO:teuthology.orchestra.run.smithi045.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-09-23T09:18:36.092 INFO:teuthology.orchestra.run.smithi045.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-09-23T09:18:36.118 INFO:teuthology.orchestra.run.smithi045.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 768 kB/s | 39 kB 00:00 2023-09-23T09:18:36.159 INFO:teuthology.orchestra.run.smithi045.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-09-23T09:18:36.185 INFO:teuthology.orchestra.run.smithi045.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-09-23T09:18:36.210 INFO:teuthology.orchestra.run.smithi045.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-23T09:18:36.216 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T09:18:36.302 INFO:teuthology.orchestra.run.smithi045.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 3.2 MB/s | 2.3 MB 00:00 2023-09-23T09:18:36.353 INFO:teuthology.orchestra.run.smithi045.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-23T09:18:36.400 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T09:18:36.611 INFO:teuthology.orchestra.run.smithi045.stdout:(130/131): librbd1-18.0.0-6336.g86266391.el8.x8 9.3 MB/s | 3.7 MB 00:00 2023-09-23T09:18:36.686 INFO:teuthology.orchestra.run.smithi045.stdout:(131/131): librados2-18.0.0-6336.g86266391.el8. 7.5 MB/s | 3.7 MB 00:00 2023-09-23T09:18:36.689 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-09-23T09:18:36.689 INFO:teuthology.orchestra.run.smithi045.stdout:Total 11 MB/s | 209 MB 00:18 2023-09-23T09:18:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T09:18:37.196 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T09:18:37.365 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T09:18:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-09-23T09:18:37.667 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2023-09-23T09:18:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-09-23T09:18:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-09-23T09:18:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T09:18:37.878 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-23T09:18:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:18:38.078 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-23T09:18:38.080 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:18:38.273 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:18:38.282 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T09:18:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:18:38.383 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T09:18:38.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T09:18:38.613 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T09:18:38.707 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-23T09:18:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-09-23T09:18:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-09-23T09:18:38.927 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-23T09:18:39.019 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T09:18:39.134 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-23T09:18:39.213 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T09:18:39.526 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-23T09:18:39.609 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T09:18:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-23T09:18:39.788 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:18:39.938 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:18:39.948 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T09:18:40.056 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T09:18:40.129 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:18:40.198 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T09:18:40.220 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:18:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T09:18:40.418 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T09:18:40.589 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-23T09:18:40.593 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-23T09:18:40.779 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-23T09:18:40.834 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-23T09:18:40.999 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-23T09:18:41.057 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-23T09:18:41.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T09:18:41.282 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T09:18:41.436 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-23T09:18:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T09:18:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-23T09:18:42.062 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T09:18:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T09:18:42.275 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T09:18:42.348 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T09:18:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T09:18:42.609 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T09:18:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-23T09:18:42.943 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T09:18:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-23T09:18:43.114 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T09:18:43.216 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-23T09:18:43.346 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T09:18:43.503 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T09:18:43.663 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T09:18:43.813 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T09:18:43.847 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-09-23T09:18:44.051 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-23T09:18:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-23T09:18:44.223 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-23T09:18:44.253 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-23T09:18:44.262 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T09:18:44.412 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-23T09:18:44.429 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T09:18:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-23T09:18:44.595 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-23T09:18:44.687 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 26/134 2023-09-23T09:18:44.689 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-23T09:18:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/134 2023-09-23T09:18:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-23T09:18:45.030 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 28/134 2023-09-23T09:18:45.059 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T09:18:45.246 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T09:18:45.247 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T09:18:45.398 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-23T09:18:45.628 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:18:45.948 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T09:18:46.055 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:18:46.202 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-23T09:18:46.236 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:18:46.401 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-23T09:18:46.407 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:18:46.664 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-23T09:18:46.785 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T09:18:46.828 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 26/134 2023-09-23T09:18:46.891 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T09:18:46.948 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/134 2023-09-23T09:18:47.016 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T09:18:47.099 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T09:18:47.191 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 28/134 2023-09-23T09:18:47.334 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-23T09:18:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T09:18:47.566 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-23T09:18:47.982 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-23T09:18:48.015 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-23T09:18:48.202 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-23T09:18:48.321 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-23T09:18:48.344 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T09:18:48.452 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T09:18:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-23T09:18:48.626 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T09:18:48.719 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T09:18:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-23T09:18:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-23T09:18:49.007 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-23T09:18:49.041 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-23T09:18:49.160 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-23T09:18:49.218 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T09:18:49.256 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-23T09:18:49.402 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 36/134 2023-09-23T09:18:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 37/134 2023-09-23T09:18:49.737 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-23T09:18:49.815 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-23T09:18:49.882 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-23T09:18:49.993 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-23T09:18:50.038 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T09:18:50.111 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T09:18:50.222 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-23T09:18:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-23T09:18:50.541 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-23T09:18:50.542 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-23T09:18:50.630 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-23T09:18:50.688 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T09:18:50.772 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T09:18:50.822 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-23T09:18:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-23T09:18:50.985 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-23T09:18:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-23T09:18:51.133 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-23T09:18:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-23T09:18:51.280 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-23T09:18:51.479 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-23T09:18:51.482 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-23T09:18:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-23T09:18:51.650 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 36/134 2023-09-23T09:18:51.743 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-23T09:18:51.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-23T09:18:51.834 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 37/134 2023-09-23T09:18:51.978 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-23T09:18:51.993 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-23T09:18:52.000 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-23T09:18:52.143 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-23T09:18:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-23T09:18:52.235 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-23T09:18:52.336 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T09:18:52.405 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T09:18:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-23T09:18:52.589 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-23T09:18:52.767 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-23T09:18:52.937 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-23T09:18:52.981 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-23T09:18:53.157 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-23T09:18:53.207 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-23T09:18:53.217 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-23T09:18:53.336 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T09:18:53.416 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-23T09:18:53.417 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 26/134 2023-09-23T09:18:53.418 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T09:18:53.556 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/134 2023-09-23T09:18:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-23T09:18:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-23T09:18:53.857 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-23T09:18:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-23T09:18:54.020 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-23T09:18:54.205 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-23T09:18:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-23T09:18:54.397 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-23T09:18:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-23T09:18:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 28/134 2023-09-23T09:18:54.548 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-23T09:18:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-23T09:18:54.744 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-23T09:18:54.834 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-23T09:18:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-23T09:18:54.960 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-23T09:18:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-23T09:18:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-23T09:18:55.207 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-23T09:18:55.327 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-23T09:18:55.357 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-23T09:18:55.484 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-23T09:18:55.559 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-23T09:18:55.679 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-23T09:18:55.742 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-23T09:18:55.914 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-23T09:18:55.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-23T09:18:56.123 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-23T09:18:56.246 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-23T09:18:56.365 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-23T09:18:56.411 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-23T09:18:56.583 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-23T09:18:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-23T09:18:56.843 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-23T09:18:56.985 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-23T09:18:57.061 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-23T09:18:57.166 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-23T09:18:57.225 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-23T09:18:57.374 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-23T09:18:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-23T09:18:57.438 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-23T09:18:57.572 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-23T09:18:57.606 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-23T09:18:57.772 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-23T09:18:57.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-23T09:18:57.863 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-23T09:18:57.990 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-23T09:18:58.091 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-23T09:18:58.108 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-23T09:18:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-23T09:18:58.305 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-23T09:18:58.350 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-23T09:18:58.361 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-23T09:18:58.499 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 36/134 2023-09-23T09:18:58.585 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-23T09:18:58.623 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-23T09:18:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 37/134 2023-09-23T09:18:58.816 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-23T09:18:58.845 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-23T09:18:58.859 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-23T09:18:59.028 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-23T09:18:59.034 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-23T09:18:59.043 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-23T09:18:59.195 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-23T09:18:59.221 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-23T09:18:59.266 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-23T09:18:59.347 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-23T09:18:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-23T09:18:59.468 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-23T09:18:59.602 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-23T09:18:59.644 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-23T09:18:59.692 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-23T09:18:59.803 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-23T09:18:59.839 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-23T09:18:59.861 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-23T09:19:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-23T09:19:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-23T09:19:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-23T09:19:00.420 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-23T09:19:00.505 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-23T09:19:00.627 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-23T09:19:00.704 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-23T09:19:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-23T09:19:00.918 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-23T09:19:00.957 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-23T09:19:00.988 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-23T09:19:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-23T09:19:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-23T09:19:01.274 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-23T09:19:01.477 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-23T09:19:01.488 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-23T09:19:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-23T09:19:01.667 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-23T09:19:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-23T09:19:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-23T09:19:01.863 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-23T09:19:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-23T09:19:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-23T09:19:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-23T09:19:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-23T09:19:02.271 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-23T09:19:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-23T09:19:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-23T09:19:02.437 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-23T09:19:02.593 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-23T09:19:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-23T09:19:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-23T09:19:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-23T09:19:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T09:19:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T09:19:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-23T09:19:03.637 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 93/134 2023-09-23T09:19:03.718 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-23T09:19:03.732 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-23T09:19:03.908 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-23T09:19:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-23T09:19:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-23T09:19:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-23T09:19:04.276 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-23T09:19:04.341 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-23T09:19:04.578 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-23T09:19:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-23T09:19:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-23T09:19:04.864 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-23T09:19:04.933 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-23T09:19:05.052 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-23T09:19:05.077 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-23T09:19:05.276 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-23T09:19:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-23T09:19:05.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-23T09:19:05.816 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-23T09:19:05.917 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-23T09:19:06.088 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-23T09:19:06.093 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T09:19:06.174 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T09:19:06.308 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-23T09:19:06.385 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 93/134 2023-09-23T09:19:06.471 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-23T09:19:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-23T09:19:06.855 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-23T09:19:07.065 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-23T09:19:07.247 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-23T09:19:07.293 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-23T09:19:07.939 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-23T09:19:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-23T09:19:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T09:19:08.471 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-23T09:19:08.488 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T09:19:08.656 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-23T09:19:08.930 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-23T09:19:09.141 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-23T09:19:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T09:19:09.317 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-23T09:19:09.545 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-23T09:19:10.048 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:10.268 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:10.755 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-23T09:19:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:11.030 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-23T09:19:11.141 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T09:19:11.221 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T09:19:11.287 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-23T09:19:11.384 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T09:19:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-23T09:19:11.892 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-23T09:19:12.124 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-23T09:19:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-23T09:19:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T09:19:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T09:19:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 93/134 2023-09-23T09:19:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:17.532 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:18.157 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T09:19:19.187 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T09:19:19.274 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T09:19:19.397 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /run 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2023-09-23T09:19:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2023-09-23T09:19:28.030 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-09-23T09:19:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:19:28.416 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T09:19:30.023 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /run 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-09-23T09:19:30.105 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:19:30.186 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T09:19:30.405 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T09:19:30.465 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T09:19:30.607 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T09:19:31.069 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T09:19:31.249 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T09:19:31.266 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T09:19:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T09:19:32.271 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T09:19:32.491 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T09:19:32.673 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T09:19:32.948 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T09:19:33.130 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T09:19:33.132 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T09:19:33.237 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 102/134 2023-09-23T09:19:33.323 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T09:19:33.340 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T09:19:33.430 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 103/134 2023-09-23T09:19:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T09:19:33.867 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T09:19:34.177 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T09:19:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T09:19:35.133 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T09:19:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T09:19:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 102/134 2023-09-23T09:19:35.314 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T09:19:35.404 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T09:19:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 103/134 2023-09-23T09:19:35.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 107/134 2023-09-23T09:19:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T09:19:35.948 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T09:19:35.949 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T09:19:35.984 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T09:19:36.122 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T09:19:36.134 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T09:19:36.246 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T09:19:37.202 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T09:19:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T09:19:37.458 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el8.x86_64 110/134 2023-09-23T09:19:37.487 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T09:19:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T09:19:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T09:19:37.830 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 107/134 2023-09-23T09:19:37.848 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T09:19:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T09:19:38.084 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T09:19:38.101 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T09:19:38.119 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T09:19:38.280 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T09:19:38.291 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T09:19:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T09:19:38.645 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T09:19:39.111 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T09:19:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2023-09-23T09:19:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2023-09-23T09:19:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-09-23T09:19:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /run 2023-09-23T09:19:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2023-09-23T09:19:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2023-09-23T09:19:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2023-09-23T09:19:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2023-09-23T09:19:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-09-23T09:19:39.112 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:19:39.528 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T09:19:39.641 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el8.x86_64 110/134 2023-09-23T09:19:39.838 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T09:19:39.872 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T09:19:40.072 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T09:19:40.113 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T09:19:40.297 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T09:19:40.654 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T09:19:40.821 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T09:19:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T09:19:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T09:19:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T09:19:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T09:19:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T09:19:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T09:19:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T09:19:43.297 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T09:19:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 115/134 2023-09-23T09:19:44.439 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T09:19:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-23T09:19:44.636 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T09:19:44.707 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 117/134 2023-09-23T09:19:44.750 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 102/134 2023-09-23T09:19:44.937 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 103/134 2023-09-23T09:19:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-23T09:19:45.120 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T09:19:45.151 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T09:19:45.310 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T09:19:45.379 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T09:19:45.435 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T09:19:45.608 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-23T09:19:45.649 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T09:19:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 121/134 2023-09-23T09:19:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T09:19:46.336 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 115/134 2023-09-23T09:19:46.665 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-23T09:19:46.671 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T09:19:46.866 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 117/134 2023-09-23T09:19:46.885 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T09:19:46.982 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T09:19:47.146 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-23T09:19:47.304 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T09:19:47.359 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 107/134 2023-09-23T09:19:47.492 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T09:19:47.559 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T09:19:47.596 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T09:19:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-23T09:19:47.793 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T09:19:47.804 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T09:19:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 121/134 2023-09-23T09:19:48.072 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T09:19:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:19:48.387 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:19:48.405 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:19:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 124/134 2023-09-23T09:19:48.878 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 125/134 2023-09-23T09:19:49.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 126/134 2023-09-23T09:19:49.218 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el8.x86_64 110/134 2023-09-23T09:19:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T09:19:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T09:19:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-23T09:19:49.626 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T09:19:49.670 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T09:19:49.762 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T09:19:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T09:19:49.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T09:19:50.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T09:19:50.074 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T09:19:50.224 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T09:19:50.245 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:19:50.253 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T09:19:50.261 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T09:19:50.397 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-23T09:19:50.397 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:19:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T09:19:50.496 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:19:50.517 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:19:50.598 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:19:50.598 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T09:19:50.683 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 124/134 2023-09-23T09:19:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T09:19:51.012 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 125/134 2023-09-23T09:19:51.035 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:19:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 126/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el8.x86_64 1/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 3/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 4/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:19:51.524 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 10/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 11/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 12/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 13/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 14/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 15/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 16/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 17/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 18/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 19/134 2023-09-23T09:19:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/134 2023-09-23T09:19:51.526 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 21/134 2023-09-23T09:19:51.526 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 22/134 2023-09-23T09:19:51.526 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 23/134 2023-09-23T09:19:51.526 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 24/134 2023-09-23T09:19:51.526 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 25/134 2023-09-23T09:19:51.535 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 26/134 2023-09-23T09:19:51.535 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 27/134 2023-09-23T09:19:51.535 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 28/134 2023-09-23T09:19:51.535 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 30/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 31/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 34/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 36/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el8.noarch 37/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 38/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-23T09:19:51.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-23T09:19:51.537 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-23T09:19:51.538 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-23T09:19:51.539 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-23T09:19:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-23T09:19:51.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-23T09:19:51.542 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-23T09:19:51.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el8.x86_64 129/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 131/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:19:51.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-23T09:19:51.690 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-23T09:19:51.903 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T09:19:51.987 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T09:19:52.220 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T09:19:52.232 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T09:19:52.428 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T09:19:52.437 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T09:19:52.564 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-23T09:19:52.564 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:19:52.765 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:19:52.765 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T09:19:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T09:19:53.235 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:19:53.759 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:19:53.759 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el8.x86_64 1/134 2023-09-23T09:19:53.759 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2/134 2023-09-23T09:19:53.759 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 3/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 4/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 10/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 11/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 12/134 2023-09-23T09:19:53.760 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 13/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 14/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 15/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 16/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 17/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 18/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 19/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 21/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 22/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 23/134 2023-09-23T09:19:53.761 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 24/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 25/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 26/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 27/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 28/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 30/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 31/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 34/134 2023-09-23T09:19:53.762 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 36/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el8.noarch 37/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 38/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-23T09:19:53.763 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-23T09:19:53.764 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-23T09:19:53.765 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-23T09:19:53.766 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-23T09:19:53.767 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-23T09:19:53.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-23T09:19:53.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-23T09:19:53.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-23T09:19:53.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el8.x86_64 129/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 131/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:19:53.772 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-23T09:19:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:19:53.948 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:50 ago on Sat 23 Sep 2023 09:18:03 AM UTC. 2023-09-23T09:19:55.370 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T09:19:55.652 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 115/134 2023-09-23T09:19:55.850 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-23T09:19:56.059 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-23T09:19:56.239 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:19:56.239 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:52 ago on Sat 23 Sep 2023 09:18:04 AM UTC. 2023-09-23T09:19:56.712 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-09-23T09:19:56.924 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:19:56.924 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2023-09-23T09:19:56.924 INFO:teuthology.orchestra.run.smithi123.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-23T09:19:56.924 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.1-7.el8.noarch 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.0.4-10.el8.x86_64 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.925 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.926 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.1-9.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-23T09:19:56.927 INFO:teuthology.orchestra.run.smithi123.stdout: jq-1.6-3.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libpq-13.5-1.el8.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-23T09:19:56.928 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-23T09:19:56.929 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-23T09:19:56.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-23T09:19:56.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-23T09:19:56.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-2.6-1.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-23T09:19:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-23T09:19:56.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-23T09:19:56.935 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-23T09:19:56.936 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-23T09:19:56.936 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-23T09:19:56.936 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-23T09:19:56.936 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:19:56.936 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-09-23T09:19:57.002 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-23T09:19:57.122 DEBUG:teuthology.parallel:result is None 2023-09-23T09:19:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-23T09:19:57.537 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 119/134 2023-09-23T09:19:57.713 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-23T09:19:57.888 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 121/134 2023-09-23T09:19:58.051 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T09:19:58.756 INFO:teuthology.orchestra.run.smithi098.stdout:Installed products updated. 2023-09-23T09:19:58.968 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.1-7.el8.noarch 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.0.4-10.el8.x86_64 2023-09-23T09:19:58.969 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.970 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.971 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.1-9.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: jq-1.6-3.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.972 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: libpq-13.5-1.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-23T09:19:58.973 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-23T09:19:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-23T09:19:58.975 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-23T09:19:58.976 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-23T09:19:58.977 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-2.6-1.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-23T09:19:58.978 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-23T09:19:58.979 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-23T09:19:58.980 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-23T09:19:58.981 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-23T09:19:58.981 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-23T09:19:58.981 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-23T09:19:58.981 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-23T09:19:58.981 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-23T09:19:58.981 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-09-23T09:19:59.158 DEBUG:teuthology.parallel:result is None 2023-09-23T09:20:00.422 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:20:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:20:00.697 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T09:20:00.867 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 124/134 2023-09-23T09:20:01.225 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 125/134 2023-09-23T09:20:01.537 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 126/134 2023-09-23T09:20:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-23T09:20:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-23T09:20:02.018 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-23T09:20:02.196 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-23T09:20:02.366 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-23T09:20:02.460 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-23T09:20:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-23T09:20:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-23T09:20:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:20:02.903 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:20:02.903 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T09:20:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T09:20:03.315 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:20:03.863 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:20:03.863 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el8.x86_64 1/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 3/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 4/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 10/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 11/134 2023-09-23T09:20:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 12/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 13/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 14/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 15/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 16/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 17/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 18/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 19/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 21/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 22/134 2023-09-23T09:20:03.865 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 23/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 24/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 25/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 26/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 27/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 28/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 30/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 31/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 34/134 2023-09-23T09:20:03.866 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 36/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el8.noarch 37/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 38/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-23T09:20:03.867 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-23T09:20:03.868 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-23T09:20:03.869 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-23T09:20:03.870 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-23T09:20:03.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-23T09:20:03.872 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-23T09:20:03.873 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-23T09:20:03.874 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-23T09:20:03.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-23T09:20:03.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-23T09:20:03.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el8.x86_64 129/134 2023-09-23T09:20:03.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-23T09:20:03.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 131/134 2023-09-23T09:20:03.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T09:20:03.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-23T09:20:06.348 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T09:20:06.348 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:01 ago on Sat 23 Sep 2023 09:18:05 AM UTC. 2023-09-23T09:20:08.877 INFO:teuthology.orchestra.run.smithi045.stdout:Installed products updated. 2023-09-23T09:20:09.111 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:20:09.111 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2023-09-23T09:20:09.111 INFO:teuthology.orchestra.run.smithi045.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-23T09:20:09.111 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.111 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.1-7.el8.noarch 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.0.4-10.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.112 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.113 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.1-9.el8.x86_64 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-23T09:20:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: jq-1.6-3.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libpq-13.5-1.el8.x86_64 2023-09-23T09:20:09.115 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-23T09:20:09.116 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:20:09.117 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-23T09:20:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-23T09:20:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-2.6-1.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-23T09:20:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-23T09:20:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.122 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T09:20:09.123 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-23T09:20:09.123 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-23T09:20:09.123 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-23T09:20:09.123 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-23T09:20:09.123 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-23T09:20:09.123 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-23T09:20:09.123 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:20:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-09-23T09:20:09.307 DEBUG:teuthology.parallel:result is None 2023-09-23T09:20:09.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:20:09.528 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T09:20:09.553 INFO:teuthology.orchestra.run.smithi045.stdout:18.0.0-6336.g86266391.el8 2023-09-23T09:20:09.554 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el8 2023-09-23T09:20:09.554 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T09:20:09.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:20:09.708 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T09:20:09.736 INFO:teuthology.orchestra.run.smithi098.stdout:18.0.0-6336.g86266391.el8 2023-09-23T09:20:09.736 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el8 2023-09-23T09:20:09.736 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T09:20:09.736 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:20:09.966 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T09:20:09.995 INFO:teuthology.orchestra.run.smithi123.stdout:18.0.0-6336.g86266391.el8 2023-09-23T09:20:09.995 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el8 2023-09-23T09:20:09.995 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T09:20:09.995 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-23T09:20:09.995 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:20:09.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T09:20:10.041 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:20:10.041 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T09:20:10.091 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:20:10.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T09:20:10.135 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-23T09:20:10.136 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:20:10.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T09:20:10.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T09:20:10.235 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:20:10.235 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T09:20:10.267 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T09:20:10.336 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:20:10.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T09:20:10.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T09:20:10.435 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-23T09:20:10.435 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:20:10.435 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T09:20:10.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T09:20:10.535 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:20:10.535 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T09:20:10.565 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T09:20:10.634 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:20:10.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T09:20:10.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T09:20:10.740 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-23T09:20:10.740 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:20:10.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T09:20:10.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T09:20:10.841 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:20:10.841 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T09:20:10.873 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T09:20:10.944 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:20:10.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T09:20:10.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T09:20:11.050 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-23T09:20:11.164 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-09-23T09:20:11.165 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:20:11.165 INFO:tasks.cephadm:Cluster fsid is 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:20:11.165 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-23T09:20:11.166 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.98', 'mon.c': '172.21.15.123'} 2023-09-23T09:20:11.166 INFO:tasks.cephadm:First mon is mon.a on smithi045 2023-09-23T09:20:11.166 INFO:tasks.cephadm:First mgr is x 2023-09-23T09:20:11.166 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-23T09:20:11.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2023-09-23T09:20:11.201 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2023-09-23T09:20:11.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2023-09-23T09:20:11.265 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-23T09:20:11.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:20:11.478 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72754/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6336-g86266391', 'node_name': '172.21.4.63+confusa01', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6336.g86266391'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-22 19:32:56.769053', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20230922.144532', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72751/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6336-g86266391', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6336.g86266391'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-22 16:40:43.037820', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20230922.144532', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-09-23T09:20:11.704 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20230922.144532, sha1 862663913a74a99e1c52d50e6a7e59dd6705749a from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T09:20:11.705 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm 2023-09-23T09:20:11.705 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:12.079 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-rw-r--. 1 ubuntu ubuntu 741164 Sep 23 09:20 /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:12.079 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:12.522 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-rw-r--. 1 ubuntu ubuntu 741164 Sep 23 09:20 /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:12.522 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:13.043 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-rw-r--. 1 ubuntu ubuntu 741164 Sep 23 09:20 /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:13.044 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:13.066 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:13.087 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-23T09:20:13.124 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a on all hosts... 2023-09-23T09:20:13.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a pull 2023-09-23T09:20:13.126 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a pull 2023-09-23T09:20:13.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a pull 2023-09-23T09:20:15.330 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T09:20:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T09:20:18.763 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T09:20:44.239 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:20:44.239 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T09:20:44.239 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c", 2023-09-23T09:20:44.240 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2023-09-23T09:20:44.240 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6882747db0c539a4e5112458cfa4d3819bf677e6f6528ea4b6d97351de2a66fa" 2023-09-23T09:20:44.240 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:20:44.240 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:20:44.438 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-09-23T09:20:44.439 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T09:20:44.439 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c", 2023-09-23T09:20:44.439 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2023-09-23T09:20:44.439 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6882747db0c539a4e5112458cfa4d3819bf677e6f6528ea4b6d97351de2a66fa" 2023-09-23T09:20:44.439 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2023-09-23T09:20:44.439 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-09-23T09:20:46.219 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-23T09:20:46.220 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T09:20:46.220 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c", 2023-09-23T09:20:46.220 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2023-09-23T09:20:46.220 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6882747db0c539a4e5112458cfa4d3819bf677e6f6528ea4b6d97351de2a66fa" 2023-09-23T09:20:46.220 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2023-09-23T09:20:46.220 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-23T09:20:46.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2023-09-23T09:20:46.276 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2023-09-23T09:20:46.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2023-09-23T09:20:46.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2023-09-23T09:20:46.372 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2023-09-23T09:20:46.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2023-09-23T09:20:46.428 INFO:tasks.cephadm:Writing seed config... 2023-09-23T09:20:46.445 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [client] debug client = 20 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-09-23T09:20:46.446 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-23T09:20:46.447 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-09-23T09:20:46.448 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-23T09:20:46.449 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-09-23T09:20:46.450 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-23T09:20:46.450 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-09-23T09:20:46.450 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:20:46.450 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-23T09:20:46.470 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 654e5bfa-59f2-11ee-8db4-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-09-23T09:20:46.470 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a.service 2023-09-23T09:20:46.512 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.x.service 2023-09-23T09:20:46.542 INFO:journalctl@ceph.mon.a.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:20:46.543 INFO:tasks.cephadm:Bootstrapping... 2023-09-23T09:20:46.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a -v bootstrap --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:20:46.571 INFO:journalctl@ceph.mgr.x.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:20:46.716 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-09-23T09:20:46.716 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a', '-v', 'bootstrap', '--fsid', '654e5bfa-59f2-11ee-8db4-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2023-09-23T09:20:46.865 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.1.1 2023-09-23T09:20:46.865 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-23T09:20:46.866 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2023-09-23T09:20:47.011 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.1.1 2023-09-23T09:20:47.011 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2023-09-23T09:20:47.011 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2023-09-23T09:20:47.018 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-23T09:20:47.018 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-23T09:20:47.024 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-23T09:20:47.024 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-09-23T09:20:47.030 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2023-09-23T09:20:47.036 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2023-09-23T09:20:47.036 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2023-09-23T09:20:47.036 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2023-09-23T09:20:47.182 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.1.1 2023-09-23T09:20:47.182 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-23T09:20:47.182 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2023-09-23T09:20:47.182 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2023-09-23T09:20:47.188 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-23T09:20:47.188 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-23T09:20:47.194 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-23T09:20:47.194 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-09-23T09:20:47.200 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2023-09-23T09:20:47.206 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2023-09-23T09:20:47.206 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2023-09-23T09:20:47.206 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2023-09-23T09:20:47.206 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:20:47.206 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 139694667229672 on /run/cephadm/654e5bfa-59f2-11ee-8db4-212e2dc638e7.lock 2023-09-23T09:20:47.206 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 139694667229672 acquired on /run/cephadm/654e5bfa-59f2-11ee-8db4-212e2dc638e7.lock 2023-09-23T09:20:47.207 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2023-09-23T09:20:47.207 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2023-09-23T09:20:47.207 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2023-09-23T09:20:47.210 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-09-23T09:20:47.210 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2023-09-23T09:20:47.213 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-23T09:20:47.213 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-09-23T09:20:47.213 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1777sec hoplimit 64 pref medium 2023-09-23T09:20:47.216 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-23T09:20:47.216 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-23T09:20:47.216 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-23T09:20:47.216 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-09-23T09:20:47.216 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-09-23T09:20:47.216 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-23T09:20:47.217 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2023-09-23T09:20:47.217 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2023-09-23T09:20:47.217 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-23T09:20:47.217 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-23T09:20:47.218 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T09:20:47.665 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c 2023-09-23T09:20:47.665 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T09:20:47.666 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2023-09-23T09:20:47.666 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:451a6a10e70d6de11ee942f3f27d2d4d1cdb7f417bfc327107b96bbd8c6f6af1 2023-09-23T09:20:47.666 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:3f4fd5c3c800c6da9ef94691df203a05a5bbbf32c483d09d620ec06eef42ea2b 2023-09-23T09:20:47.666 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c 2023-09-23T09:20:47.666 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-23T09:20:47.666 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Storing signatures 2023-09-23T09:20:48.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T09:20:48.411 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T09:20:48.411 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2023-09-23T09:20:49.059 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2023-09-23T09:20:49.059 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2023-09-23T09:20:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDxrQ5lyJ3IGRAANAWFGN6bpd0G9yn5roSxGw== 2023-09-23T09:20:50.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDyrQ5leyopCRAAqqzvRHfdIFS3ZJbWaYHpDg== 2023-09-23T09:20:51.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDyrQ5lWWMRMBAABSkZyeNC2xx6ZwAF9usa/A== 2023-09-23T09:20:51.105 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2023-09-23T09:20:51.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = pacific 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:20:51.754 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2023-09-23T09:20:52.401 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2023-09-23T09:20:52.526 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-23T09:20:52.623 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-23T09:20:52.743 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7.target → /etc/systemd/system/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7.target. 2023-09-23T09:20:52.743 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7.target → /etc/systemd/system/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7.target. 2023-09-23T09:20:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a 2023-09-23T09:20:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a.service: Unit ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a.service not loaded. 2023-09-23T09:20:53.036 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7.target.wants/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@.service. 2023-09-23T09:20:53.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:53 smithi045 systemd[1]: Starting Ceph mon.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:20:53.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:53 smithi045 podman[98081]: 2023-09-23T09:20:53.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:53 smithi045 bash[98081]: 52ab00e37a0eff06eff8ac0e795ded24ff9cdd2b0c56a30fb923357849e702f9 2023-09-23T09:20:53.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:53 smithi045 ceph-mon[98122]: mkfs 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:20:53.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:53 smithi045 ceph-mon[98122]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-23T09:20:53.646 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T09:20:53.646 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-09-23T09:20:53.653 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T09:20:53.653 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-09-23T09:20:53.654 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-09-23T09:20:53.654 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2023-09-23T09:20:53.654 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2023-09-23T09:20:53.654 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2023-09-23T09:20:53.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:53 smithi045 systemd[1]: Started Ceph mon.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:20:54.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:54 smithi045 ceph-mon[98122]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-23T09:20:54.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:54 smithi045 ceph-mon[98122]: monmap e1: 1 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0]} removed_ranks: {} 2023-09-23T09:20:54.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:54 smithi045 ceph-mon[98122]: fsmap 2023-09-23T09:20:54.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:54 smithi045 ceph-mon[98122]: osdmap e1: 0 total, 0 up, 0 in 2023-09-23T09:20:54.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:54 smithi045 ceph-mon[98122]: mgrmap e1: no daemons active 2023-09-23T09:20:54.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:54 smithi045 ceph-mon[98122]: from='client.? 172.21.15.45:0/3371203113' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-23T09:20:55.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2023-09-23T09:20:55.310 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.696868s) 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2023-09-23T09:20:55.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.117+0000 7f5730bb1700 1 Processor -- start 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.117+0000 7f5730bb1700 1 -- start start 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.117+0000 7f5730bb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.117+0000 7f5730bb1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c106450 con 0x7f572c105a60 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57468/0 (socket says 172.21.15.45:57468) 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572a59c700 1 -- 172.21.15.45:0/966972107 learned_addr learned my addr 172.21.15.45:0/966972107 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572a59c700 1 -- 172.21.15.45:0/966972107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c106780 con 0x7f572c105a60 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572a59c700 1 --2- 172.21.15.45:0/966972107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c105e80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f571c0098a0 tx=0x7f571c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5599cfea73dc77b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572959a700 1 -- 172.21.15.45:0/966972107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f571c004160 con 0x7f572c105a60 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572959a700 1 -- 172.21.15.45:0/966972107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c0042c0 con 0x7f572c105a60 2023-09-23T09:20:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.118+0000 7f572959a700 1 -- 172.21.15.45:0/966972107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f571c0045a0 con 0x7f572c105a60 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.119+0000 7f5730bb1700 1 -- 172.21.15.45:0/966972107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 msgr2=0x7f572c105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.119+0000 7f5730bb1700 1 --2- 172.21.15.45:0/966972107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c105e80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f571c0098a0 tx=0x7f571c009bb0 comp rx=0 tx=0).stop 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.119+0000 7f5730bb1700 1 -- 172.21.15.45:0/966972107 shutdown_connections 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.119+0000 7f5730bb1700 1 --2- 172.21.15.45:0/966972107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c105e80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.119+0000 7f5730bb1700 1 -- 172.21.15.45:0/966972107 >> 172.21.15.45:0/966972107 conn(0x7f572c1002c0 msgr2=0x7f572c1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.119+0000 7f5730bb1700 1 -- 172.21.15.45:0/966972107 shutdown_connections 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.119+0000 7f5730bb1700 1 -- 172.21.15.45:0/966972107 wait complete. 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.120+0000 7f5730bb1700 1 Processor -- start 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.120+0000 7f5730bb1700 1 -- start start 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.120+0000 7f5730bb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.120+0000 7f5730bb1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c19c170 con 0x7f572c105a60 2023-09-23T09:20:55.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.120+0000 7f572a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.120+0000 7f572a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57470/0 (socket says 172.21.15.45:57470) 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.120+0000 7f572a59c700 1 -- 172.21.15.45:0/3371203113 learned_addr learned my addr 172.21.15.45:0/3371203113 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f572a59c700 1 -- 172.21.15.45:0/3371203113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c009550 con 0x7f572c105a60 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f572a59c700 1 --2- 172.21.15.45:0/3371203113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c19bc30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f572c066c60 tx=0x7f571c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f571b7fe700 1 -- 172.21.15.45:0/3371203113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f571c011630 con 0x7f572c105a60 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f571b7fe700 1 -- 172.21.15.45:0/3371203113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c011790 con 0x7f572c105a60 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c19c370 con 0x7f572c105a60 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f571b7fe700 1 -- 172.21.15.45:0/3371203113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f571c011a30 con 0x7f572c105a60 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c19c790 con 0x7f572c105a60 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f571b7fe700 1 -- 172.21.15.45:0/3371203113 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f571c022070 con 0x7f572c105a60 2023-09-23T09:20:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.121+0000 7f571b7fe700 1 -- 172.21.15.45:0/3371203113 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f571c01c450 con 0x7f572c105a60 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.122+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572c191090 con 0x7f572c105a60 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.125+0000 7f571b7fe700 1 -- 172.21.15.45:0/3371203113 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f571c01b450 con 0x7f572c105a60 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.258+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f572c106a90 con 0x7f572c105a60 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.258+0000 7f571b7fe700 1 -- 172.21.15.45:0/3371203113 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f571c01b5f0 con 0x7f572c105a60 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.259+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 msgr2=0x7f572c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.259+0000 7f5730bb1700 1 --2- 172.21.15.45:0/3371203113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c19bc30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f572c066c60 tx=0x7f571c004750 comp rx=0 tx=0).stop 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.259+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 shutdown_connections 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.259+0000 7f5730bb1700 1 --2- 172.21.15.45:0/3371203113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f572c105a60 0x7f572c19bc30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.259+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 >> 172.21.15.45:0/3371203113 conn(0x7f572c1002c0 msgr2=0x7f572c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.259+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 shutdown_connections 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:54.259+0000 7f5730bb1700 1 -- 172.21.15.45:0/3371203113 wait complete. 2023-09-23T09:20:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2023-09-23T09:20:55.316 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2023-09-23T09:20:57.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:57 smithi045 ceph-mon[98122]: from='client.? 172.21.15.45:0/2605489658' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-23T09:20:57.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:57 smithi045 ceph-mon[98122]: from='client.? 172.21.15.45:0/2605489658' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-23T09:20:57.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:20:57.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2023-09-23T09:20:57.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:20:57.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2023-09-23T09:20:57.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-23T09:20:57.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-23T09:20:57.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-23T09:20:57.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.283+0000 7fcb38f2c700 1 Processor -- start 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.283+0000 7fcb38f2c700 1 -- start start 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.284+0000 7fcb38f2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.284+0000 7fcb38f2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34079e50 con 0x7fcb341070e0 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.284+0000 7fcb3259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.284+0000 7fcb3259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57472/0 (socket says 172.21.15.45:57472) 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.284+0000 7fcb3259c700 1 -- 172.21.15.45:0/2496591063 learned_addr learned my addr 172.21.15.45:0/2496591063 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.284+0000 7fcb3259c700 1 -- 172.21.15.45:0/2496591063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34109a10 con 0x7fcb341070e0 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.284+0000 7fcb3259c700 1 --2- 172.21.15.45:0/2496591063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341094d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcb240098a0 tx=0x7fcb24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9223c54b0c4eca20 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb3159a700 1 -- 172.21.15.45:0/2496591063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb24004160 con 0x7fcb341070e0 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb3159a700 1 -- 172.21.15.45:0/2496591063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb240042c0 con 0x7fcb341070e0 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb3159a700 1 -- 172.21.15.45:0/2496591063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb240045a0 con 0x7fcb341070e0 2023-09-23T09:20:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb38f2c700 1 -- 172.21.15.45:0/2496591063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 msgr2=0x7fcb341094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb38f2c700 1 --2- 172.21.15.45:0/2496591063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341094d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcb240098a0 tx=0x7fcb24009bb0 comp rx=0 tx=0).stop 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb38f2c700 1 -- 172.21.15.45:0/2496591063 shutdown_connections 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb38f2c700 1 --2- 172.21.15.45:0/2496591063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341094d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb38f2c700 1 -- 172.21.15.45:0/2496591063 >> 172.21.15.45:0/2496591063 conn(0x7fcb34100ad0 msgr2=0x7fcb34102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb38f2c700 1 -- 172.21.15.45:0/2496591063 shutdown_connections 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.285+0000 7fcb38f2c700 1 -- 172.21.15.45:0/2496591063 wait complete. 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.286+0000 7fcb38f2c700 1 Processor -- start 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.286+0000 7fcb38f2c700 1 -- start start 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.286+0000 7fcb38f2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.286+0000 7fcb38f2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34199f00 con 0x7fcb341070e0 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb3259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:20:57.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb3259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57474/0 (socket says 172.21.15.45:57474) 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb3259c700 1 -- 172.21.15.45:0/2605489658 learned_addr learned my addr 172.21.15.45:0/2605489658 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb3259c700 1 -- 172.21.15.45:0/2605489658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24009550 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb3259c700 1 --2- 172.21.15.45:0/2605489658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341999c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb34079290 tx=0x7fcb24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb24011630 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24011790 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb38f2c700 1 -- 172.21.15.45:0/2605489658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3419a100 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb24011a30 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.287+0000 7fcb38f2c700 1 -- 172.21.15.45:0/2605489658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3419a520 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.288+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcb24022070 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.288+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcb2401c790 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.289+0000 7fcb297fa700 1 -- 172.21.15.45:0/2605489658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb180052f0 con 0x7fcb341070e0 2023-09-23T09:20:57.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.291+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fcb2401bcf0 con 0x7fcb341070e0 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.418+0000 7fcb297fa700 1 -- 172.21.15.45:0/2605489658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fcb18002400 con 0x7fcb341070e0 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.420+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fcb2401b450 con 0x7fcb341070e0 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.421+0000 7fcb2b7fe700 1 -- 172.21.15.45:0/2605489658 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcb2401be90 con 0x7fcb341070e0 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.422+0000 7fcb297fa700 1 -- 172.21.15.45:0/2605489658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 msgr2=0x7fcb341999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.422+0000 7fcb297fa700 1 --2- 172.21.15.45:0/2605489658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341999c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb34079290 tx=0x7fcb24004750 comp rx=0 tx=0).stop 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.422+0000 7fcb297fa700 1 -- 172.21.15.45:0/2605489658 shutdown_connections 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.422+0000 7fcb297fa700 1 --2- 172.21.15.45:0/2605489658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb341070e0 0x7fcb341999c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.422+0000 7fcb297fa700 1 -- 172.21.15.45:0/2605489658 >> 172.21.15.45:0/2605489658 conn(0x7fcb34100ad0 msgr2=0x7fcb34102350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.422+0000 7fcb297fa700 1 -- 172.21.15.45:0/2605489658 shutdown_connections 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:57.422+0000 7fcb297fa700 1 -- 172.21.15.45:0/2605489658 wait complete. 2023-09-23T09:20:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2023-09-23T09:20:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8cf15df700 1 Processor -- start 2023-09-23T09:20:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8cf15df700 1 -- start start 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8cf15df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8cf15df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec105f40 con 0x7f8cec105550 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57476/0 (socket says 172.21.15.45:57476) 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8ceaffd700 1 -- 172.21.15.45:0/2542112120 learned_addr learned my addr 172.21.15.45:0/2542112120 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8ceaffd700 1 -- 172.21.15.45:0/2542112120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec106750 con 0x7f8cec105550 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.228+0000 7f8ceaffd700 1 --2- 172.21.15.45:0/2542112120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec105970 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098a0 tx=0x7f8ce0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c5640fb745933c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8ce9ffb700 1 -- 172.21.15.45:0/2542112120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ce0004160 con 0x7f8cec105550 2023-09-23T09:20:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8ce9ffb700 1 -- 172.21.15.45:0/2542112120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ce00042c0 con 0x7f8cec105550 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8ce9ffb700 1 -- 172.21.15.45:0/2542112120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ce0011a00 con 0x7f8cec105550 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8cf15df700 1 -- 172.21.15.45:0/2542112120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 msgr2=0x7f8cec105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8cf15df700 1 --2- 172.21.15.45:0/2542112120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec105970 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098a0 tx=0x7f8ce0009bb0 comp rx=0 tx=0).stop 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8cf15df700 1 -- 172.21.15.45:0/2542112120 shutdown_connections 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8cf15df700 1 --2- 172.21.15.45:0/2542112120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec105970 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8cf15df700 1 -- 172.21.15.45:0/2542112120 >> 172.21.15.45:0/2542112120 conn(0x7f8cec100b10 msgr2=0x7f8cec102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8cf15df700 1 -- 172.21.15.45:0/2542112120 shutdown_connections 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.229+0000 7f8cf15df700 1 -- 172.21.15.45:0/2542112120 wait complete. 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.230+0000 7f8cf15df700 1 Processor -- start 2023-09-23T09:20:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.230+0000 7f8cf15df700 1 -- start start 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.230+0000 7f8cf15df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.230+0000 7f8cf15df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec19a740 con 0x7f8cec105550 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57478/0 (socket says 172.21.15.45:57478) 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8ceaffd700 1 -- 172.21.15.45:0/4101746305 learned_addr learned my addr 172.21.15.45:0/4101746305 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8ceaffd700 1 -- 172.21.15.45:0/4101746305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce0009550 con 0x7f8cec105550 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8ceaffd700 1 --2- 172.21.15.45:0/4101746305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec19a200 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0006af0 tx=0x7f8ce0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8cd3fff700 1 -- 172.21.15.45:0/4101746305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ce0003a40 con 0x7f8cec105550 2023-09-23T09:20:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8cd3fff700 1 -- 172.21.15.45:0/4101746305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ce0003ba0 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec19a940 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8cd3fff700 1 -- 172.21.15.45:0/4101746305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ce001a700 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec19ad60 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8cd3fff700 1 -- 172.21.15.45:0/4101746305 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8ce001ac30 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.231+0000 7f8cd3fff700 1 -- 172.21.15.45:0/4101746305 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8ce0019700 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.232+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec062450 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.235+0000 7f8cd3fff700 1 -- 172.21.15.45:0/4101746305 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f8ce0024ad0 con 0x7f8cec105550 2023-09-23T09:20:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.362+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8cec109980 con 0x7f8cec105550 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.362+0000 7f8cd3fff700 1 -- 172.21.15.45:0/4101746305 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f8ce0024c70 con 0x7f8cec105550 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.363+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 msgr2=0x7f8cec19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.363+0000 7f8cf15df700 1 --2- 172.21.15.45:0/4101746305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec19a200 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0006af0 tx=0x7f8ce0011ad0 comp rx=0 tx=0).stop 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.363+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 shutdown_connections 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.363+0000 7f8cf15df700 1 --2- 172.21.15.45:0/4101746305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec105550 0x7f8cec19a200 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.363+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 >> 172.21.15.45:0/4101746305 conn(0x7f8cec100b10 msgr2=0x7f8cec079110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.363+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 shutdown_connections 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:20:58.363+0000 7f8cf15df700 1 -- 172.21.15.45:0/4101746305 wait complete. 2023-09-23T09:20:58.717 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2023-09-23T09:20:58.718 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:58 smithi045 ceph-mon[98122]: from='client.? 172.21.15.45:0/4101746305' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:20:59.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:58 smithi045 systemd[1]: Stopping Ceph mon.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:20:59.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:59 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-a[98098]: 2023-09-23T09:20:59.049+0000 7f41936af700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:20:59.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:59 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-a[98098]: 2023-09-23T09:20:59.049+0000 7f41936af700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-23T09:20:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:59 smithi045 bash[98460]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-a 2023-09-23T09:20:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:59 smithi045 systemd[1]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a.service: Succeeded. 2023-09-23T09:20:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:59 smithi045 systemd[1]: Stopped Ceph mon.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:20:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:59 smithi045 systemd[1]: Starting Ceph mon.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:21:00.085 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:20:59 smithi045 podman[98587]: 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 bash[98587]: 300e349c0765b1214cc10b188576a0a20770c2f29b892efa8e23d74b4985d834 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: set uid:gid to 167:167 (ceph:ceph) 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev), process ceph-mon, pid 7 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: pidfile_write: ignore empty --pid-file 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: load: jerasure load: lrc 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: RocksDB version: 7.9.2 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Git sha 0 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Compile date 2023-09-22 16:46:33 2023-09-23T09:21:00.283 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: DB SUMMARY 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: DB Session ID: 165XFHNXQ62SCP8NCYQY 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: CURRENT file: CURRENT 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: IDENTITY file: IDENTITY 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99278 ; 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.error_if_exists: 0 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.create_if_missing: 0 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.paranoid_checks: 1 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-23T09:21:00.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.env: 0x55a763be1da0 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.fs: PosixFileSystem 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.info_log: 0x55a7656fd5a0 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_file_opening_threads: 16 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.statistics: (nil) 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.use_fsync: 0 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_log_file_size: 0 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.keep_log_file_num: 1000 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.recycle_log_file_num: 0 2023-09-23T09:21:00.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.allow_fallocate: 1 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.allow_mmap_reads: 0 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.allow_mmap_writes: 0 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.use_direct_reads: 0 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.create_missing_column_families: 0 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.db_log_dir: 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.wal_dir: 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-23T09:21:00.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.advise_random_on_open: 1 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.db_write_buffer_size: 0 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.write_buffer_manager: 0x55a7656b54a0 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.rate_limiter: (nil) 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.wal_recovery_mode: 2 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.enable_thread_tracking: 0 2023-09-23T09:21:00.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.enable_pipelined_write: 0 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.unordered_write: 0 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.row_cache: None 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.wal_filter: None 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.allow_ingest_behind: 0 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.two_write_queues: 0 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.manual_wal_flush: 0 2023-09-23T09:21:00.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.wal_compression: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.atomic_flush: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.log_readahead_size: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.best_efforts_recovery: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.allow_data_in_errors: 0 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.db_host_id: __hostname__ 2023-09-23T09:21:00.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_background_jobs: 2 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_background_compactions: -1 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_subcompactions: 1 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_total_wal_size: 0 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-23T09:21:00.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_open_files: -1 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bytes_per_sync: 0 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_readahead_size: 0 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_background_flushes: -1 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Compression algorithms supported: 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kZSTD supported: 0 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kXpressCompression supported: 0 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kLZ4HCCompression supported: 1 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kZlibCompression supported: 1 2023-09-23T09:21:00.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kSnappyCompression supported: 1 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kLZ4Compression supported: 1 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: kBZip2Compression supported: 0 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.merge_operator: 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_filter: None 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_filter_factory: None 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.sst_partitioner_factory: None 2023-09-23T09:21:00.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a7656fd6a0) 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2023-09-23T09:21:00.293 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x55a7657531f0 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2023-09-23T09:21:00.294 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2023-09-23T09:21:00.295 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.write_buffer_size: 33554432 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_write_buffer_number: 2 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression: NoCompression 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression: Disabled 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.prefix_extractor: nullptr 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.num_levels: 7 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-23T09:21:00.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.level: 32767 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.strategy: 0 2023-09-23T09:21:00.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.enabled: false 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.target_file_size_base: 67108864 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-23T09:21:00.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-23T09:21:00.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-23T09:21:00.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.arena_block_size: 1048576 2023-09-23T09:21:00.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-23T09:21:00.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-23T09:21:00.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.disable_auto_compactions: 0 2023-09-23T09:21:00.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-23T09:21:00.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-23T09:21:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.table_properties_collectors: 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.inplace_update_support: 0 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.bloom_locality: 0 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.max_successive_merges: 0 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.paranoid_file_checks: 0 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.force_consistency_checks: 1 2023-09-23T09:21:00.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.report_bg_io_stats: 0 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.ttl: 2592000 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.enable_blob_files: false 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.min_blob_size: 0 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.blob_file_size: 268435456 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-23T09:21:00.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.blob_file_starting_level: 0 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ef3de4e2-d56c-42ec-8156-12bcdc432497 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460860053609, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460860054685, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94275, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 323, "table_properties": {"data_size": 92280, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13656, "raw_average_key_size": 49, "raw_value_size": 84832, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 274, "num_filter_entries": 274, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695460860, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ef3de4e2-d56c-42ec-8156-12bcdc432497", "db_session_id": "165XFHNXQ62SCP8NCYQY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460860054757, "job": 1, "event": "recovery_finished"} 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-23T09:21:00.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a7657f0000 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: DB pointer 0x55a7657dc000 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T09:21:00.305 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 93.90 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 93.90 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-23T09:21:00.306 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 28.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 28.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-23T09:21:00.307 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x55a7657531f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???) e1 preinit fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???).mds e1 new map 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???).mds e1 print_map 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2023-09-23T09:21:00.308 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expand map: {default=false} 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta from 'false' to 'false' 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expanded map: {default=false} 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expand map: {default=info} 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta from 'info' to 'info' 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expanded map: {default=info} 2023-09-23T09:21:00.309 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expand map: {default=daemon} 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expanded map: {default=daemon} 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expand map: {default=debug} 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta from 'debug' to 'debug' 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: expand_channel_meta expanded map: {default=debug} 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 systemd[1]: Started Ceph mon.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: monmap e1: 1 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0]} removed_ranks: {} 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: fsmap 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: osdmap e1: 0 total, 0 up, 0 in 2023-09-23T09:21:00.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:00 smithi045 ceph-mon[98627]: mgrmap e1: no daemons active 2023-09-23T09:21:01.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.537+0000 7f9220bdb700 1 Processor -- start 2023-09-23T09:21:01.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.537+0000 7f9220bdb700 1 -- start start 2023-09-23T09:21:01.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.537+0000 7f9220bdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c1094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.537+0000 7f9220bdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c079e50 con 0x7f921c1070e0 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.538+0000 7f921a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c1094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.538+0000 7f921a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c1094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57480/0 (socket says 172.21.15.45:57480) 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.538+0000 7f921a59c700 1 -- 172.21.15.45:0/917216525 learned_addr learned my addr 172.21.15.45:0/917216525 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.538+0000 7f921a59c700 1 -- 172.21.15.45:0/917216525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c109a10 con 0x7f921c1070e0 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f921a59c700 1 --2- 172.21.15.45:0/917216525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c1094d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f92040098a0 tx=0x7f9204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec26ecdc2e780b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f921959a700 1 -- 172.21.15.45:0/917216525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9204004160 con 0x7f921c1070e0 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f921959a700 1 -- 172.21.15.45:0/917216525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92040042c0 con 0x7f921c1070e0 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f921959a700 1 -- 172.21.15.45:0/917216525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9204011a00 con 0x7f921c1070e0 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f9220bdb700 1 -- 172.21.15.45:0/917216525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 msgr2=0x7f921c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:01.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f9220bdb700 1 --2- 172.21.15.45:0/917216525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c1094d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f92040098a0 tx=0x7f9204009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f9220bdb700 1 -- 172.21.15.45:0/917216525 shutdown_connections 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f9220bdb700 1 --2- 172.21.15.45:0/917216525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c1094d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f9220bdb700 1 -- 172.21.15.45:0/917216525 >> 172.21.15.45:0/917216525 conn(0x7f921c100ad0 msgr2=0x7f921c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f9220bdb700 1 -- 172.21.15.45:0/917216525 shutdown_connections 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.539+0000 7f9220bdb700 1 -- 172.21.15.45:0/917216525 wait complete. 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.540+0000 7f9220bdb700 1 Processor -- start 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.540+0000 7f9220bdb700 1 -- start start 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.540+0000 7f9220bdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f9220bdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c106f80 con 0x7f921c1070e0 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f921a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f921a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57482/0 (socket says 172.21.15.45:57482) 2023-09-23T09:21:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f921a59c700 1 -- 172.21.15.45:0/2447333424 learned_addr learned my addr 172.21.15.45:0/2447333424 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f921a59c700 1 -- 172.21.15.45:0/2447333424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204009550 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f921a59c700 1 --2- 172.21.15.45:0/2447333424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c106a40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9204006af0 tx=0x7f9204011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f92137fe700 1 -- 172.21.15.45:0/2447333424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9204003a40 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f92137fe700 1 -- 172.21.15.45:0/2447333424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9204003ba0 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c105090 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f92137fe700 1 -- 172.21.15.45:0/2447333424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f920401a6e0 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c1054b0 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.541+0000 7f92137fe700 1 -- 172.21.15.45:0/2447333424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f920401ab60 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.542+0000 7f92137fe700 1 -- 172.21.15.45:0/2447333424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9204019700 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.542+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921c10a7b0 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.545+0000 7f92137fe700 1 -- 172.21.15.45:0/2447333424 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f920402dde0 con 0x7f921c1070e0 2023-09-23T09:21:01.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.671+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f921c1059a0 con 0x7f921c1070e0 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.673+0000 7f92137fe700 1 -- 172.21.15.45:0/2447333424 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9204024a60 con 0x7f921c1070e0 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.673+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 msgr2=0x7f921c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.673+0000 7f9220bdb700 1 --2- 172.21.15.45:0/2447333424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c106a40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9204006af0 tx=0x7f9204011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.673+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 shutdown_connections 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.674+0000 7f9220bdb700 1 --2- 172.21.15.45:0/2447333424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f921c1070e0 0x7f921c106a40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.674+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 >> 172.21.15.45:0/2447333424 conn(0x7f921c100ad0 msgr2=0x7f921c1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.674+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 shutdown_connections 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:00.674+0000 7f9220bdb700 1 -- 172.21.15.45:0/2447333424 wait complete. 2023-09-23T09:21:01.041 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-23T09:21:01.056 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:21:01.056 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2023-09-23T09:21:01.057 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-23T09:21:01.057 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-23T09:21:01.272 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.x 2023-09-23T09:21:01.272 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.x.service: Unit ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.x.service not loaded. 2023-09-23T09:21:01.366 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7.target.wants/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@.service. 2023-09-23T09:21:01.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:01 smithi045 systemd[1]: Starting Ceph mgr.x for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:21:01.961 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:01 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2447333424' entity='client.admin' 2023-09-23T09:21:01.962 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:01 smithi045 podman[98852]: 2023-09-23T09:21:01.962 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:01 smithi045 bash[98852]: 246ececf6b181b634ae0cd95d77e0694b5a6c701a86028f8945c9f6c1f3dfcff 2023-09-23T09:21:01.975 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T09:21:01.976 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-09-23T09:21:01.984 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T09:21:01.984 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-09-23T09:21:01.984 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-09-23T09:21:01.984 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2023-09-23T09:21:01.994 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T09:21:01.994 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-09-23T09:21:02.002 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T09:21:02.003 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-09-23T09:21:02.003 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-09-23T09:21:02.003 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-23T09:21:02.003 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2023-09-23T09:21:02.003 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2023-09-23T09:21:02.215 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:01 smithi045 systemd[1]: Started Ceph mgr.x for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:21:02.215 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:02 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:02.087+0000 7fe02f41f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:21:02.215 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:02 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:02.184+0000 7fe02f41f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:21:02.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:02 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:02.300+0000 7fe02f41f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:21:02.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:02 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:02.415+0000 7fe02f41f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:21:02.930 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:02 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3376356326' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-23T09:21:02.931 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:02 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:02.627+0000 7fe02f41f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:21:02.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:21:02.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:02.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:21:02.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-09-23T09:21:02.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-09-23T09:21:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-23T09:21:02.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-09-23T09:20:53.564461+0000", 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.497+0000 7f3e5c781700 1 Processor -- start 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.497+0000 7f3e5c781700 1 -- start start 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.497+0000 7f3e5c781700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e541094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.497+0000 7f3e5c781700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54079eb0 con 0x7f3e541070c0 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.497+0000 7f3e5a51d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e541094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:02.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.497+0000 7f3e5a51d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e541094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57488/0 (socket says 172.21.15.45:57488) 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.497+0000 7f3e5a51d700 1 -- 172.21.15.45:0/1425259892 learned_addr learned my addr 172.21.15.45:0/1425259892 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.498+0000 7f3e5a51d700 1 -- 172.21.15.45:0/1425259892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e541099f0 con 0x7f3e541070c0 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.498+0000 7f3e5a51d700 1 --2- 172.21.15.45:0/1425259892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e541094b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e500098a0 tx=0x7f3e50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c007d0af8227d35 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.498+0000 7f3e5951b700 1 -- 172.21.15.45:0/1425259892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e50004160 con 0x7f3e541070c0 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.498+0000 7f3e5951b700 1 -- 172.21.15.45:0/1425259892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e500042c0 con 0x7f3e541070c0 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.498+0000 7f3e5951b700 1 -- 172.21.15.45:0/1425259892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e50011a00 con 0x7f3e541070c0 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 -- 172.21.15.45:0/1425259892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 msgr2=0x7f3e541094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 --2- 172.21.15.45:0/1425259892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e541094b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e500098a0 tx=0x7f3e50009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 -- 172.21.15.45:0/1425259892 shutdown_connections 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 --2- 172.21.15.45:0/1425259892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e541094b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 -- 172.21.15.45:0/1425259892 >> 172.21.15.45:0/1425259892 conn(0x7f3e54100b10 msgr2=0x7f3e54102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:02.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 -- 172.21.15.45:0/1425259892 shutdown_connections 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 -- 172.21.15.45:0/1425259892 wait complete. 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 Processor -- start 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.499+0000 7f3e5c781700 1 -- start start 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5c781700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e54106a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5c781700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54106f60 con 0x7f3e541070c0 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5a51d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e54106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5a51d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e54106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57490/0 (socket says 172.21.15.45:57490) 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5a51d700 1 -- 172.21.15.45:0/3376356326 learned_addr learned my addr 172.21.15.45:0/3376356326 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5a51d700 1 -- 172.21.15.45:0/3376356326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009550 con 0x7f3e541070c0 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5a51d700 1 --2- 172.21.15.45:0/3376356326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e54106a20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009fb0 tx=0x7f3e50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e4b7fe700 1 -- 172.21.15.45:0/3376356326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e50003930 con 0x7f3e541070c0 2023-09-23T09:21:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e4b7fe700 1 -- 172.21.15.45:0/3376356326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e50003a90 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e54105070 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.500+0000 7f3e4b7fe700 1 -- 172.21.15.45:0/3376356326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e50019610 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.501+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e54105490 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.501+0000 7f3e4b7fe700 1 -- 172.21.15.45:0/3376356326 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3e50019ba0 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.501+0000 7f3e4b7fe700 1 -- 172.21.15.45:0/3376356326 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e500243f0 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.502+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5410a7f0 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.504+0000 7f3e4b7fe700 1 -- 172.21.15.45:0/3376356326 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f3e50003c00 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.638+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3e541056d0 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.638+0000 7f3e4b7fe700 1 -- 172.21.15.45:0/3376356326 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3e5001a9d0 con 0x7f3e541070c0 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.639+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 msgr2=0x7f3e54106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.639+0000 7f3e5c781700 1 --2- 172.21.15.45:0/3376356326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e54106a20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009fb0 tx=0x7f3e50011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.639+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 shutdown_connections 2023-09-23T09:21:02.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.639+0000 7f3e5c781700 1 --2- 172.21.15.45:0/3376356326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541070c0 0x7f3e54106a20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:02.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.639+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 >> 172.21.15.45:0/3376356326 conn(0x7f3e54100b10 msgr2=0x7f3e541017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:02.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.639+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 shutdown_connections 2023-09-23T09:21:02.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:02.640+0000 7f3e5c781700 1 -- 172.21.15.45:0/3376356326 wait complete. 2023-09-23T09:21:02.971 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2023-09-23T09:21:03.276 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:03 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:03.117+0000 7fe02f41f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:21:03.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:03 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:03.274+0000 7fe02f41f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:21:03.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:03 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:03.382+0000 7fe02f41f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:21:03.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:03 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:03.497+0000 7fe02f41f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:21:04.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:04 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:04.010+0000 7fe02f41f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:21:05.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:04 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:04.805+0000 7fe02f41f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:21:05.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:04 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:04.903+0000 7fe02f41f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-23T09:21:05.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-23T09:21:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-23T09:21:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-23T09:21:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-23T09:21:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-23T09:21:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-23T09:21:05.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-23T09:21:05.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-23T09:21:05.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-23T09:21:05.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:05.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-09-23T09:20:53.564461+0000", 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.434+0000 7f79be6d4700 1 Processor -- start 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.434+0000 7f79be6d4700 1 -- start start 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.435+0000 7f79be6d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b8105350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.435+0000 7f79be6d4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b8079e30 con 0x7f79b8104f30 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.435+0000 7f79b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b8105350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.435+0000 7f79b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b8105350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57492/0 (socket says 172.21.15.45:57492) 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.435+0000 7f79b7fff700 1 -- 172.21.15.45:0/1667397215 learned_addr learned my addr 172.21.15.45:0/1667397215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.435+0000 7f79b7fff700 1 -- 172.21.15.45:0/1667397215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b8105890 con 0x7f79b8104f30 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.435+0000 7f79b7fff700 1 --2- 172.21.15.45:0/1667397215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b8105350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79a80098a0 tx=0x7f79a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b24787ee2659d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79b6ffd700 1 -- 172.21.15.45:0/1667397215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79a8004160 con 0x7f79b8104f30 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79b6ffd700 1 -- 172.21.15.45:0/1667397215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79a80042c0 con 0x7f79b8104f30 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79b6ffd700 1 -- 172.21.15.45:0/1667397215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79a8011a00 con 0x7f79b8104f30 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79be6d4700 1 -- 172.21.15.45:0/1667397215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 msgr2=0x7f79b8105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79be6d4700 1 --2- 172.21.15.45:0/1667397215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b8105350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79a80098a0 tx=0x7f79a8009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79be6d4700 1 -- 172.21.15.45:0/1667397215 shutdown_connections 2023-09-23T09:21:05.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79be6d4700 1 --2- 172.21.15.45:0/1667397215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b8105350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79be6d4700 1 -- 172.21.15.45:0/1667397215 >> 172.21.15.45:0/1667397215 conn(0x7f79b8100b10 msgr2=0x7f79b8102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79be6d4700 1 -- 172.21.15.45:0/1667397215 shutdown_connections 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.436+0000 7f79be6d4700 1 -- 172.21.15.45:0/1667397215 wait complete. 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.437+0000 7f79be6d4700 1 Processor -- start 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.437+0000 7f79be6d4700 1 -- start start 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.437+0000 7f79be6d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.437+0000 7f79be6d4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b819c910 con 0x7f79b8104f30 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b819c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57494/0 (socket says 172.21.15.45:57494) 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b7fff700 1 -- 172.21.15.45:0/2804598732 learned_addr learned my addr 172.21.15.45:0/2804598732 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b7fff700 1 -- 172.21.15.45:0/2804598732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8009550 con 0x7f79b8104f30 2023-09-23T09:21:05.921 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b7fff700 1 --2- 172.21.15.45:0/2804598732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b819c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006af0 tx=0x7f79a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b57fa700 1 -- 172.21.15.45:0/2804598732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79a8003a40 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b57fa700 1 -- 172.21.15.45:0/2804598732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79a8003ba0 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b57fa700 1 -- 172.21.15.45:0/2804598732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79a801a6e0 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b819cb10 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b819cf30 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b57fa700 1 -- 172.21.15.45:0/2804598732 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f79a801ab60 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.438+0000 7f79b57fa700 1 -- 172.21.15.45:0/2804598732 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f79a8019700 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.439+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b8195c60 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.442+0000 7f79b57fa700 1 -- 172.21.15.45:0/2804598732 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f79a802dde0 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.574+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f79b8106700 con 0x7f79b8104f30 2023-09-23T09:21:05.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.574+0000 7f79b57fa700 1 -- 172.21.15.45:0/2804598732 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f79a8024a60 con 0x7f79b8104f30 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.575+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 msgr2=0x7f79b819c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.575+0000 7f79be6d4700 1 --2- 172.21.15.45:0/2804598732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b819c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006af0 tx=0x7f79a8011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.575+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 shutdown_connections 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.575+0000 7f79be6d4700 1 --2- 172.21.15.45:0/2804598732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b8104f30 0x7f79b819c3d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.575+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 >> 172.21.15.45:0/2804598732 conn(0x7f79b8100b10 msgr2=0x7f79b8190af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.575+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 shutdown_connections 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:05.575+0000 7f79be6d4700 1 -- 172.21.15.45:0/2804598732 wait complete. 2023-09-23T09:21:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2023-09-23T09:21:06.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:06 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2804598732' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-23T09:21:06.249 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:05 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:05.913+0000 7fe02f41f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:21:06.249 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:06.025+0000 7fe02f41f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:21:06.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:06.246+0000 7fe02f41f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:21:06.851 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:06.597+0000 7fe02f41f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:21:07.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:06.850+0000 7fe02f41f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:21:07.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:07 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:07.032+0000 7fe02f41f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:21:07.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:07 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:07.363+0000 7fe02f41f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:21:08.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:08.189+0000 7fe02f41f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:21:08.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:08.348+0000 7fe02f41f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:21:09.011 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:08.594+0000 7fe02f41f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:21:09.011 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:08.732+0000 7fe02f41f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:21:09.011 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:09 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:09.009+0000 7fe02f41f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:21:10.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:10 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:10.066+0000 7fe02f41f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:21:11.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: Activating manager daemon x 2023-09-23T09:21:11.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: mgrmap e2: x(active, starting, since 0.0028173s) 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: Manager daemon x is now available 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' 2023-09-23T09:21:11.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='mgr.14100 172.21.15.45:0/361316349' entity='mgr.x' 2023-09-23T09:21:11.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:11 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2965808510' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-23T09:21:12.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:12 smithi045 ceph-mon[98627]: mgrmap e3: x(active, since 1.00716s) 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-23T09:21:13.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-23T09:21:13.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-23T09:21:13.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:13.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-23T09:21:13.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:13.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-23T09:21:13.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-23T09:21:13.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-23T09:21:13.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-23T09:21:13.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:13.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-09-23T09:20:53.564461+0000", 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.184+0000 7f2798504700 1 Processor -- start 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.184+0000 7f2798504700 1 -- start start 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.184+0000 7f2798504700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f2790109510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.184+0000 7f2798504700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2790079e50 con 0x7f2790107120 2023-09-23T09:21:13.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.184+0000 7f27962a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f2790109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.184+0000 7f27962a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f2790109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57510/0 (socket says 172.21.15.45:57510) 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.184+0000 7f27962a0700 1 -- 172.21.15.45:0/1326329703 learned_addr learned my addr 172.21.15.45:0/1326329703 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.185+0000 7f27962a0700 1 -- 172.21.15.45:0/1326329703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790109a50 con 0x7f2790107120 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.185+0000 7f27962a0700 1 --2- 172.21.15.45:0/1326329703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f2790109510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f278c0098a0 tx=0x7f278c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf6017037dac36b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.185+0000 7f279529e700 1 -- 172.21.15.45:0/1326329703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f278c004160 con 0x7f2790107120 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.185+0000 7f279529e700 1 -- 172.21.15.45:0/1326329703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f278c0042c0 con 0x7f2790107120 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.185+0000 7f279529e700 1 -- 172.21.15.45:0/1326329703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f278c011a00 con 0x7f2790107120 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 -- 172.21.15.45:0/1326329703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 msgr2=0x7f2790109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 --2- 172.21.15.45:0/1326329703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f2790109510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f278c0098a0 tx=0x7f278c009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 -- 172.21.15.45:0/1326329703 shutdown_connections 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 --2- 172.21.15.45:0/1326329703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f2790109510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:13.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 -- 172.21.15.45:0/1326329703 >> 172.21.15.45:0/1326329703 conn(0x7f2790100b10 msgr2=0x7f2790102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 -- 172.21.15.45:0/1326329703 shutdown_connections 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 -- 172.21.15.45:0/1326329703 wait complete. 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.186+0000 7f2798504700 1 Processor -- start 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f2798504700 1 -- start start 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f2798504700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f279019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f2798504700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279019a6b0 con 0x7f2790107120 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27962a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f279019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27962a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f279019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57512/0 (socket says 172.21.15.45:57512) 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27962a0700 1 -- 172.21.15.45:0/2965808510 learned_addr learned my addr 172.21.15.45:0/2965808510 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27962a0700 1 -- 172.21.15.45:0/2965808510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c009550 con 0x7f2790107120 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27962a0700 1 --2- 172.21.15.45:0/2965808510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f279019a170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f278c009fb0 tx=0x7f278c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:13.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27877fe700 1 -- 172.21.15.45:0/2965808510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f278c003930 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27877fe700 1 -- 172.21.15.45:0/2965808510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f278c003a90 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279019a8b0 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.187+0000 7f27877fe700 1 -- 172.21.15.45:0/2965808510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f278c019610 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.188+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279019acd0 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.188+0000 7f27877fe700 1 -- 172.21.15.45:0/2965808510 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 2) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f278c01a9d0 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.189+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2790062450 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.189+0000 7f27877fe700 1 -- 172.21.15.45:0/2965808510 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f278c04c2e0 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.191+0000 7f27877fe700 1 -- 172.21.15.45:0/2965808510 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f278c003c00 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.325+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f279010a7f0 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.325+0000 7f27877fe700 1 -- 172.21.15.45:0/2965808510 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f278c023390 con 0x7f2790107120 2023-09-23T09:21:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.326+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 msgr2=0x7f279019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:13.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.326+0000 7f2798504700 1 --2- 172.21.15.45:0/2965808510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f279019a170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f278c009fb0 tx=0x7f278c011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:13.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.326+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 shutdown_connections 2023-09-23T09:21:13.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.326+0000 7f2798504700 1 --2- 172.21.15.45:0/2965808510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2790107120 0x7f279019a170 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:13.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.326+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 >> 172.21.15.45:0/2965808510 conn(0x7f2790100b10 msgr2=0x7f2790101790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:13.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.327+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 shutdown_connections 2023-09-23T09:21:13.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:10.327+0000 7f2798504700 1 -- 172.21.15.45:0/2965808510 wait complete. 2023-09-23T09:21:13.184 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2023-09-23T09:21:14.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:14 smithi045 ceph-mon[98627]: mgrmap e4: x(active, since 3s) 2023-09-23T09:21:17.676 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:17 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3248743644' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-23T09:21:17.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:21:17.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:17.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:21:17.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-09-23T09:21:17.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:17.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-23T09:21:17.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-23T09:21:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-23T09:21:17.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-09-23T09:20:53.564461+0000", 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-23T09:21:17.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.808+0000 7f3828559700 1 Processor -- start 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.808+0000 7f3828559700 1 -- start start 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.809+0000 7f3828559700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f3820105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.809+0000 7f3828559700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820105f40 con 0x7f3820105550 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.809+0000 7f38262f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f3820105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.809+0000 7f38262f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f3820105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57524/0 (socket says 172.21.15.45:57524) 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.809+0000 7f38262f5700 1 -- 172.21.15.45:0/2948984457 learned_addr learned my addr 172.21.15.45:0/2948984457 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.809+0000 7f38262f5700 1 -- 172.21.15.45:0/2948984457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3820106750 con 0x7f3820105550 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.809+0000 7f38262f5700 1 --2- 172.21.15.45:0/2948984457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f3820105970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f38100098a0 tx=0x7f3810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2887896e9281f250 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:17.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f38252f3700 1 -- 172.21.15.45:0/2948984457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810004160 con 0x7f3820105550 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f38252f3700 1 -- 172.21.15.45:0/2948984457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38100042c0 con 0x7f3820105550 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f38252f3700 1 -- 172.21.15.45:0/2948984457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810011a00 con 0x7f3820105550 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f3828559700 1 -- 172.21.15.45:0/2948984457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 msgr2=0x7f3820105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f3828559700 1 --2- 172.21.15.45:0/2948984457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f3820105970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f38100098a0 tx=0x7f3810009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f3828559700 1 -- 172.21.15.45:0/2948984457 shutdown_connections 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f3828559700 1 --2- 172.21.15.45:0/2948984457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f3820105970 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.810+0000 7f3828559700 1 -- 172.21.15.45:0/2948984457 >> 172.21.15.45:0/2948984457 conn(0x7f3820100b10 msgr2=0x7f3820102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.811+0000 7f3828559700 1 -- 172.21.15.45:0/2948984457 shutdown_connections 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.811+0000 7f3828559700 1 -- 172.21.15.45:0/2948984457 wait complete. 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.811+0000 7f3828559700 1 Processor -- start 2023-09-23T09:21:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.811+0000 7f3828559700 1 -- start start 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f3828559700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f38201990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f3828559700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820199600 con 0x7f3820105550 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38262f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f38201990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38262f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f38201990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57526/0 (socket says 172.21.15.45:57526) 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38262f5700 1 -- 172.21.15.45:0/3248743644 learned_addr learned my addr 172.21.15.45:0/3248743644 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38262f5700 1 -- 172.21.15.45:0/3248743644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810009550 con 0x7f3820105550 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38262f5700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f38201990c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3810009fb0 tx=0x7f3810011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38177fe700 1 -- 172.21.15.45:0/3248743644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810003930 con 0x7f3820105550 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3820199800 con 0x7f3820105550 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38177fe700 1 -- 172.21.15.45:0/3248743644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3810003a90 con 0x7f3820105550 2023-09-23T09:21:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f38177fe700 1 -- 172.21.15.45:0/3248743644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38100195e0 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.812+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3820199c20 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.813+0000 7f38177fe700 1 -- 172.21.15.45:0/3248743644 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f381001a9d0 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.814+0000 7f38177fe700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f380c0388a0 0x7f380c03ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.814+0000 7f38177fe700 1 -- 172.21.15.45:0/3248743644 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f381004c670 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.814+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3820062450 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.814+0000 7f3825af4700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f380c0388a0 0x7f380c03ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.814+0000 7f3825af4700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f380c0388a0 0x7f380c03ad60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:16.819+0000 7f38177fe700 1 -- 172.21.15.45:0/3248743644 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f38100198a0 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.341+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3820109950 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.341+0000 7f38177fe700 1 -- 172.21.15.45:0/3248743644 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f38100172b0 con 0x7f3820105550 2023-09-23T09:21:17.745 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f380c0388a0 msgr2=0x7f380c03ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f380c0388a0 0x7f380c03ad60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 msgr2=0x7f38201990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f38201990c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3810009fb0 tx=0x7f3810011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 shutdown_connections 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f380c0388a0 0x7f380c03ad60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 --2- 172.21.15.45:0/3248743644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820105550 0x7f38201990c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 >> 172.21.15.45:0/3248743644 conn(0x7f3820100b10 msgr2=0x7f38201017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 shutdown_connections 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:17.343+0000 7f3828559700 1 -- 172.21.15.45:0/3248743644 wait complete. 2023-09-23T09:21:17.746 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2023-09-23T09:21:18.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:18 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1206027114' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-23T09:21:18.982 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:21:18.982 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2023-09-23T09:21:18.983 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.206+0000 7fcab19cd700 1 Processor -- start 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.206+0000 7fcab19cd700 1 -- start start 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.207+0000 7fcab19cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac105330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.207+0000 7fcab19cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac079e50 con 0x7fcaac104f10 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.207+0000 7fcaaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac105330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.207+0000 7fcaaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac105330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57530/0 (socket says 172.21.15.45:57530) 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.207+0000 7fcaaaffd700 1 -- 172.21.15.45:0/534067845 learned_addr learned my addr 172.21.15.45:0/534067845 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:18.984 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.207+0000 7fcaaaffd700 1 -- 172.21.15.45:0/534067845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac105870 con 0x7fcaac104f10 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.207+0000 7fcaaaffd700 1 --2- 172.21.15.45:0/534067845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac105330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0098a0 tx=0x7fca9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c96a83cbbc6428 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcaa9ffb700 1 -- 172.21.15.45:0/534067845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca9c004160 con 0x7fcaac104f10 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcaa9ffb700 1 -- 172.21.15.45:0/534067845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca9c0042c0 con 0x7fcaac104f10 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcaa9ffb700 1 -- 172.21.15.45:0/534067845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca9c011a00 con 0x7fcaac104f10 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcab19cd700 1 -- 172.21.15.45:0/534067845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 msgr2=0x7fcaac105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcab19cd700 1 --2- 172.21.15.45:0/534067845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac105330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0098a0 tx=0x7fca9c009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcab19cd700 1 -- 172.21.15.45:0/534067845 shutdown_connections 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcab19cd700 1 --2- 172.21.15.45:0/534067845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac105330 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcab19cd700 1 -- 172.21.15.45:0/534067845 >> 172.21.15.45:0/534067845 conn(0x7fcaac100b10 msgr2=0x7fcaac102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcab19cd700 1 -- 172.21.15.45:0/534067845 shutdown_connections 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.208+0000 7fcab19cd700 1 -- 172.21.15.45:0/534067845 wait complete. 2023-09-23T09:21:18.985 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.209+0000 7fcab19cd700 1 Processor -- start 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.209+0000 7fcab19cd700 1 -- start start 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.209+0000 7fcab19cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.209+0000 7fcab19cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac19c900 con 0x7fcaac104f10 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.209+0000 7fcaaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.209+0000 7fcaaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac19c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57532/0 (socket says 172.21.15.45:57532) 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.209+0000 7fcaaaffd700 1 -- 172.21.15.45:0/1206027114 learned_addr learned my addr 172.21.15.45:0/1206027114 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.210+0000 7fcaaaffd700 1 -- 172.21.15.45:0/1206027114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c009550 con 0x7fcaac104f10 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.210+0000 7fcaaaffd700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac19c3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fca9c006af0 tx=0x7fca9c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.210+0000 7fcab09cb700 1 -- 172.21.15.45:0/1206027114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca9c003a40 con 0x7fcaac104f10 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.210+0000 7fcab09cb700 1 -- 172.21.15.45:0/1206027114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca9c003ba0 con 0x7fcaac104f10 2023-09-23T09:21:18.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.210+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac19cb00 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.210+0000 7fcab09cb700 1 -- 172.21.15.45:0/1206027114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca9c01a6e0 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.210+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac19cf20 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.211+0000 7fcab09cb700 1 -- 172.21.15.45:0/1206027114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45631+0+0 (secure 0 0 0) 0x7fca9c024a60 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.211+0000 7fcab09cb700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fca980388f0 0x7fca9803adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.211+0000 7fcab09cb700 1 -- 172.21.15.45:0/1206027114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fca9c04c890 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.211+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac195c70 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.211+0000 7fcaaa7fc700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fca980388f0 0x7fca9803adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.212+0000 7fcaaa7fc700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fca980388f0 0x7fca9803adb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca94006fd0 tx=0x7fca94006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.216+0000 7fcab09cb700 1 -- 172.21.15.45:0/1206027114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fca9c024430 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.579+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fcaac062450 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.580+0000 7fcab09cb700 1 -- 172.21.15.45:0/1206027114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fca9c018350 con 0x7fcaac104f10 2023-09-23T09:21:18.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fca980388f0 msgr2=0x7fca9803adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fca980388f0 0x7fca9803adb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca94006fd0 tx=0x7fca94006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 msgr2=0x7fcaac19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac19c3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fca9c006af0 tx=0x7fca9c011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 shutdown_connections 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fca980388f0 0x7fca9803adb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 --2- 172.21.15.45:0/1206027114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaac104f10 0x7fcaac19c3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 >> 172.21.15.45:0/1206027114 conn(0x7fcaac100b10 msgr2=0x7fcaac190ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 shutdown_connections 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:18.582+0000 7fcab19cd700 1 -- 172.21.15.45:0/1206027114 wait complete. 2023-09-23T09:21:18.988 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2023-09-23T09:21:20.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/364804031' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-23T09:21:20.291 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: ignoring --setuser ceph since I am not root 2023-09-23T09:21:20.291 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: ignoring --setgroup ceph since I am not root 2023-09-23T09:21:20.291 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:20.192+0000 7f627f295200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:21:20.291 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:20.288+0000 7f627f295200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:21:20.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.472+0000 7f93c015f700 1 Processor -- start 2023-09-23T09:21:20.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.472+0000 7f93c015f700 1 -- start start 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.472+0000 7f93c015f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b81094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.472+0000 7f93c015f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8079eb0 con 0x7f93b81070c0 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bdefb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b81094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bdefb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b81094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57536/0 (socket says 172.21.15.45:57536) 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bdefb700 1 -- 172.21.15.45:0/653702946 learned_addr learned my addr 172.21.15.45:0/653702946 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bdefb700 1 -- 172.21.15.45:0/653702946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b81099f0 con 0x7f93b81070c0 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bdefb700 1 --2- 172.21.15.45:0/653702946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b81094b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93a80098a0 tx=0x7f93a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e72d41eafbea5da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bcef9700 1 -- 172.21.15.45:0/653702946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a8004160 con 0x7f93b81070c0 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bcef9700 1 -- 172.21.15.45:0/653702946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93a80042c0 con 0x7f93b81070c0 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.473+0000 7f93bcef9700 1 -- 172.21.15.45:0/653702946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a8011a00 con 0x7f93b81070c0 2023-09-23T09:21:20.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.474+0000 7f93c015f700 1 -- 172.21.15.45:0/653702946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 msgr2=0x7f93b81094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.474+0000 7f93c015f700 1 --2- 172.21.15.45:0/653702946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b81094b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93a80098a0 tx=0x7f93a8009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.474+0000 7f93c015f700 1 -- 172.21.15.45:0/653702946 shutdown_connections 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.474+0000 7f93c015f700 1 --2- 172.21.15.45:0/653702946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b81094b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.474+0000 7f93c015f700 1 -- 172.21.15.45:0/653702946 >> 172.21.15.45:0/653702946 conn(0x7f93b8100b10 msgr2=0x7f93b8102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.474+0000 7f93c015f700 1 -- 172.21.15.45:0/653702946 shutdown_connections 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.474+0000 7f93c015f700 1 -- 172.21.15.45:0/653702946 wait complete. 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93c015f700 1 Processor -- start 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93c015f700 1 -- start start 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93c015f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b8106a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93c015f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8106f60 con 0x7f93b81070c0 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93bdefb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b8106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:20.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93bdefb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b8106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57538/0 (socket says 172.21.15.45:57538) 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93bdefb700 1 -- 172.21.15.45:0/364804031 learned_addr learned my addr 172.21.15.45:0/364804031 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93bdefb700 1 -- 172.21.15.45:0/364804031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a8009550 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93bdefb700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b8106a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009fb0 tx=0x7f93a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.475+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a8003930 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.476+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93a8003a90 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.476+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b8105070 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.476+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a8019610 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.476+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b8105490 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.477+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f93a801a9d0 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.477+0000 7f93aeffd700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f93a4038940 0x7f93a403ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.477+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f93a804c640 con 0x7f93b81070c0 2023-09-23T09:21:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.477+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b810a7f0 con 0x7f93b81070c0 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.477+0000 7f93bd6fa700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f93a4038940 0x7f93a403ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.477+0000 7f93bd6fa700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f93a4038940 0x7f93a403ae00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93b4006fd0 tx=0x7f93b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.482+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f93a8019880 con 0x7f93b81070c0 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:19.954+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f93b8105bd0 con 0x7f93b81070c0 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.005+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f93a80212a0 con 0x7f93b81070c0 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.007+0000 7f93aeffd700 1 -- 172.21.15.45:0/364804031 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 5) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f93a8019c00 con 0x7f93b81070c0 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f93a4038940 msgr2=0x7f93a403ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f93a4038940 0x7f93a403ae00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93b4006fd0 tx=0x7f93b4006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 msgr2=0x7f93b8106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b8106a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009fb0 tx=0x7f93a8011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 shutdown_connections 2023-09-23T09:21:20.434 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f93a4038940 0x7f93a403ae00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:20.434 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 --2- 172.21.15.45:0/364804031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93b81070c0 0x7f93b8106a20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:20.434 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 >> 172.21.15.45:0/364804031 conn(0x7f93b8100b10 msgr2=0x7f93b81017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:20.434 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.008+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 shutdown_connections 2023-09-23T09:21:20.434 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.009+0000 7f93c015f700 1 -- 172.21.15.45:0/364804031 wait complete. 2023-09-23T09:21:20.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:20.398+0000 7f627f295200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:21:20.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:20.514+0000 7f627f295200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:21:21.011 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:20.729+0000 7f627f295200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:21:21.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:21 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/364804031' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-23T09:21:21.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:21 smithi045 ceph-mon[98627]: mgrmap e5: x(active, since 9s) 2023-09-23T09:21:21.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:21.208+0000 7f627f295200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.883+0000 7f6925c50700 1 Processor -- start 2023-09-23T09:21:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.883+0000 7f6925c50700 1 -- start start 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f6925c50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f6925c50700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6920105f40 con 0x7f6920105550 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57552/0 (socket says 172.21.15.45:57552) 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691f7fe700 1 -- 172.21.15.45:0/1073309959 learned_addr learned my addr 172.21.15.45:0/1073309959 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691f7fe700 1 -- 172.21.15.45:0/1073309959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920106750 con 0x7f6920105550 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691f7fe700 1 --2- 172.21.15.45:0/1073309959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920105970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f69100098a0 tx=0x7f6910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7608ccde1f62937d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691e7fc700 1 -- 172.21.15.45:0/1073309959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6910004160 con 0x7f6920105550 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691e7fc700 1 -- 172.21.15.45:0/1073309959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69100042c0 con 0x7f6920105550 2023-09-23T09:21:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.884+0000 7f691e7fc700 1 -- 172.21.15.45:0/1073309959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6910011a00 con 0x7f6920105550 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.885+0000 7f6925c50700 1 -- 172.21.15.45:0/1073309959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 msgr2=0x7f6920105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.885+0000 7f6925c50700 1 --2- 172.21.15.45:0/1073309959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920105970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f69100098a0 tx=0x7f6910009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.885+0000 7f6925c50700 1 -- 172.21.15.45:0/1073309959 shutdown_connections 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.885+0000 7f6925c50700 1 --2- 172.21.15.45:0/1073309959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920105970 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.885+0000 7f6925c50700 1 -- 172.21.15.45:0/1073309959 >> 172.21.15.45:0/1073309959 conn(0x7f6920100b10 msgr2=0x7f6920102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.885+0000 7f6925c50700 1 -- 172.21.15.45:0/1073309959 shutdown_connections 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.885+0000 7f6925c50700 1 -- 172.21.15.45:0/1073309959 wait complete. 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.886+0000 7f6925c50700 1 Processor -- start 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.886+0000 7f6925c50700 1 -- start start 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.886+0000 7f6925c50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920079b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.886+0000 7f6925c50700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692019ceb0 con 0x7f6920105550 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.886+0000 7f691f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.886+0000 7f691f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57554/0 (socket says 172.21.15.45:57554) 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.886+0000 7f691f7fe700 1 -- 172.21.15.45:0/2119978452 learned_addr learned my addr 172.21.15.45:0/2119978452 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.887+0000 7f691f7fe700 1 -- 172.21.15.45:0/2119978452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910009550 con 0x7f6920105550 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.887+0000 7f691f7fe700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920079b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6910009fb0 tx=0x7f6910011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.887+0000 7f691cff9700 1 -- 172.21.15.45:0/2119978452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6910003930 con 0x7f6920105550 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.887+0000 7f691cff9700 1 -- 172.21.15.45:0/2119978452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6910003a90 con 0x7f6920105550 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.887+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692019cff0 con 0x7f6920105550 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.887+0000 7f691cff9700 1 -- 172.21.15.45:0/2119978452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69100195e0 con 0x7f6920105550 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.887+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692019d410 con 0x7f6920105550 2023-09-23T09:21:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.888+0000 7f691cff9700 1 -- 172.21.15.45:0/2119978452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f69100198a0 con 0x7f6920105550 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.888+0000 7f691cff9700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 0x7f690803adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.888+0000 7f691cff9700 1 -- 172.21.15.45:0/2119978452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f691004c7d0 con 0x7f6920105550 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.888+0000 7f691effd700 1 -- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 msgr2=0x7f690803adb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4072483061 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.888+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6920062450 con 0x7f6920105550 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.888+0000 7f691effd700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 0x7f690803adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:20.893+0000 7f691cff9700 1 -- 172.21.15.45:0/2119978452 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6910019ba0 con 0x7f6920105550 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.089+0000 7f691effd700 1 -- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 msgr2=0x7f690803adb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4072483061 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.089+0000 7f691effd700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 0x7f690803adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.369+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f692019d650 con 0x7f6920105550 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.370+0000 7f691cff9700 1 -- 172.21.15.45:0/2119978452 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6910018350 con 0x7f6920105550 2023-09-23T09:21:21.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 msgr2=0x7f690803adb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 0x7f690803adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 msgr2=0x7f6920079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920079b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6910009fb0 tx=0x7f6910011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 shutdown_connections 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7f69080388f0 0x7f690803adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 --2- 172.21.15.45:0/2119978452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6920105550 0x7f6920079b90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 >> 172.21.15.45:0/2119978452 conn(0x7f6920100b10 msgr2=0x7f69201017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.372+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 shutdown_connections 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:21.373+0000 7f6925c50700 1 -- 172.21.15.45:0/2119978452 wait complete. 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2023-09-23T09:21:21.780 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2023-09-23T09:21:21.782 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:21.366+0000 7f627f295200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:21:21.782 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:21.489+0000 7f627f295200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:21:21.782 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:21.605+0000 7f627f295200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:21:22.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:22 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2119978452' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-23T09:21:22.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:22.127+0000 7f627f295200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:21:23.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:22.913+0000 7f627f295200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:21:23.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:23.011+0000 7f627f295200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:21:24.318 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:23.983+0000 7f627f295200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:21:24.318 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:24.094+0000 7f627f295200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:21:24.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:24.316+0000 7f627f295200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:21:24.922 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:24.666+0000 7f627f295200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:21:24.923 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:24.920+0000 7f627f295200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:21:25.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:25.103+0000 7f627f295200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:21:25.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:25.434+0000 7f627f295200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:21:26.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:26.257+0000 7f627f295200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:21:26.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:26.415+0000 7f627f295200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:21:27.073 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:26.661+0000 7f627f295200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:21:27.074 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:26.798+0000 7f627f295200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:21:27.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:27 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:27.071+0000 7f627f295200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:21:28.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:28.126+0000 7f627f295200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:21:28.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: Active manager daemon x restarted 2023-09-23T09:21:28.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: Activating manager daemon x 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: osdmap e2: 0 total, 0 up, 0 in 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: mgrmap e6: x(active, starting, since 0.0033829s) 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: Manager daemon x is now available 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:28.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:28 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.269+0000 7fef3914a700 1 Processor -- start 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.269+0000 7fef3914a700 1 -- start start 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.269+0000 7fef3914a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef341094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:29.538 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.269+0000 7fef3914a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34079eb0 con 0x7fef341070c0 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef32d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef341094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef32d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef341094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57562/0 (socket says 172.21.15.45:57562) 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef32d9d700 1 -- 172.21.15.45:0/1449416903 learned_addr learned my addr 172.21.15.45:0/1449416903 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef32d9d700 1 -- 172.21.15.45:0/1449416903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef341099f0 con 0x7fef341070c0 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef32d9d700 1 --2- 172.21.15.45:0/1449416903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef341094b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef240098a0 tx=0x7fef24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6323ecd30a941bee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef31d9b700 1 -- 172.21.15.45:0/1449416903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef24004160 con 0x7fef341070c0 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef31d9b700 1 -- 172.21.15.45:0/1449416903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef240042c0 con 0x7fef341070c0 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.270+0000 7fef31d9b700 1 -- 172.21.15.45:0/1449416903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef24011a00 con 0x7fef341070c0 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.271+0000 7fef3914a700 1 -- 172.21.15.45:0/1449416903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 msgr2=0x7fef341094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.271+0000 7fef3914a700 1 --2- 172.21.15.45:0/1449416903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef341094b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef240098a0 tx=0x7fef24009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:29.539 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.271+0000 7fef3914a700 1 -- 172.21.15.45:0/1449416903 shutdown_connections 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.271+0000 7fef3914a700 1 --2- 172.21.15.45:0/1449416903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef341094b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.271+0000 7fef3914a700 1 -- 172.21.15.45:0/1449416903 >> 172.21.15.45:0/1449416903 conn(0x7fef34100b10 msgr2=0x7fef34102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.271+0000 7fef3914a700 1 -- 172.21.15.45:0/1449416903 shutdown_connections 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.271+0000 7fef3914a700 1 -- 172.21.15.45:0/1449416903 wait complete. 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.272+0000 7fef3914a700 1 Processor -- start 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.272+0000 7fef3914a700 1 -- start start 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.272+0000 7fef3914a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef3419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.272+0000 7fef3914a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3419a6d0 con 0x7fef341070c0 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.272+0000 7fef32d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef3419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.272+0000 7fef32d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef3419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57564/0 (socket says 172.21.15.45:57564) 2023-09-23T09:21:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.272+0000 7fef32d9d700 1 -- 172.21.15.45:0/447139625 learned_addr learned my addr 172.21.15.45:0/447139625 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.273+0000 7fef32d9d700 1 -- 172.21.15.45:0/447139625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24009550 con 0x7fef341070c0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.273+0000 7fef32d9d700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef3419a190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef24006af0 tx=0x7fef24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.273+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef24003a40 con 0x7fef341070c0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.273+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef24003ba0 con 0x7fef341070c0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.273+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3419a8d0 con 0x7fef341070c0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.273+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef2401a6e0 con 0x7fef341070c0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.273+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3419acf0 con 0x7fef341070c0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.274+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 45642+0+0 (secure 0 0 0) 0x7fef2401a950 con 0x7fef341070c0 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.274+0000 7fef1bfff700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.274+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 --> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef1c03b510 con 0x7fef1c038940 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.274+0000 7fef3259c700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 msgr2=0x7fef1c03ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4072483061 2023-09-23T09:21:29.541 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.275+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fef2404c930 con 0x7fef341070c0 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.275+0000 7fef3259c700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.476+0000 7fef3259c700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 msgr2=0x7fef1c03ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4072483061 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.476+0000 7fef3259c700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.877+0000 7fef3259c700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 msgr2=0x7fef1c03ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4072483061 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:22.877+0000 7fef3259c700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:23.678+0000 7fef3259c700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 msgr2=0x7fef1c03ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4072483061 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:23.678+0000 7fef3259c700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:25.279+0000 7fef3259c700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 msgr2=0x7fef1c03ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4072483061 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:25.279+0000 7fef3259c700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:28.142+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 6) v1 ==== 45409+0+0 (secure 0 0 0) 0x7fef2402d960 con 0x7fef341070c0 2023-09-23T09:21:29.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:28.142+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 msgr2=0x7fef1c03ae00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:28.142+0000 7fef1bfff700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.143+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) v1 ==== 45536+0+0 (secure 0 0 0) 0x7fef2404e0e0 con 0x7fef341070c0 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.143+0000 7fef1bfff700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fef1c03bfe0 0x7fef1c03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.143+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef1c03b510 con 0x7fef1c03bfe0 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.144+0000 7fef3259c700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fef1c03bfe0 0x7fef1c03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.144+0000 7fef3259c700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fef1c03bfe0 0x7fef1c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fef28003a10 tx=0x7fef28009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.145+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fef1c03b510 con 0x7fef1c03bfe0 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fef3419b4d0 con 0x7fef1c03bfe0 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef1bfff700 1 -- 172.21.15.45:0/447139625 <== mgr.14120 v2:172.21.15.45:6800/1791488577 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fef3419b4d0 con 0x7fef1c03bfe0 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fef1c03bfe0 msgr2=0x7fef1c03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fef1c03bfe0 0x7fef1c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fef28003a10 tx=0x7fef28009290 comp rx=0 tx=0).stop 2023-09-23T09:21:29.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 msgr2=0x7fef3419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef3419a190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef24006af0 tx=0x7fef24011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 shutdown_connections 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fef1c03bfe0 0x7fef1c03e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341070c0 0x7fef3419a190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 --2- 172.21.15.45:0/447139625 >> [v2:172.21.15.45:6800/4072483061,v1:172.21.15.45:6801/4072483061] conn(0x7fef1c038940 0x7fef1c03ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.156+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 >> 172.21.15.45:0/447139625 conn(0x7fef34100b10 msgr2=0x7fef341017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.157+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 shutdown_connections 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:29.157+0000 7fef3914a700 1 -- 172.21.15.45:0/447139625 wait complete. 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2023-09-23T09:21:29.544 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2023-09-23T09:21:29.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:29 smithi045 ceph-mon[98627]: Found migration_current of "None". Setting to last migration. 2023-09-23T09:21:29.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:29 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:29.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:29 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T09:21:29.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:29 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T09:21:29.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:29 smithi045 ceph-mon[98627]: mgrmap e7: x(active, since 1.00539s) 2023-09-23T09:21:30.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:30 smithi045 ceph-mon[98627]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-23T09:21:30.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:30 smithi045 ceph-mon[98627]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-23T09:21:30.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:30 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:30.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:30 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:30.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:30 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:30] ENGINE Bus STARTING 2023-09-23T09:21:30.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.008+0000 7fc8fff24700 1 Processor -- start 2023-09-23T09:21:30.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.009+0000 7fc8fff24700 1 -- start start 2023-09-23T09:21:30.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.009+0000 7fc8fff24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f81094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:30.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.009+0000 7fc8fff24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8079eb0 con 0x7fc8f81070c0 2023-09-23T09:21:30.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.009+0000 7fc8fdcc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f81094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:30.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.009+0000 7fc8fdcc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f81094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57608/0 (socket says 172.21.15.45:57608) 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.009+0000 7fc8fdcc0700 1 -- 172.21.15.45:0/4292798422 learned_addr learned my addr 172.21.15.45:0/4292798422 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.009+0000 7fc8fdcc0700 1 -- 172.21.15.45:0/4292798422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f81099f0 con 0x7fc8f81070c0 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fdcc0700 1 --2- 172.21.15.45:0/4292798422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f81094b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40098a0 tx=0x7fc8f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b606a2340303d17 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fccbe700 1 -- 172.21.15.45:0/4292798422 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8f4004160 con 0x7fc8f81070c0 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fccbe700 1 -- 172.21.15.45:0/4292798422 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8f40042c0 con 0x7fc8f81070c0 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fccbe700 1 -- 172.21.15.45:0/4292798422 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8f4011a00 con 0x7fc8f81070c0 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fff24700 1 -- 172.21.15.45:0/4292798422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 msgr2=0x7fc8f81094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fff24700 1 --2- 172.21.15.45:0/4292798422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f81094b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40098a0 tx=0x7fc8f4009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fff24700 1 -- 172.21.15.45:0/4292798422 shutdown_connections 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.010+0000 7fc8fff24700 1 --2- 172.21.15.45:0/4292798422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f81094b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.011+0000 7fc8fff24700 1 -- 172.21.15.45:0/4292798422 >> 172.21.15.45:0/4292798422 conn(0x7fc8f8100b10 msgr2=0x7fc8f8102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.011+0000 7fc8fff24700 1 -- 172.21.15.45:0/4292798422 shutdown_connections 2023-09-23T09:21:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.011+0000 7fc8fff24700 1 -- 172.21.15.45:0/4292798422 wait complete. 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.011+0000 7fc8fff24700 1 Processor -- start 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.011+0000 7fc8fff24700 1 -- start start 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.011+0000 7fc8fff24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f8106a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.011+0000 7fc8fff24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8106f60 con 0x7fc8f81070c0 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8fdcc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f8106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8fdcc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f8106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57610/0 (socket says 172.21.15.45:57610) 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8fdcc0700 1 -- 172.21.15.45:0/3760714018 learned_addr learned my addr 172.21.15.45:0/3760714018 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8fdcc0700 1 -- 172.21.15.45:0/3760714018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4009550 con 0x7fc8f81070c0 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8fdcc0700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f8106a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4006af0 tx=0x7fc8f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8eeffd700 1 -- 172.21.15.45:0/3760714018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8f4003a40 con 0x7fc8f81070c0 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f8105070 con 0x7fc8f81070c0 2023-09-23T09:21:30.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8eeffd700 1 -- 172.21.15.45:0/3760714018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8f4003ba0 con 0x7fc8f81070c0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8eeffd700 1 -- 172.21.15.45:0/3760714018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8f401a6e0 con 0x7fc8f81070c0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.012+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f8105490 con 0x7fc8f81070c0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.013+0000 7fc8eeffd700 1 -- 172.21.15.45:0/3760714018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 45536+0+0 (secure 0 0 0) 0x7fc8f401a920 con 0x7fc8f81070c0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.013+0000 7fc8eeffd700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fc8e4038860 0x7fc8e403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.013+0000 7fc8eeffd700 1 -- 172.21.15.45:0/3760714018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc8f404c740 con 0x7fc8f81070c0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.013+0000 7fc8fd4bf700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fc8e4038860 0x7fc8e403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.014+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f810a7f0 con 0x7fc8f81070c0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.014+0000 7fc8fd4bf700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fc8e4038860 0x7fc8e403ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8006fd0 tx=0x7fc8e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.019+0000 7fc8eeffd700 1 -- 172.21.15.45:0/3760714018 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc8f401abd0 con 0x7fc8f81070c0 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.416+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8f8105b40 con 0x7fc8e4038860 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.422+0000 7fc8eeffd700 1 -- 172.21.15.45:0/3760714018 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc8f8105b40 con 0x7fc8e4038860 2023-09-23T09:21:30.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.424+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fc8e4038860 msgr2=0x7fc8e403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.424+0000 7fc8fff24700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fc8e4038860 0x7fc8e403ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8006fd0 tx=0x7fc8e8006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.424+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 msgr2=0x7fc8f8106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.424+0000 7fc8fff24700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f8106a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4006af0 tx=0x7fc8f4011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.425+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 shutdown_connections 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.425+0000 7fc8fff24700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fc8e4038860 0x7fc8e403ad20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.425+0000 7fc8fff24700 1 --2- 172.21.15.45:0/3760714018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f81070c0 0x7fc8f8106a20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.425+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 >> 172.21.15.45:0/3760714018 conn(0x7fc8f8100b10 msgr2=0x7fc8f81017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.425+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 shutdown_connections 2023-09-23T09:21:30.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:30.425+0000 7fc8fff24700 1 -- 172.21.15.45:0/3760714018 wait complete. 2023-09-23T09:21:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:30] ENGINE Serving on https://172.21.15.45:7150 2023-09-23T09:21:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:30] ENGINE Serving on http://172.21.15.45:8765 2023-09-23T09:21:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:30] ENGINE Bus STARTED 2023-09-23T09:21:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:31.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:31.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:31 smithi045 ceph-mon[98627]: mgrmap e8: x(active, since 2s) 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.257+0000 7f82e89a7700 1 Processor -- start 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.257+0000 7f82e89a7700 1 -- start start 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.257+0000 7f82e89a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.257+0000 7f82e89a7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e4079df0 con 0x7f82e4107180 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.257+0000 7f82e259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.257+0000 7f82e259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57622/0 (socket says 172.21.15.45:57622) 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.257+0000 7f82e259c700 1 -- 172.21.15.45:0/3572194187 learned_addr learned my addr 172.21.15.45:0/3572194187 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:32.232 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.258+0000 7f82e259c700 1 -- 172.21.15.45:0/3572194187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4079f30 con 0x7f82e4107180 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.258+0000 7f82e259c700 1 --2- 172.21.15.45:0/3572194187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4109570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82d40098a0 tx=0x7f82d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5112d2f2118e381 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.258+0000 7f82e159a700 1 -- 172.21.15.45:0/3572194187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82d4004160 con 0x7f82e4107180 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.258+0000 7f82e159a700 1 -- 172.21.15.45:0/3572194187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82d40042c0 con 0x7f82e4107180 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.258+0000 7f82e159a700 1 -- 172.21.15.45:0/3572194187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82d4011a00 con 0x7f82e4107180 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 -- 172.21.15.45:0/3572194187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 msgr2=0x7f82e4109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 --2- 172.21.15.45:0/3572194187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4109570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82d40098a0 tx=0x7f82d4009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 -- 172.21.15.45:0/3572194187 shutdown_connections 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 --2- 172.21.15.45:0/3572194187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4109570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 -- 172.21.15.45:0/3572194187 >> 172.21.15.45:0/3572194187 conn(0x7f82e4100b10 msgr2=0x7f82e4102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 -- 172.21.15.45:0/3572194187 shutdown_connections 2023-09-23T09:21:32.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 -- 172.21.15.45:0/3572194187 wait complete. 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.259+0000 7f82e89a7700 1 Processor -- start 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e89a7700 1 -- start start 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e89a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e89a7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e4107020 con 0x7f82e4107180 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57624/0 (socket says 172.21.15.45:57624) 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e259c700 1 -- 172.21.15.45:0/2742937802 learned_addr learned my addr 172.21.15.45:0/2742937802 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e259c700 1 -- 172.21.15.45:0/2742937802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d4009550 con 0x7f82e4107180 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e259c700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4106ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82d4006af0 tx=0x7f82d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82d37fe700 1 -- 172.21.15.45:0/2742937802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82d4003a40 con 0x7f82e4107180 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e4105130 con 0x7f82e4107180 2023-09-23T09:21:32.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82d37fe700 1 -- 172.21.15.45:0/2742937802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82d4003ba0 con 0x7f82e4107180 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.260+0000 7f82d37fe700 1 -- 172.21.15.45:0/2742937802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82d401a6e0 con 0x7f82e4107180 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.261+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e4105550 con 0x7f82e4107180 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.261+0000 7f82d37fe700 1 -- 172.21.15.45:0/2742937802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f82d401a950 con 0x7f82e4107180 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.261+0000 7f82d37fe700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f82cc038990 0x7f82cc03ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.262+0000 7f82d37fe700 1 -- 172.21.15.45:0/2742937802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f82d404c9d0 con 0x7f82e4107180 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.262+0000 7f82e1d9b700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f82cc038990 0x7f82cc03ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.262+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e410a7e0 con 0x7f82e4107180 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.262+0000 7f82e1d9b700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f82cc038990 0x7f82cc03ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f82d8006fd0 tx=0x7f82d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.267+0000 7f82d37fe700 1 -- 172.21.15.45:0/2742937802 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f82d401ac00 con 0x7f82e4107180 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.633+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f82e4105c60 con 0x7f82cc038990 2023-09-23T09:21:32.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.633+0000 7f82d37fe700 1 -- 172.21.15.45:0/2742937802 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f82e4105c60 con 0x7f82cc038990 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.635+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f82cc038990 msgr2=0x7f82cc03ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.635+0000 7f82e89a7700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f82cc038990 0x7f82cc03ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f82d8006fd0 tx=0x7f82d8006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.635+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 msgr2=0x7f82e4106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.635+0000 7f82e89a7700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4106ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82d4006af0 tx=0x7f82d4011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.636+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 shutdown_connections 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.636+0000 7f82e89a7700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f82cc038990 0x7f82cc03ae50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.636+0000 7f82e89a7700 1 --2- 172.21.15.45:0/2742937802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e4107180 0x7f82e4106ae0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.636+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 >> 172.21.15.45:0/2742937802 conn(0x7f82e4100b10 msgr2=0x7f82e4101760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.636+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 shutdown_connections 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:31.636+0000 7f82e89a7700 1 -- 172.21.15.45:0/2742937802 wait complete. 2023-09-23T09:21:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2023-09-23T09:21:32.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:32 smithi045 ceph-mon[98627]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:33.794 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: Generating public/private rsa key pair. 2023-09-23T09:21:33.794 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: Your identification has been saved in /tmp/tmpmljzvijk/key. 2023-09-23T09:21:33.794 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: Your public key has been saved in /tmp/tmpmljzvijk/key.pub. 2023-09-23T09:21:33.794 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: The key fingerprint is: 2023-09-23T09:21:33.794 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: SHA256:RaO1KTcVTyY7n1n1mYZAPFPnAHwIosWAidT8DHcU8B4 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:21:33.794 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: The key's randomart image is: 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: +---[RSA 3072]----+ 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: |..+ oo==..O+B+o..| 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | . * .+o.+ @.B= =| 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | =..Eo * *..=o| 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | o. .+ . o.+ | 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | .S + | 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | | 2023-09-23T09:21:33.795 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | | 2023-09-23T09:21:33.796 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | | 2023-09-23T09:21:33.796 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: | | 2023-09-23T09:21:33.796 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: +----[SHA256]-----+ 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.721+0000 7ff665212700 1 Processor -- start 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff665212700 1 -- start start 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff665212700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff660106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff665212700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff660107050 con 0x7ff660106660 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff65ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff660106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff65ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff660106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57628/0 (socket says 172.21.15.45:57628) 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff65ed9d700 1 -- 172.21.15.45:0/3033254707 learned_addr learned my addr 172.21.15.45:0/3033254707 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff65ed9d700 1 -- 172.21.15.45:0/3033254707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660107860 con 0x7ff660106660 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff65ed9d700 1 --2- 172.21.15.45:0/3033254707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff660106a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6500098a0 tx=0x7ff650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14adcd60ea81ae8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.722+0000 7ff65dd9b700 1 -- 172.21.15.45:0/3033254707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff650004160 con 0x7ff660106660 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff65dd9b700 1 -- 172.21.15.45:0/3033254707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6500042c0 con 0x7ff660106660 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff65dd9b700 1 -- 172.21.15.45:0/3033254707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff650011a00 con 0x7ff660106660 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff665212700 1 -- 172.21.15.45:0/3033254707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 msgr2=0x7ff660106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff665212700 1 --2- 172.21.15.45:0/3033254707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff660106a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6500098a0 tx=0x7ff650009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff665212700 1 -- 172.21.15.45:0/3033254707 shutdown_connections 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff665212700 1 --2- 172.21.15.45:0/3033254707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff660106a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff665212700 1 -- 172.21.15.45:0/3033254707 >> 172.21.15.45:0/3033254707 conn(0x7ff660101c40 msgr2=0x7ff660104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff665212700 1 -- 172.21.15.45:0/3033254707 shutdown_connections 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.723+0000 7ff665212700 1 -- 172.21.15.45:0/3033254707 wait complete. 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.724+0000 7ff665212700 1 Processor -- start 2023-09-23T09:21:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.724+0000 7ff665212700 1 -- start start 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.724+0000 7ff665212700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff66007bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.724+0000 7ff665212700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66007c190 con 0x7ff660106660 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.724+0000 7ff65ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff66007bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.724+0000 7ff65ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff66007bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57630/0 (socket says 172.21.15.45:57630) 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.724+0000 7ff65ed9d700 1 -- 172.21.15.45:0/1344932258 learned_addr learned my addr 172.21.15.45:0/1344932258 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.725+0000 7ff65ed9d700 1 -- 172.21.15.45:0/1344932258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff650009550 con 0x7ff660106660 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.725+0000 7ff65ed9d700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff66007bc50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff650009fb0 tx=0x7ff650011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.725+0000 7ff657fff700 1 -- 172.21.15.45:0/1344932258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff650003930 con 0x7ff660106660 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.725+0000 7ff657fff700 1 -- 172.21.15.45:0/1344932258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff650003a90 con 0x7ff660106660 2023-09-23T09:21:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.725+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66007a2a0 con 0x7ff660106660 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.725+0000 7ff657fff700 1 -- 172.21.15.45:0/1344932258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6500195e0 con 0x7ff660106660 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.725+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66007a6c0 con 0x7ff660106660 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.726+0000 7ff657fff700 1 -- 172.21.15.45:0/1344932258 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7ff650019850 con 0x7ff660106660 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.726+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff660062450 con 0x7ff660106660 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.726+0000 7ff657fff700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff64c038940 0x7ff64c03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.726+0000 7ff657fff700 1 -- 172.21.15.45:0/1344932258 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff65004c990 con 0x7ff660106660 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.726+0000 7ff65e59c700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff64c038940 0x7ff64c03ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.727+0000 7ff65e59c700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff64c038940 0x7ff64c03ae00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff648006fd0 tx=0x7ff648006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:32.731+0000 7ff657fff700 1 -- 172.21.15.45:0/1344932258 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff65002b340 con 0x7ff660106660 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.094+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff66010aa60 con 0x7ff64c038940 2023-09-23T09:21:33.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.507+0000 7ff657fff700 1 -- 172.21.15.45:0/1344932258 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff66010aa60 con 0x7ff64c038940 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.509+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff64c038940 msgr2=0x7ff64c03ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.509+0000 7ff665212700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff64c038940 0x7ff64c03ae00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff648006fd0 tx=0x7ff648006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.509+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 msgr2=0x7ff66007bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.509+0000 7ff665212700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff66007bc50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff650009fb0 tx=0x7ff650011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.510+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 shutdown_connections 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.510+0000 7ff665212700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff64c038940 0x7ff64c03ae00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.510+0000 7ff665212700 1 --2- 172.21.15.45:0/1344932258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff660106660 0x7ff66007bc50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.510+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 >> 172.21.15.45:0/1344932258 conn(0x7ff660101c40 msgr2=0x7ff6601028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.510+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 shutdown_connections 2023-09-23T09:21:33.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:33.510+0000 7ff665212700 1 -- 172.21.15.45:0/1344932258 wait complete. 2023-09-23T09:21:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:34 smithi045 ceph-mon[98627]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:34 smithi045 ceph-mon[98627]: Generating ssh key... 2023-09-23T09:21:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:34 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:34 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:35.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRVSoM1blfbQpO8vxOEuCdWE2czWM3sWTCA2cEeLIQjYadsFldfh2vsamfH5n+nmRfhllcptjPQjOrHZnIrWFmaHkadm066NhgGX0AbRh+tYwilLrJrSZtjufsK8WIbdSd8Gv3Lk5WRCvDHrACJp3zrxheyMYtfhMg1q/o/Cekrsy4rmq+o/Xgs1TgP0B5sEQCHxYjtQeCdu2VoaZmWY6uNY2XMw48obrYJVYxfk103uTvuqHcEjdl9ynZGzZWBpOQzzcSwfx4TmVRwoMIW1FapwbDEq3/CqG2egy8QrsrEi+ZLwUcT6h3bg/LuW57sPlAr2tGtGtHazecKzMKYMHkts7m1nXTZCsWM+mUuB/xnoseI+6HapbxpY0iw8v/KCCY1CBKut+fmdAR88u09dAN+Bjw1X7HdTosGhHkm0jKtQxqknQGhI1XFejR9UFiVjzMb+vmeA+Y9uTPQ/BkOZO/NvD/QnbmOgNbRYDJvXPfiM2/6t5cs871wpm89aLmiQs= ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:21:35.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.335+0000 7f9e295ea700 1 Processor -- start 2023-09-23T09:21:35.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e295ea700 1 -- start start 2023-09-23T09:21:35.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e295ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e295ea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24106450 con 0x7f9e24105a60 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57634/0 (socket says 172.21.15.45:57634) 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e22ffd700 1 -- 172.21.15.45:0/3240815634 learned_addr learned my addr 172.21.15.45:0/3240815634 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e22ffd700 1 -- 172.21.15.45:0/3240815634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24106780 con 0x7f9e24105a60 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.336+0000 7f9e22ffd700 1 --2- 172.21.15.45:0/3240815634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105e80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098a0 tx=0x7f9e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339743f0d17eaee2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e21ffb700 1 -- 172.21.15.45:0/3240815634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e14004160 con 0x7f9e24105a60 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e21ffb700 1 -- 172.21.15.45:0/3240815634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e140042c0 con 0x7f9e24105a60 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e21ffb700 1 -- 172.21.15.45:0/3240815634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e14011a00 con 0x7f9e24105a60 2023-09-23T09:21:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e295ea700 1 -- 172.21.15.45:0/3240815634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 msgr2=0x7f9e24105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e295ea700 1 --2- 172.21.15.45:0/3240815634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105e80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098a0 tx=0x7f9e14009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e295ea700 1 -- 172.21.15.45:0/3240815634 shutdown_connections 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e295ea700 1 --2- 172.21.15.45:0/3240815634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105e80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e295ea700 1 -- 172.21.15.45:0/3240815634 >> 172.21.15.45:0/3240815634 conn(0x7f9e241002c0 msgr2=0x7f9e241026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e295ea700 1 -- 172.21.15.45:0/3240815634 shutdown_connections 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.337+0000 7f9e295ea700 1 -- 172.21.15.45:0/3240815634 wait complete. 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.338+0000 7f9e295ea700 1 Processor -- start 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.338+0000 7f9e295ea700 1 -- start start 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.338+0000 7f9e295ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.338+0000 7f9e295ea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2419cf10 con 0x7f9e24105a60 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.338+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:35.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.338+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57636/0 (socket says 172.21.15.45:57636) 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.338+0000 7f9e22ffd700 1 -- 172.21.15.45:0/3066409305 learned_addr learned my addr 172.21.15.45:0/3066409305 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.339+0000 7f9e22ffd700 1 -- 172.21.15.45:0/3066409305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14009550 con 0x7f9e24105a60 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.339+0000 7f9e22ffd700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009fb0 tx=0x7f9e14011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.339+0000 7f9e0bfff700 1 -- 172.21.15.45:0/3066409305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e14003930 con 0x7f9e24105a60 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.339+0000 7f9e0bfff700 1 -- 172.21.15.45:0/3066409305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e14003a90 con 0x7f9e24105a60 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.339+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2419d050 con 0x7f9e24105a60 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.339+0000 7f9e0bfff700 1 -- 172.21.15.45:0/3066409305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e14019610 con 0x7f9e24105a60 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.339+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2419d470 con 0x7f9e24105a60 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.340+0000 7f9e0bfff700 1 -- 172.21.15.45:0/3066409305 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f9e140198d0 con 0x7f9e24105a60 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.340+0000 7f9e0bfff700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f9e0c0388f0 0x7f9e0c03adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:35.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.340+0000 7f9e0bfff700 1 -- 172.21.15.45:0/3066409305 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9e1404cb90 con 0x7f9e24105a60 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.340+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e24062450 con 0x7f9e24105a60 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.340+0000 7f9e227fc700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f9e0c0388f0 0x7f9e0c03adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.341+0000 7f9e227fc700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f9e0c0388f0 0x7f9e0c03adb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9e18006fd0 tx=0x7f9e18006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.345+0000 7f9e0bfff700 1 -- 172.21.15.45:0/3066409305 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9e14019bd0 con 0x7f9e24105a60 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.712+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e24101650 con 0x7f9e0c0388f0 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.713+0000 7f9e0bfff700 1 -- 172.21.15.45:0/3066409305 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f9e24101650 con 0x7f9e0c0388f0 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f9e0c0388f0 msgr2=0x7f9e0c03adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f9e0c0388f0 0x7f9e0c03adb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9e18006fd0 tx=0x7f9e18006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 msgr2=0x7f9e24105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009fb0 tx=0x7f9e14011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 shutdown_connections 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f9e0c0388f0 0x7f9e0c03adb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 --2- 172.21.15.45:0/3066409305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e24105a60 0x7f9e24105120 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 >> 172.21.15.45:0/3066409305 conn(0x7f9e241002c0 msgr2=0x7f9e24100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 shutdown_connections 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:34.715+0000 7f9e295ea700 1 -- 172.21.15.45:0/3066409305 wait complete. 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2023-09-23T09:21:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2023-09-23T09:21:35.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:35 smithi045 ceph-mon[98627]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:36.760 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:36 smithi045 ceph-mon[98627]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:37 smithi045 ceph-mon[98627]: Deploying cephadm binary to smithi045 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.558+0000 7f77204de700 1 Processor -- start 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.558+0000 7f77204de700 1 -- start start 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.558+0000 7f77204de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f7718106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.558+0000 7f77204de700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718107050 con 0x7f7718106660 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.558+0000 7f771e27a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f7718106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.558+0000 7f771e27a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f7718106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57640/0 (socket says 172.21.15.45:57640) 2023-09-23T09:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.558+0000 7f771e27a700 1 -- 172.21.15.45:0/3389529267 learned_addr learned my addr 172.21.15.45:0/3389529267 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.559+0000 7f771e27a700 1 -- 172.21.15.45:0/3389529267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718107860 con 0x7f7718106660 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.559+0000 7f771e27a700 1 --2- 172.21.15.45:0/3389529267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f7718106a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f77140098a0 tx=0x7f7714009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f39015e7244512 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.559+0000 7f771d278700 1 -- 172.21.15.45:0/3389529267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7714004160 con 0x7f7718106660 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.559+0000 7f771d278700 1 -- 172.21.15.45:0/3389529267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77140042c0 con 0x7f7718106660 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.559+0000 7f771d278700 1 -- 172.21.15.45:0/3389529267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7714011a00 con 0x7f7718106660 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 -- 172.21.15.45:0/3389529267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 msgr2=0x7f7718106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 --2- 172.21.15.45:0/3389529267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f7718106a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f77140098a0 tx=0x7f7714009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 -- 172.21.15.45:0/3389529267 shutdown_connections 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 --2- 172.21.15.45:0/3389529267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f7718106a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 -- 172.21.15.45:0/3389529267 >> 172.21.15.45:0/3389529267 conn(0x7f7718101c40 msgr2=0x7f7718104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 -- 172.21.15.45:0/3389529267 shutdown_connections 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 -- 172.21.15.45:0/3389529267 wait complete. 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 Processor -- start 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.560+0000 7f77204de700 1 -- start start 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f77204de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f771819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f77204de700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771819a720 con 0x7f7718106660 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f771e27a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f771819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f771e27a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f771819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57642/0 (socket says 172.21.15.45:57642) 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f771e27a700 1 -- 172.21.15.45:0/3895032709 learned_addr learned my addr 172.21.15.45:0/3895032709 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f771e27a700 1 -- 172.21.15.45:0/3895032709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714009550 con 0x7f7718106660 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f771e27a700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f771819a1e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fb0 tx=0x7f7714011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f770f7fe700 1 -- 172.21.15.45:0/3895032709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7714003930 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f770f7fe700 1 -- 172.21.15.45:0/3895032709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7714003a90 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771819a920 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f770f7fe700 1 -- 172.21.15.45:0/3895032709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77140195e0 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.561+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771819ad40 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.562+0000 7f770f7fe700 1 -- 172.21.15.45:0/3895032709 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f77140198a0 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.562+0000 7f770f7fe700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f77040388f0 0x7f770403adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.562+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7718062450 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.563+0000 7f770f7fe700 1 -- 172.21.15.45:0/3895032709 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f771404cb90 con 0x7f7718106660 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.563+0000 7f771da79700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f77040388f0 0x7f770403adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.563+0000 7f771da79700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f77040388f0 0x7f770403adb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7708006fd0 tx=0x7f7708006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.568+0000 7f770f7fe700 1 -- 172.21.15.45:0/3895032709 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f77140188f0 con 0x7f7718106660 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:35.934+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) v1 -- 0x7f771810aa60 con 0x7f77040388f0 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.804+0000 7f770f7fe700 1 -- 172.21.15.45:0/3895032709 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f771810aa60 con 0x7f77040388f0 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.807+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f77040388f0 msgr2=0x7f770403adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.807+0000 7f77204de700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f77040388f0 0x7f770403adb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7708006fd0 tx=0x7f7708006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.807+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 msgr2=0x7f771819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.807+0000 7f77204de700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f771819a1e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fb0 tx=0x7f7714011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.808+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 shutdown_connections 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.808+0000 7f77204de700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f77040388f0 0x7f770403adb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.808+0000 7f77204de700 1 --2- 172.21.15.45:0/3895032709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718106660 0x7f771819a1e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.808+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 >> 172.21.15.45:0/3895032709 conn(0x7f7718101c40 msgr2=0x7f77181028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.808+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 shutdown_connections 2023-09-23T09:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:38.808+0000 7f77204de700 1 -- 172.21.15.45:0/3895032709 wait complete. 2023-09-23T09:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2023-09-23T09:21:40.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:39 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:40.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:39 smithi045 ceph-mon[98627]: Added host smithi045 2023-09-23T09:21:40.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:39 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:40.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-23T09:21:40.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.693+0000 7fb19b516700 1 Processor -- start 2023-09-23T09:21:40.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.693+0000 7fb19b516700 1 -- start start 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.693+0000 7fb19b516700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb194106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.693+0000 7fb19b516700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb194107010 con 0x7fb194106620 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.693+0000 7fb1992b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb194106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.694+0000 7fb1992b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb194106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57676/0 (socket says 172.21.15.45:57676) 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.694+0000 7fb1992b2700 1 -- 172.21.15.45:0/3489107225 learned_addr learned my addr 172.21.15.45:0/3489107225 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.694+0000 7fb1992b2700 1 -- 172.21.15.45:0/3489107225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb194107820 con 0x7fb194106620 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.694+0000 7fb1992b2700 1 --2- 172.21.15.45:0/3489107225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb194106a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1900098a0 tx=0x7fb190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d78e7f643a7452c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.694+0000 7fb18bfff700 1 -- 172.21.15.45:0/3489107225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb190004160 con 0x7fb194106620 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.694+0000 7fb18bfff700 1 -- 172.21.15.45:0/3489107225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb1900042c0 con 0x7fb194106620 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.694+0000 7fb18bfff700 1 -- 172.21.15.45:0/3489107225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb190011a00 con 0x7fb194106620 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.695+0000 7fb19b516700 1 -- 172.21.15.45:0/3489107225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 msgr2=0x7fb194106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.695+0000 7fb19b516700 1 --2- 172.21.15.45:0/3489107225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb194106a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1900098a0 tx=0x7fb190009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.695+0000 7fb19b516700 1 -- 172.21.15.45:0/3489107225 shutdown_connections 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.695+0000 7fb19b516700 1 --2- 172.21.15.45:0/3489107225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb194106a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.695+0000 7fb19b516700 1 -- 172.21.15.45:0/3489107225 >> 172.21.15.45:0/3489107225 conn(0x7fb194101bc0 msgr2=0x7fb194104000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.695+0000 7fb19b516700 1 -- 172.21.15.45:0/3489107225 shutdown_connections 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.695+0000 7fb19b516700 1 -- 172.21.15.45:0/3489107225 wait complete. 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.696+0000 7fb19b516700 1 Processor -- start 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.696+0000 7fb19b516700 1 -- start start 2023-09-23T09:21:40.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.696+0000 7fb19b516700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb19407bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.696+0000 7fb19b516700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19407c190 con 0x7fb194106620 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.696+0000 7fb1992b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb19407bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb1992b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb19407bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57678/0 (socket says 172.21.15.45:57678) 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb1992b2700 1 -- 172.21.15.45:0/1925695841 learned_addr learned my addr 172.21.15.45:0/1925695841 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb1992b2700 1 -- 172.21.15.45:0/1925695841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb190009550 con 0x7fb194106620 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb1992b2700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb19407bc50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb190009fb0 tx=0x7fb190011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb18a7fc700 1 -- 172.21.15.45:0/1925695841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb190003930 con 0x7fb194106620 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb18a7fc700 1 -- 172.21.15.45:0/1925695841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb190003a90 con 0x7fb194106620 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb18a7fc700 1 -- 172.21.15.45:0/1925695841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1900195e0 con 0x7fb194106620 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19407a2a0 con 0x7fb194106620 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.697+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19407a6c0 con 0x7fb194106620 2023-09-23T09:21:40.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.698+0000 7fb18a7fc700 1 -- 172.21.15.45:0/1925695841 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7fb190019850 con 0x7fb194106620 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.698+0000 7fb18a7fc700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fb180038940 0x7fb18003ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.699+0000 7fb18a7fc700 1 -- 172.21.15.45:0/1925695841 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb19004cbf0 con 0x7fb194106620 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.699+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb194062450 con 0x7fb194106620 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.699+0000 7fb198ab1700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fb180038940 0x7fb18003ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.699+0000 7fb198ab1700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fb180038940 0x7fb18003ae00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb184006fd0 tx=0x7fb184006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:39.704+0000 7fb18a7fc700 1 -- 172.21.15.45:0/1925695841 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb190019b50 con 0x7fb194106620 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.098+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb19410abe0 con 0x7fb180038940 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.103+0000 7fb18a7fc700 1 -- 172.21.15.45:0/1925695841 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb19410abe0 con 0x7fb180038940 2023-09-23T09:21:40.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.105+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fb180038940 msgr2=0x7fb18003ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.105+0000 7fb19b516700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fb180038940 0x7fb18003ae00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb184006fd0 tx=0x7fb184006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.105+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 msgr2=0x7fb19407bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.105+0000 7fb19b516700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb19407bc50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb190009fb0 tx=0x7fb190011ad0 comp rx=0 tx=0).stop 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.106+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 shutdown_connections 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.106+0000 7fb19b516700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fb180038940 0x7fb18003ae00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.106+0000 7fb19b516700 1 --2- 172.21.15.45:0/1925695841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb194106620 0x7fb19407bc50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.106+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 >> 172.21.15.45:0/1925695841 conn(0x7fb194101bc0 msgr2=0x7fb194102900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.106+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 shutdown_connections 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:40.106+0000 7fb19b516700 1 -- 172.21.15.45:0/1925695841 wait complete. 2023-09-23T09:21:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2023-09-23T09:21:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:41 smithi045 ceph-mon[98627]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:41 smithi045 ceph-mon[98627]: Saving service mon spec with placement count:5 2023-09-23T09:21:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:41 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:41 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:42.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.293+0000 7f59d2cf4700 1 Processor -- start 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.293+0000 7f59d2cf4700 1 -- start start 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.293+0000 7f59d2cf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc1094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.293+0000 7f59d2cf4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc079eb0 con 0x7f59cc1070c0 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59d0a90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc1094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59d0a90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc1094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57682/0 (socket says 172.21.15.45:57682) 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59d0a90700 1 -- 172.21.15.45:0/3430276052 learned_addr learned my addr 172.21.15.45:0/3430276052 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59d0a90700 1 -- 172.21.15.45:0/3430276052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc1099f0 con 0x7f59cc1070c0 2023-09-23T09:21:42.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59d0a90700 1 --2- 172.21.15.45:0/3430276052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc1094b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59c00098a0 tx=0x7f59c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c01918b6607b32 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:42.088 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59cb7fe700 1 -- 172.21.15.45:0/3430276052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c0004160 con 0x7f59cc1070c0 2023-09-23T09:21:42.088 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59cb7fe700 1 -- 172.21.15.45:0/3430276052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59c00042c0 con 0x7f59cc1070c0 2023-09-23T09:21:42.088 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.294+0000 7f59cb7fe700 1 -- 172.21.15.45:0/3430276052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c00119b0 con 0x7f59cc1070c0 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.295+0000 7f59d2cf4700 1 -- 172.21.15.45:0/3430276052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 msgr2=0x7f59cc1094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.295+0000 7f59d2cf4700 1 --2- 172.21.15.45:0/3430276052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc1094b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59c00098a0 tx=0x7f59c0009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.295+0000 7f59d2cf4700 1 -- 172.21.15.45:0/3430276052 shutdown_connections 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.295+0000 7f59d2cf4700 1 --2- 172.21.15.45:0/3430276052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc1094b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.295+0000 7f59d2cf4700 1 -- 172.21.15.45:0/3430276052 >> 172.21.15.45:0/3430276052 conn(0x7f59cc100b10 msgr2=0x7f59cc102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.295+0000 7f59d2cf4700 1 -- 172.21.15.45:0/3430276052 shutdown_connections 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.295+0000 7f59d2cf4700 1 -- 172.21.15.45:0/3430276052 wait complete. 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.296+0000 7f59d2cf4700 1 Processor -- start 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.296+0000 7f59d2cf4700 1 -- start start 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.296+0000 7f59d2cf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.296+0000 7f59d2cf4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc19a6f0 con 0x7f59cc1070c0 2023-09-23T09:21:42.089 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.296+0000 7f59d0a90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.296+0000 7f59d0a90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57684/0 (socket says 172.21.15.45:57684) 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.296+0000 7f59d0a90700 1 -- 172.21.15.45:0/2577258054 learned_addr learned my addr 172.21.15.45:0/2577258054 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.297+0000 7f59d0a90700 1 -- 172.21.15.45:0/2577258054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0009550 con 0x7f59cc1070c0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.297+0000 7f59d0a90700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc19a1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59c0003e60 tx=0x7f59c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.297+0000 7f59c9ffb700 1 -- 172.21.15.45:0/2577258054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c0003970 con 0x7f59cc1070c0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.297+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc19a8f0 con 0x7f59cc1070c0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.297+0000 7f59c9ffb700 1 -- 172.21.15.45:0/2577258054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59c0003ad0 con 0x7f59cc1070c0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.297+0000 7f59c9ffb700 1 -- 172.21.15.45:0/2577258054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c00195e0 con 0x7f59cc1070c0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.297+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc19ad10 con 0x7f59cc1070c0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.298+0000 7f59c9ffb700 1 -- 172.21.15.45:0/2577258054 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f59c00198a0 con 0x7f59cc1070c0 2023-09-23T09:21:42.090 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.298+0000 7f59c9ffb700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f59bc0388f0 0x7f59bc03adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.298+0000 7f59c9ffb700 1 -- 172.21.15.45:0/2577258054 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f59c004ca90 con 0x7f59cc1070c0 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.298+0000 7f59cbfff700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f59bc0388f0 0x7f59bc03adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.299+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc062450 con 0x7f59cc1070c0 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.299+0000 7f59cbfff700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f59bc0388f0 0x7f59bc03adb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.304+0000 7f59c9ffb700 1 -- 172.21.15.45:0/2577258054 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f59c0019ba0 con 0x7f59cc1070c0 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.705+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f59cc101eb0 con 0x7f59bc0388f0 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.710+0000 7f59c9ffb700 1 -- 172.21.15.45:0/2577258054 <== mgr.14120 v2:172.21.15.45:6800/1791488577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f59cc101eb0 con 0x7f59bc0388f0 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f59bc0388f0 msgr2=0x7f59bc03adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f59bc0388f0 0x7f59bc03adb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 msgr2=0x7f59cc19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc19a1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59c0003e60 tx=0x7f59c0011aa0 comp rx=0 tx=0).stop 2023-09-23T09:21:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 shutdown_connections 2023-09-23T09:21:42.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f59bc0388f0 0x7f59bc03adb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:42.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 --2- 172.21.15.45:0/2577258054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc1070c0 0x7f59cc19a1b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:42.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 >> 172.21.15.45:0/2577258054 conn(0x7f59cc100b10 msgr2=0x7f59cc1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:42.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 shutdown_connections 2023-09-23T09:21:42.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:41.712+0000 7f59d2cf4700 1 -- 172.21.15.45:0/2577258054 wait complete. 2023-09-23T09:21:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:42 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:42 smithi045 ceph-mon[98627]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:42 smithi045 ceph-mon[98627]: Saving service mgr spec with placement count:2 2023-09-23T09:21:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:42 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:43.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdca0c6c700 1 Processor -- start 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdca0c6c700 1 -- start start 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdca0c6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c109510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdca0c6c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c079e50 con 0x7fdc9c107120 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdc9a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdc9a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57688/0 (socket says 172.21.15.45:57688) 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdc9a59c700 1 -- 172.21.15.45:0/426531159 learned_addr learned my addr 172.21.15.45:0/426531159 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.637+0000 7fdc9a59c700 1 -- 172.21.15.45:0/426531159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c109a50 con 0x7fdc9c107120 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdc9a59c700 1 --2- 172.21.15.45:0/426531159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c109510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0098a0 tx=0x7fdc8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2888178d5b8ca3d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdc99d9b700 1 -- 172.21.15.45:0/426531159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc8c004160 con 0x7fdc9c107120 2023-09-23T09:21:43.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdc99d9b700 1 -- 172.21.15.45:0/426531159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8c0042c0 con 0x7fdc9c107120 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdc99d9b700 1 -- 172.21.15.45:0/426531159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc8c0119b0 con 0x7fdc9c107120 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdca0c6c700 1 -- 172.21.15.45:0/426531159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 msgr2=0x7fdc9c109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdca0c6c700 1 --2- 172.21.15.45:0/426531159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c109510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0098a0 tx=0x7fdc8c009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdca0c6c700 1 -- 172.21.15.45:0/426531159 shutdown_connections 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdca0c6c700 1 --2- 172.21.15.45:0/426531159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c109510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdca0c6c700 1 -- 172.21.15.45:0/426531159 >> 172.21.15.45:0/426531159 conn(0x7fdc9c100b10 msgr2=0x7fdc9c102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdca0c6c700 1 -- 172.21.15.45:0/426531159 shutdown_connections 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.638+0000 7fdca0c6c700 1 -- 172.21.15.45:0/426531159 wait complete. 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.639+0000 7fdca0c6c700 1 Processor -- start 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.639+0000 7fdca0c6c700 1 -- start start 2023-09-23T09:21:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.639+0000 7fdca0c6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c106a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.639+0000 7fdca0c6c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c106fc0 con 0x7fdc9c107120 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.639+0000 7fdc9a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdc9a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57690/0 (socket says 172.21.15.45:57690) 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdc9a59c700 1 -- 172.21.15.45:0/523860253 learned_addr learned my addr 172.21.15.45:0/523860253 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdc9a59c700 1 -- 172.21.15.45:0/523860253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c009550 con 0x7fdc9c107120 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdc9a59c700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c106a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c003e60 tx=0x7fdc8c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdc937fe700 1 -- 172.21.15.45:0/523860253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc8c003970 con 0x7fdc9c107120 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c1050d0 con 0x7fdc9c107120 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdc937fe700 1 -- 172.21.15.45:0/523860253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8c003ad0 con 0x7fdc9c107120 2023-09-23T09:21:43.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdc937fe700 1 -- 172.21.15.45:0/523860253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc8c0195e0 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.640+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c1054f0 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.641+0000 7fdc937fe700 1 -- 172.21.15.45:0/523860253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7fdc8c0198a0 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.641+0000 7fdc937fe700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fdc880388f0 0x7fdc8803adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.642+0000 7fdc937fe700 1 -- 172.21.15.45:0/523860253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdc8c04cd30 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.641+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c10a830 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.642+0000 7fdc93fff700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fdc880388f0 0x7fdc8803adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.642+0000 7fdc93fff700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fdc880388f0 0x7fdc8803adb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc84006fd0 tx=0x7fdc84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:42.646+0000 7fdc937fe700 1 -- 172.21.15.45:0/523860253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fdc8c019ba0 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.012+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fdc9c10aa70 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.016+0000 7fdc937fe700 1 -- 172.21.15.45:0/523860253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fdc8c018b40 con 0x7fdc9c107120 2023-09-23T09:21:43.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fdc880388f0 msgr2=0x7fdc8803adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fdc880388f0 0x7fdc8803adb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc84006fd0 tx=0x7fdc84006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 msgr2=0x7fdc9c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c106a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c003e60 tx=0x7fdc8c011aa0 comp rx=0 tx=0).stop 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 shutdown_connections 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fdc880388f0 0x7fdc8803adb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 --2- 172.21.15.45:0/523860253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc9c107120 0x7fdc9c106a80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 >> 172.21.15.45:0/523860253 conn(0x7fdc9c100b10 msgr2=0x7fdc9c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.019+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 shutdown_connections 2023-09-23T09:21:43.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.020+0000 7fdca0c6c700 1 -- 172.21.15.45:0/523860253 wait complete. 2023-09-23T09:21:44.273 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:44 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:44.274 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:44 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/523860253' entity='client.admin' 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd6ff9700 1 Processor -- start 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd6ff9700 1 -- start start 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd6ff9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd0105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd6ff9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0105f40 con 0x7f4cd0105550 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd4d95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd0105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd4d95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd0105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57694/0 (socket says 172.21.15.45:57694) 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd4d95700 1 -- 172.21.15.45:0/793931537 learned_addr learned my addr 172.21.15.45:0/793931537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:44.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.853+0000 7f4cd4d95700 1 -- 172.21.15.45:0/793931537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd0106750 con 0x7f4cd0105550 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.854+0000 7f4cd4d95700 1 --2- 172.21.15.45:0/793931537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd0105970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40098a0 tx=0x7f4cc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb6dcfe53f58198 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.854+0000 7f4ccf7fe700 1 -- 172.21.15.45:0/793931537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc4004160 con 0x7f4cd0105550 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.854+0000 7f4ccf7fe700 1 -- 172.21.15.45:0/793931537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc40042c0 con 0x7f4cd0105550 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.854+0000 7f4ccf7fe700 1 -- 172.21.15.45:0/793931537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc40119b0 con 0x7f4cd0105550 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/793931537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 msgr2=0x7f4cd0105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 --2- 172.21.15.45:0/793931537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd0105970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40098a0 tx=0x7f4cc4009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/793931537 shutdown_connections 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 --2- 172.21.15.45:0/793931537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd0105970 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/793931537 >> 172.21.15.45:0/793931537 conn(0x7f4cd0100b10 msgr2=0x7f4cd0102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/793931537 shutdown_connections 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/793931537 wait complete. 2023-09-23T09:21:44.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.855+0000 7f4cd6ff9700 1 Processor -- start 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd6ff9700 1 -- start start 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd6ff9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd01907a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd6ff9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0190ce0 con 0x7f4cd0105550 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd4d95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd01907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd4d95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd01907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57696/0 (socket says 172.21.15.45:57696) 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd4d95700 1 -- 172.21.15.45:0/129015716 learned_addr learned my addr 172.21.15.45:0/129015716 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd4d95700 1 -- 172.21.15.45:0/129015716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4009550 con 0x7f4cd0105550 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4cd4d95700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd01907a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c00 tx=0x7f4cc4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.856+0000 7f4ccdffb700 1 -- 172.21.15.45:0/129015716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc40039f0 con 0x7f4cd0105550 2023-09-23T09:21:44.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.857+0000 7f4ccdffb700 1 -- 172.21.15.45:0/129015716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc4003b50 con 0x7f4cd0105550 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.857+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd018edf0 con 0x7f4cd0105550 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.857+0000 7f4ccdffb700 1 -- 172.21.15.45:0/129015716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc401a780 con 0x7f4cd0105550 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.857+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd018f210 con 0x7f4cd0105550 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.858+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd0109980 con 0x7f4cd0105550 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.858+0000 7f4ccdffb700 1 -- 172.21.15.45:0/129015716 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f4cc4024440 con 0x7f4cd0105550 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.858+0000 7f4ccdffb700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f4cc0038940 0x7f4cc003ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.859+0000 7f4ccdffb700 1 -- 172.21.15.45:0/129015716 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4cc404c930 con 0x7f4cd0105550 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.859+0000 7f4ccffff700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f4cc0038940 0x7f4cc003ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.859+0000 7f4ccffff700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f4cc0038940 0x7f4cc003ae00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006fd0 tx=0x7f4cbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:44.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:43.867+0000 7f4ccdffb700 1 -- 172.21.15.45:0/129015716 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4cc40189a0 con 0x7f4cd0105550 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.239+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f4cd0109bc0 con 0x7f4cd0105550 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.241+0000 7f4ccdffb700 1 -- 172.21.15.45:0/129015716 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f4cc4018350 con 0x7f4cd0105550 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f4cc0038940 msgr2=0x7f4cc003ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f4cc0038940 0x7f4cc003ae00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006fd0 tx=0x7f4cbc006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 msgr2=0x7f4cd01907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd01907a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c00 tx=0x7f4cc4011a30 comp rx=0 tx=0).stop 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 shutdown_connections 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f4cc0038940 0x7f4cc003ae00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 --2- 172.21.15.45:0/129015716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0105550 0x7f4cd01907a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 >> 172.21.15.45:0/129015716 conn(0x7f4cd0100b10 msgr2=0x7f4cd00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 shutdown_connections 2023-09-23T09:21:44.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:44.244+0000 7f4cd6ff9700 1 -- 172.21.15.45:0/129015716 wait complete. 2023-09-23T09:21:44.620 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2023-09-23T09:21:45.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:45 smithi045 ceph-mon[98627]: from='mgr.14120 172.21.15.45:0/572690185' entity='mgr.x' 2023-09-23T09:21:45.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:45 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/129015716' entity='client.admin' 2023-09-23T09:21:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/390587651' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-23T09:21:46.331 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: ignoring --setuser ceph since I am not root 2023-09-23T09:21:46.331 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: ignoring --setgroup ceph since I am not root 2023-09-23T09:21:46.331 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:46.284+0000 7f7c30dcd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:21:46.470 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.095+0000 7f7c1f89f700 1 Processor -- start 2023-09-23T09:21:46.470 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.095+0000 7f7c1f89f700 1 -- start start 2023-09-23T09:21:46.470 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c1f89f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c18105350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:46.470 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c1f89f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c18079e30 con 0x7f7c18104f30 2023-09-23T09:21:46.470 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c1d63b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c18105350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:46.470 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c1d63b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c18105350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57700/0 (socket says 172.21.15.45:57700) 2023-09-23T09:21:46.470 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c1d63b700 1 -- 172.21.15.45:0/519443110 learned_addr learned my addr 172.21.15.45:0/519443110 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c1d63b700 1 -- 172.21.15.45:0/519443110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c18105890 con 0x7f7c18104f30 2023-09-23T09:21:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c1d63b700 1 --2- 172.21.15.45:0/519443110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c18105350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c080098a0 tx=0x7f7c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362aa879fc8bdc20 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c0ffff700 1 -- 172.21.15.45:0/519443110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c08004160 con 0x7f7c18104f30 2023-09-23T09:21:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c0ffff700 1 -- 172.21.15.45:0/519443110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c080042c0 con 0x7f7c18104f30 2023-09-23T09:21:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.096+0000 7f7c0ffff700 1 -- 172.21.15.45:0/519443110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c080119b0 con 0x7f7c18104f30 2023-09-23T09:21:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.097+0000 7f7c1f89f700 1 -- 172.21.15.45:0/519443110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 msgr2=0x7f7c18105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.097+0000 7f7c1f89f700 1 --2- 172.21.15.45:0/519443110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c18105350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c080098a0 tx=0x7f7c08009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.097+0000 7f7c1f89f700 1 -- 172.21.15.45:0/519443110 shutdown_connections 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.097+0000 7f7c1f89f700 1 --2- 172.21.15.45:0/519443110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c18105350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.097+0000 7f7c1f89f700 1 -- 172.21.15.45:0/519443110 >> 172.21.15.45:0/519443110 conn(0x7f7c18100b10 msgr2=0x7f7c18102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.097+0000 7f7c1f89f700 1 -- 172.21.15.45:0/519443110 shutdown_connections 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.097+0000 7f7c1f89f700 1 -- 172.21.15.45:0/519443110 wait complete. 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1f89f700 1 Processor -- start 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1f89f700 1 -- start start 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1f89f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c1819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1f89f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1819c930 con 0x7f7c18104f30 2023-09-23T09:21:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1d63b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c1819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:46.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1d63b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c1819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57702/0 (socket says 172.21.15.45:57702) 2023-09-23T09:21:46.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1d63b700 1 -- 172.21.15.45:0/390587651 learned_addr learned my addr 172.21.15.45:0/390587651 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:46.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.098+0000 7f7c1d63b700 1 -- 172.21.15.45:0/390587651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08009550 con 0x7f7c18104f30 2023-09-23T09:21:46.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.099+0000 7f7c1d63b700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c1819c3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c08011980 tx=0x7f7c08011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:46.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.099+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c08003a60 con 0x7f7c18104f30 2023-09-23T09:21:46.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.099+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c08003bc0 con 0x7f7c18104f30 2023-09-23T09:21:46.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.099+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0801a670 con 0x7f7c18104f30 2023-09-23T09:21:46.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.099+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1819cb30 con 0x7f7c18104f30 2023-09-23T09:21:46.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.099+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1819cf50 con 0x7f7c18104f30 2023-09-23T09:21:46.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.100+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f7c0801a8e0 con 0x7f7c18104f30 2023-09-23T09:21:46.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.100+0000 7f7c0e7fc700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f7c04038990 0x7f7c0403ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:46.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.100+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7c0804cad0 con 0x7f7c18104f30 2023-09-23T09:21:46.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.100+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c18195c70 con 0x7f7c18104f30 2023-09-23T09:21:46.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.100+0000 7f7c1ce3a700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f7c04038990 0x7f7c0403ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.100+0000 7f7c1ce3a700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f7c04038990 0x7f7c0403ae50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7c14006fd0 tx=0x7f7c14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.105+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7c0801ab90 con 0x7f7c18104f30 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:45.584+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f7c18106740 con 0x7f7c18104f30 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.069+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f7c080187c0 con 0x7f7c18104f30 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.071+0000 7f7c0e7fc700 1 -- 172.21.15.45:0/390587651 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 9) v1 ==== 45655+0+0 (secure 0 0 0) 0x7f7c0802d910 con 0x7f7c18104f30 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.072+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f7c04038990 msgr2=0x7f7c0403ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.072+0000 7f7c1f89f700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f7c04038990 0x7f7c0403ae50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7c14006fd0 tx=0x7f7c14006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:46.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.072+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 msgr2=0x7f7c1819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.072+0000 7f7c1f89f700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c1819c3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c08011980 tx=0x7f7c08011a60 comp rx=0 tx=0).stop 2023-09-23T09:21:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.073+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 shutdown_connections 2023-09-23T09:21:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.073+0000 7f7c1f89f700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7f7c04038990 0x7f7c0403ae50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.073+0000 7f7c1f89f700 1 --2- 172.21.15.45:0/390587651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c18104f30 0x7f7c1819c3f0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c08011980 tx=0x7f7c08011a60 comp rx=0 tx=0).stop 2023-09-23T09:21:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.073+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 >> 172.21.15.45:0/390587651 conn(0x7f7c18100b10 msgr2=0x7f7c18190af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.073+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 shutdown_connections 2023-09-23T09:21:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.073+0000 7f7c1f89f700 1 -- 172.21.15.45:0/390587651 wait complete. 2023-09-23T09:21:46.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:46.381+0000 7f7c30dcd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:21:46.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:46.493+0000 7f7c30dcd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:21:46.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:46.608+0000 7f7c30dcd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:21:47.071 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:46 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:46.827+0000 7f7c30dcd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:21:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:47 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/390587651' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-23T09:21:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:47 smithi045 ceph-mon[98627]: mgrmap e9: x(active, since 17s) 2023-09-23T09:21:47.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:47 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:47.311+0000 7f7c30dcd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:21:47.697 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:47 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:47.466+0000 7f7c30dcd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:21:47.697 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:47 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:47.579+0000 7f7c30dcd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:21:47.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:47.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-23T09:21:47.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-23T09:21:47.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.924+0000 7ff2b17d2700 1 Processor -- start 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.924+0000 7ff2b17d2700 1 -- start start 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.925+0000 7ff2b17d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac109590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.925+0000 7ff2b17d2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac079dd0 con 0x7ff2ac1071a0 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.925+0000 7ff2aaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.925+0000 7ff2aaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57716/0 (socket says 172.21.15.45:57716) 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.925+0000 7ff2aaffd700 1 -- 172.21.15.45:0/1310935300 learned_addr learned my addr 172.21.15.45:0/1310935300 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:47.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.926+0000 7ff2aaffd700 1 -- 172.21.15.45:0/1310935300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac079f10 con 0x7ff2ac1071a0 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.926+0000 7ff2aaffd700 1 --2- 172.21.15.45:0/1310935300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac109590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0098a0 tx=0x7ff29c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03fa0380163ba2a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.926+0000 7ff2a9ffb700 1 -- 172.21.15.45:0/1310935300 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff29c004160 con 0x7ff2ac1071a0 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.926+0000 7ff2a9ffb700 1 -- 172.21.15.45:0/1310935300 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff29c0042c0 con 0x7ff2ac1071a0 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.926+0000 7ff2a9ffb700 1 -- 172.21.15.45:0/1310935300 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff29c0119b0 con 0x7ff2ac1071a0 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 -- 172.21.15.45:0/1310935300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 msgr2=0x7ff2ac109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 --2- 172.21.15.45:0/1310935300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac109590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0098a0 tx=0x7ff29c009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 -- 172.21.15.45:0/1310935300 shutdown_connections 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 --2- 172.21.15.45:0/1310935300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac109590 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 -- 172.21.15.45:0/1310935300 >> 172.21.15.45:0/1310935300 conn(0x7ff2ac100b10 msgr2=0x7ff2ac102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:47.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 -- 172.21.15.45:0/1310935300 shutdown_connections 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 -- 172.21.15.45:0/1310935300 wait complete. 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.927+0000 7ff2b17d2700 1 Processor -- start 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2b17d2700 1 -- start start 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2b17d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac106b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2b17d2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac107040 con 0x7ff2ac1071a0 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2aaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac106b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2aaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac106b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57718/0 (socket says 172.21.15.45:57718) 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2aaffd700 1 -- 172.21.15.45:0/905559504 learned_addr learned my addr 172.21.15.45:0/905559504 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2aaffd700 1 -- 172.21.15.45:0/905559504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009550 con 0x7ff2ac1071a0 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.928+0000 7ff2aaffd700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac106b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff29c011980 tx=0x7ff29c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.929+0000 7ff293fff700 1 -- 172.21.15.45:0/905559504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff29c003a60 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.929+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac105150 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.929+0000 7ff293fff700 1 -- 172.21.15.45:0/905559504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff29c003bc0 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.929+0000 7ff293fff700 1 -- 172.21.15.45:0/905559504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff29c01a670 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.929+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac105570 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.930+0000 7ff293fff700 1 -- 172.21.15.45:0/905559504 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 45655+0+0 (secure 0 0 0) 0x7ff29c01a930 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.930+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac10a840 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.930+0000 7ff293fff700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 0x7ff29403ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.930+0000 7ff293fff700 1 -- 172.21.15.45:0/905559504 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff29c04cb80 con 0x7ff2ac1071a0 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.930+0000 7ff2aa7fc700 1 -- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 msgr2=0x7ff29403ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1791488577 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.930+0000 7ff2aa7fc700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 0x7ff29403ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:21:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:46.939+0000 7ff293fff700 1 -- 172.21.15.45:0/905559504 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff29c018d90 con 0x7ff2ac1071a0 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.131+0000 7ff2aa7fc700 1 -- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 msgr2=0x7ff29403ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1791488577 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.131+0000 7ff2aa7fc700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 0x7ff29403ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.415+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff2ac10aab0 con 0x7ff2ac1071a0 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.416+0000 7ff293fff700 1 -- 172.21.15.45:0/905559504 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7ff29c04ce60 con 0x7ff2ac1071a0 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 msgr2=0x7ff29403ae50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 0x7ff29403ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 msgr2=0x7ff2ac106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac106b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff29c011980 tx=0x7ff29c011a60 comp rx=0 tx=0).stop 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 shutdown_connections 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7ff294038990 0x7ff29403ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 --2- 172.21.15.45:0/905559504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ac1071a0 0x7ff2ac106b00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:47.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.418+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 >> 172.21.15.45:0/905559504 conn(0x7ff2ac100b10 msgr2=0x7ff2ac101790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:47.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.419+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 shutdown_connections 2023-09-23T09:21:47.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:47.419+0000 7ff2b17d2700 1 -- 172.21.15.45:0/905559504 wait complete. 2023-09-23T09:21:47.722 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2023-09-23T09:21:47.722 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2023-09-23T09:21:47.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:47 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:47.695+0000 7f7c30dcd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:21:48.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:48 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/905559504' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-23T09:21:48.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:48 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:48.231+0000 7f7c30dcd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:21:49.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:49 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:49.012+0000 7f7c30dcd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:21:49.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:49 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:49.111+0000 7f7c30dcd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:21:50.420 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:50 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:50.084+0000 7f7c30dcd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:21:50.420 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:50 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:50.196+0000 7f7c30dcd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:21:50.772 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:50 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:50.418+0000 7f7c30dcd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:21:51.026 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:50 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:50.770+0000 7f7c30dcd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:21:51.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:51 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:51.024+0000 7f7c30dcd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:21:51.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:51 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:51.207+0000 7f7c30dcd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:21:51.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:51 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:51.538+0000 7f7c30dcd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:21:52.771 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:52 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:52.364+0000 7f7c30dcd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:21:52.771 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:52 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:52.522+0000 7f7c30dcd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:21:53.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:52 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:52.769+0000 7f7c30dcd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:21:53.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:52 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:52.906+0000 7f7c30dcd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:21:53.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:53 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:53.180+0000 7f7c30dcd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:21:54.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:21:54.234+0000 7f7c30dcd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: Active manager daemon x restarted 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: Activating manager daemon x 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: osdmap e3: 0 total, 0 up, 0 in 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: mgrmap e10: x(active, starting, since 0.00312511s) 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: Manager daemon x is now available 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:54.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:21:55.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T09:21:55.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T09:21:55.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:55 smithi045 ceph-mon[98627]: mgrmap e11: x(active, since 1.00669s) 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.187+0000 7fcf64545700 1 Processor -- start 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.187+0000 7fcf64545700 1 -- start start 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.188+0000 7fcf64545700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.188+0000 7fcf64545700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c105f40 con 0x7fcf5c105550 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.188+0000 7fcf622e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:55.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.188+0000 7fcf622e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57726/0 (socket says 172.21.15.45:57726) 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.188+0000 7fcf622e1700 1 -- 172.21.15.45:0/1083438411 learned_addr learned my addr 172.21.15.45:0/1083438411 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.188+0000 7fcf622e1700 1 -- 172.21.15.45:0/1083438411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c106750 con 0x7fcf5c105550 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.188+0000 7fcf622e1700 1 --2- 172.21.15.45:0/1083438411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c105970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf580098a0 tx=0x7fcf58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ddd5b76a7f14dd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf612df700 1 -- 172.21.15.45:0/1083438411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf58004160 con 0x7fcf5c105550 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf612df700 1 -- 172.21.15.45:0/1083438411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf580042c0 con 0x7fcf5c105550 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf612df700 1 -- 172.21.15.45:0/1083438411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf580119b0 con 0x7fcf5c105550 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf64545700 1 -- 172.21.15.45:0/1083438411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 msgr2=0x7fcf5c105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf64545700 1 --2- 172.21.15.45:0/1083438411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c105970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf580098a0 tx=0x7fcf58009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf64545700 1 -- 172.21.15.45:0/1083438411 shutdown_connections 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf64545700 1 --2- 172.21.15.45:0/1083438411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c105970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf64545700 1 -- 172.21.15.45:0/1083438411 >> 172.21.15.45:0/1083438411 conn(0x7fcf5c100b10 msgr2=0x7fcf5c102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:55.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf64545700 1 -- 172.21.15.45:0/1083438411 shutdown_connections 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.189+0000 7fcf64545700 1 -- 172.21.15.45:0/1083438411 wait complete. 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.190+0000 7fcf64545700 1 Processor -- start 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.190+0000 7fcf64545700 1 -- start start 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.190+0000 7fcf64545700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c1907a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.190+0000 7fcf64545700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c190ce0 con 0x7fcf5c105550 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.190+0000 7fcf622e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c1907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.190+0000 7fcf622e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c1907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57728/0 (socket says 172.21.15.45:57728) 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.190+0000 7fcf622e1700 1 -- 172.21.15.45:0/763030728 learned_addr learned my addr 172.21.15.45:0/763030728 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.191+0000 7fcf622e1700 1 -- 172.21.15.45:0/763030728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009550 con 0x7fcf5c105550 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.191+0000 7fcf622e1700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c1907a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf58003e60 tx=0x7fcf58011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:55.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.191+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf58003970 con 0x7fcf5c105550 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.191+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf58003ad0 con 0x7fcf5c105550 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.191+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf580195e0 con 0x7fcf5c105550 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.191+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c18edf0 con 0x7fcf5c105550 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.191+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c18f210 con 0x7fcf5c105550 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.192+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 45655+0+0 (secure 0 0 0) 0x7fcf58019850 con 0x7fcf5c105550 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.192+0000 7fcf4f7fe700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.192+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 --> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf4803b560 con 0x7fcf48038990 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.192+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcf5804ca50 con 0x7fcf5c105550 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.192+0000 7fcf61ae0700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 msgr2=0x7fcf4803ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1791488577 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.192+0000 7fcf61ae0700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.393+0000 7fcf61ae0700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 msgr2=0x7fcf4803ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1791488577 2023-09-23T09:21:55.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.393+0000 7fcf61ae0700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.794+0000 7fcf61ae0700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 msgr2=0x7fcf4803ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1791488577 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:48.794+0000 7fcf61ae0700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:49.595+0000 7fcf61ae0700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 msgr2=0x7fcf4803ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1791488577 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:49.595+0000 7fcf61ae0700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:51.196+0000 7fcf61ae0700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 msgr2=0x7fcf4803ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1791488577 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:51.196+0000 7fcf61ae0700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:54.249+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fcf58019b50 con 0x7fcf5c105550 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:54.249+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 msgr2=0x7fcf4803ae50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:54.249+0000 7fcf4f7fe700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.251+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fcf5804e590 con 0x7fcf5c105550 2023-09-23T09:21:55.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.251+0000 7fcf4f7fe700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcf4803c550 0x7fcf4803e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.251+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf4803b560 con 0x7fcf4803c550 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.252+0000 7fcf61ae0700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcf4803c550 0x7fcf4803e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.252+0000 7fcf61ae0700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcf4803c550 0x7fcf4803e940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcf50003a10 tx=0x7fcf50009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.253+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fcf4803b560 con 0x7fcf4803c550 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.264+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcf5c190340 con 0x7fcf4803c550 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.264+0000 7fcf4f7fe700 1 -- 172.21.15.45:0/763030728 <== mgr.14152 v2:172.21.15.45:6800/944196506 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fcf5c190340 con 0x7fcf4803c550 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcf4803c550 msgr2=0x7fcf4803e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcf4803c550 0x7fcf4803e940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcf50003a10 tx=0x7fcf50009270 comp rx=0 tx=0).stop 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 msgr2=0x7fcf5c1907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c1907a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf58003e60 tx=0x7fcf58011aa0 comp rx=0 tx=0).stop 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 shutdown_connections 2023-09-23T09:21:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcf4803c550 0x7fcf4803e940 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:55.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf5c105550 0x7fcf5c1907a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:55.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 --2- 172.21.15.45:0/763030728 >> [v2:172.21.15.45:6800/1791488577,v1:172.21.15.45:6801/1791488577] conn(0x7fcf48038990 0x7fcf4803ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:55.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 >> 172.21.15.45:0/763030728 conn(0x7fcf5c100b10 msgr2=0x7fcf5c079940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:55.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 shutdown_connections 2023-09-23T09:21:55.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:55.265+0000 7fcf64545700 1 -- 172.21.15.45:0/763030728 wait complete. 2023-09-23T09:21:55.886 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2023-09-23T09:21:55.886 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2023-09-23T09:21:57.274 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:55] ENGINE Bus STARTING 2023-09-23T09:21:57.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:55] ENGINE Serving on https://172.21.15.45:7150 2023-09-23T09:21:57.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:21:57.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:56] ENGINE Serving on http://172.21.15.45:8765 2023-09-23T09:21:57.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: [23/Sep/2023:09:21:56] ENGINE Bus STARTED 2023-09-23T09:21:57.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:21:57.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:21:57.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.403+0000 7f0196608700 1 Processor -- start 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.403+0000 7f0196608700 1 -- start start 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.403+0000 7f0196608700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.403+0000 7f0196608700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190105f40 con 0x7f0190105550 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.403+0000 7f018ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.403+0000 7f018ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57782/0 (socket says 172.21.15.45:57782) 2023-09-23T09:21:57.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.403+0000 7f018ffff700 1 -- 172.21.15.45:0/661370488 learned_addr learned my addr 172.21.15.45:0/661370488 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.404+0000 7f018ffff700 1 -- 172.21.15.45:0/661370488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190106750 con 0x7f0190105550 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.404+0000 7f018ffff700 1 --2- 172.21.15.45:0/661370488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190105970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01800098a0 tx=0x7f0180009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c486e1d3548f9a11 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.404+0000 7f018effd700 1 -- 172.21.15.45:0/661370488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0180004160 con 0x7f0190105550 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.404+0000 7f018effd700 1 -- 172.21.15.45:0/661370488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01800042c0 con 0x7f0190105550 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.404+0000 7f018effd700 1 -- 172.21.15.45:0/661370488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01800119b0 con 0x7f0190105550 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 -- 172.21.15.45:0/661370488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 msgr2=0x7f0190105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 --2- 172.21.15.45:0/661370488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190105970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01800098a0 tx=0x7f0180009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 -- 172.21.15.45:0/661370488 shutdown_connections 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 --2- 172.21.15.45:0/661370488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190105970 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 -- 172.21.15.45:0/661370488 >> 172.21.15.45:0/661370488 conn(0x7f0190100b10 msgr2=0x7f0190102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 -- 172.21.15.45:0/661370488 shutdown_connections 2023-09-23T09:21:57.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 -- 172.21.15.45:0/661370488 wait complete. 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 Processor -- start 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.405+0000 7f0196608700 1 -- start start 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.406+0000 7f0196608700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190079b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.406+0000 7f0196608700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019019ced0 con 0x7f0190105550 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.406+0000 7f018ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.406+0000 7f018ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57784/0 (socket says 172.21.15.45:57784) 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.406+0000 7f018ffff700 1 -- 172.21.15.45:0/1975515380 learned_addr learned my addr 172.21.15.45:0/1975515380 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.406+0000 7f018ffff700 1 -- 172.21.15.45:0/1975515380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0180009550 con 0x7f0190105550 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.406+0000 7f018ffff700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190079b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0180003e60 tx=0x7f0180011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.407+0000 7f018d7fa700 1 -- 172.21.15.45:0/1975515380 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0180003970 con 0x7f0190105550 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.407+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019019d010 con 0x7f0190105550 2023-09-23T09:21:57.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.407+0000 7f018d7fa700 1 -- 172.21.15.45:0/1975515380 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0180003ad0 con 0x7f0190105550 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.407+0000 7f018d7fa700 1 -- 172.21.15.45:0/1975515380 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01800195e0 con 0x7f0190105550 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.407+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019019d430 con 0x7f0190105550 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.408+0000 7f018d7fa700 1 -- 172.21.15.45:0/1975515380 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f01800198a0 con 0x7f0190105550 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.408+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0190109980 con 0x7f0190105550 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.408+0000 7f018d7fa700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0178038820 0x7f017803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.408+0000 7f018d7fa700 1 -- 172.21.15.45:0/1975515380 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f018004cba0 con 0x7f0190105550 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.408+0000 7f018f7fe700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0178038820 0x7f017803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.409+0000 7f018f7fe700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0178038820 0x7f017803ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0184006fd0 tx=0x7f0184006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.413+0000 7f018d7fa700 1 -- 172.21.15.45:0/1975515380 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f018002b340 con 0x7f0190105550 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.792+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f0190101ef0 con 0x7f0178038820 2023-09-23T09:21:57.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.920+0000 7f018d7fa700 1 -- 172.21.15.45:0/1975515380 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f0190101ef0 con 0x7f0178038820 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0178038820 msgr2=0x7f017803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0178038820 0x7f017803ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0184006fd0 tx=0x7f0184006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 msgr2=0x7f0190079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190079b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0180003e60 tx=0x7f0180011aa0 comp rx=0 tx=0).stop 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 shutdown_connections 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0178038820 0x7f017803ace0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 --2- 172.21.15.45:0/1975515380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0190105550 0x7f0190079b90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 >> 172.21.15.45:0/1975515380 conn(0x7f0190100b10 msgr2=0x7f01901017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 shutdown_connections 2023-09-23T09:21:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:56.922+0000 7f0196608700 1 -- 172.21.15.45:0/1975515380 wait complete. 2023-09-23T09:21:57.331 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2023-09-23T09:21:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:58 smithi045 ceph-mon[98627]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:58 smithi045 ceph-mon[98627]: mgrmap e12: x(active, since 2s) 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$XK69rvTg3A7utPHl2yk5xeiJK3Ar2iOnKr1MSfvumw1xJuwqYZHRG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1695460918, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.866+0000 7f813b76c700 1 Processor -- start 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.866+0000 7f813b76c700 1 -- start start 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f813b76c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f8134105310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f813b76c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8134079e30 con 0x7f8134104ef0 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f8139508700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f8134105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f8139508700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f8134105310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57788/0 (socket says 172.21.15.45:57788) 2023-09-23T09:21:58.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f8139508700 1 -- 172.21.15.45:0/695209566 learned_addr learned my addr 172.21.15.45:0/695209566 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f8139508700 1 -- 172.21.15.45:0/695209566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134105850 con 0x7f8134104ef0 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f8139508700 1 --2- 172.21.15.45:0/695209566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f8134105310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f81300098a0 tx=0x7f8130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0a8be4b6a2fc96 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.867+0000 7f812bfff700 1 -- 172.21.15.45:0/695209566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8130004160 con 0x7f8134104ef0 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f812bfff700 1 -- 172.21.15.45:0/695209566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81300042c0 con 0x7f8134104ef0 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f812bfff700 1 -- 172.21.15.45:0/695209566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81300119b0 con 0x7f8134104ef0 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f813b76c700 1 -- 172.21.15.45:0/695209566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 msgr2=0x7f8134105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f813b76c700 1 --2- 172.21.15.45:0/695209566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f8134105310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f81300098a0 tx=0x7f8130009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f813b76c700 1 -- 172.21.15.45:0/695209566 shutdown_connections 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f813b76c700 1 --2- 172.21.15.45:0/695209566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f8134105310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f813b76c700 1 -- 172.21.15.45:0/695209566 >> 172.21.15.45:0/695209566 conn(0x7f8134100ad0 msgr2=0x7f8134102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f813b76c700 1 -- 172.21.15.45:0/695209566 shutdown_connections 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.868+0000 7f813b76c700 1 -- 172.21.15.45:0/695209566 wait complete. 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f813b76c700 1 Processor -- start 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f813b76c700 1 -- start start 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f813b76c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f813419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f813b76c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813419c950 con 0x7f8134104ef0 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f8139508700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f813419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f8139508700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f813419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57790/0 (socket says 172.21.15.45:57790) 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f8139508700 1 -- 172.21.15.45:0/665830136 learned_addr learned my addr 172.21.15.45:0/665830136 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.869+0000 7f8139508700 1 -- 172.21.15.45:0/665830136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130009550 con 0x7f8134104ef0 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.870+0000 7f8139508700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f813419c410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8130011980 tx=0x7f8130011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.870+0000 7f812a7fc700 1 -- 172.21.15.45:0/665830136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8130003a60 con 0x7f8134104ef0 2023-09-23T09:21:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.870+0000 7f812a7fc700 1 -- 172.21.15.45:0/665830136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8130003bc0 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.870+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813419cb50 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.870+0000 7f812a7fc700 1 -- 172.21.15.45:0/665830136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f813001a670 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.870+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813419cf70 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.871+0000 7f812a7fc700 1 -- 172.21.15.45:0/665830136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45655+0+0 (secure 0 0 0) 0x7f813001a930 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.871+0000 7f812a7fc700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8120038990 0x7f812003ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.871+0000 7f812a7fc700 1 -- 172.21.15.45:0/665830136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f813004cb80 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.871+0000 7f8138d07700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8120038990 0x7f812003ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.871+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8134195cd0 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.871+0000 7f8138d07700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8120038990 0x7f812003ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8124006fd0 tx=0x7f8124006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:57.876+0000 7f812a7fc700 1 -- 172.21.15.45:0/665830136 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f813001ac30 con 0x7f8134104ef0 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.279+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8134062270 con 0x7f8120038990 2023-09-23T09:21:58.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.526+0000 7f812a7fc700 1 -- 172.21.15.45:0/665830136 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8134062270 con 0x7f8120038990 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8120038990 msgr2=0x7f812003ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8120038990 0x7f812003ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8124006fd0 tx=0x7f8124006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 msgr2=0x7f813419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f813419c410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8130011980 tx=0x7f8130011a60 comp rx=0 tx=0).stop 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 shutdown_connections 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8120038990 0x7f812003ae50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 --2- 172.21.15.45:0/665830136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134104ef0 0x7f813419c410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 >> 172.21.15.45:0/665830136 conn(0x7f8134100ad0 msgr2=0x7f8134190ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.528+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 shutdown_connections 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:58.529+0000 7f813b76c700 1 -- 172.21.15.45:0/665830136 wait complete. 2023-09-23T09:21:58.828 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2023-09-23T09:21:59.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:59 smithi045 ceph-mon[98627]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:21:59.783 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:21:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:21:59.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2023-09-23T09:21:59.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.266+0000 7fca4bcfe700 1 Processor -- start 2023-09-23T09:21:59.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.266+0000 7fca4bcfe700 1 -- start start 2023-09-23T09:21:59.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.266+0000 7fca4bcfe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca44106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:59.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.266+0000 7fca4bcfe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44107050 con 0x7fca44106660 2023-09-23T09:21:59.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca49a9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca44106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:59.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca49a9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca44106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57794/0 (socket says 172.21.15.45:57794) 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca49a9a700 1 -- 172.21.15.45:0/2616654000 learned_addr learned my addr 172.21.15.45:0/2616654000 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca49a9a700 1 -- 172.21.15.45:0/2616654000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44107860 con 0x7fca44106660 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca49a9a700 1 --2- 172.21.15.45:0/2616654000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca44106a80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca340098a0 tx=0x7fca34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf6d03d41e8bd26 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca48a98700 1 -- 172.21.15.45:0/2616654000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca34004160 con 0x7fca44106660 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca48a98700 1 -- 172.21.15.45:0/2616654000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca340042c0 con 0x7fca44106660 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.267+0000 7fca48a98700 1 -- 172.21.15.45:0/2616654000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca340119b0 con 0x7fca44106660 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.268+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2616654000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 msgr2=0x7fca44106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.268+0000 7fca4bcfe700 1 --2- 172.21.15.45:0/2616654000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca44106a80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca340098a0 tx=0x7fca34009bb0 comp rx=0 tx=0).stop 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.268+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2616654000 shutdown_connections 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.268+0000 7fca4bcfe700 1 --2- 172.21.15.45:0/2616654000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca44106a80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.268+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2616654000 >> 172.21.15.45:0/2616654000 conn(0x7fca44101c40 msgr2=0x7fca44104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.268+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2616654000 shutdown_connections 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.268+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2616654000 wait complete. 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca4bcfe700 1 Processor -- start 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca4bcfe700 1 -- start start 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca4bcfe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca4407bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca4bcfe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4407c190 con 0x7fca44106660 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca49a9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca4407bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca49a9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca4407bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57796/0 (socket says 172.21.15.45:57796) 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca49a9a700 1 -- 172.21.15.45:0/2392579354 learned_addr learned my addr 172.21.15.45:0/2392579354 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.269+0000 7fca49a9a700 1 -- 172.21.15.45:0/2392579354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009550 con 0x7fca44106660 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.270+0000 7fca49a9a700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca4407bc50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca34003e60 tx=0x7fca34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:59.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.270+0000 7fca3affd700 1 -- 172.21.15.45:0/2392579354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca34003970 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.270+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4407a2a0 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.270+0000 7fca3affd700 1 -- 172.21.15.45:0/2392579354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca34003ad0 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.270+0000 7fca3affd700 1 -- 172.21.15.45:0/2392579354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca340195e0 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.270+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4407a6c0 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.271+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4410aa90 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.271+0000 7fca3affd700 1 -- 172.21.15.45:0/2392579354 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45655+0+0 (secure 0 0 0) 0x7fca34019850 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.272+0000 7fca3affd700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fca30038990 0x7fca3003ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.272+0000 7fca3affd700 1 -- 172.21.15.45:0/2392579354 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca3404cb60 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.272+0000 7fca49299700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fca30038990 0x7fca3003ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.272+0000 7fca49299700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fca30038990 0x7fca3003ae50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca40006fd0 tx=0x7fca40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.277+0000 7fca3affd700 1 -- 172.21.15.45:0/2392579354 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fca34003c40 con 0x7fca44106660 2023-09-23T09:21:59.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.643+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fca4410acd0 con 0x7fca44106660 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.644+0000 7fca3affd700 1 -- 172.21.15.45:0/2392579354 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fca34018350 con 0x7fca44106660 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fca30038990 msgr2=0x7fca3003ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fca30038990 0x7fca3003ae50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca40006fd0 tx=0x7fca40006e40 comp rx=0 tx=0).stop 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 msgr2=0x7fca4407bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca4407bc50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca34003e60 tx=0x7fca34011aa0 comp rx=0 tx=0).stop 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 shutdown_connections 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fca30038990 0x7fca3003ae50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 --2- 172.21.15.45:0/2392579354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca44106660 0x7fca4407bc50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 >> 172.21.15.45:0/2392579354 conn(0x7fca44101c40 msgr2=0x7fca441028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 shutdown_connections 2023-09-23T09:21:59.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:21:59.646+0000 7fca4bcfe700 1 -- 172.21.15.45:0/2392579354 wait complete. 2023-09-23T09:21:59.978 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T09:21:59.978 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-09-23T09:21:59.985 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T09:21:59.986 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-09-23T09:21:59.986 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-09-23T09:21:59.986 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-23T09:21:59.987 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2023-09-23T09:21:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:21:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045:8443/ 2023-09-23T09:21:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2023-09-23T09:21:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: Password: 4ybzf202nd 2023-09-23T09:21:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:21:59.988 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config directory 2023-09-23T09:21:59.988 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2023-09-23T09:22:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:00 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2392579354' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-23T09:22:01.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f374acc6700 1 Processor -- start 2023-09-23T09:22:01.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f374acc6700 1 -- start start 2023-09-23T09:22:01.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f374acc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f37441094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:01.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f374acc6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744079e50 con 0x7f37441070e0 2023-09-23T09:22:01.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f3748a62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f37441094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:01.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f3748a62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f37441094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57804/0 (socket says 172.21.15.45:57804) 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f3748a62700 1 -- 172.21.15.45:0/1797942177 learned_addr learned my addr 172.21.15.45:0/1797942177 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f3748a62700 1 -- 172.21.15.45:0/1797942177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744109a10 con 0x7f37441070e0 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.589+0000 7f3748a62700 1 --2- 172.21.15.45:0/1797942177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f37441094d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f37380098a0 tx=0x7f3738009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944769d1e7cd7058 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f37437fe700 1 -- 172.21.15.45:0/1797942177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3738004160 con 0x7f37441070e0 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f37437fe700 1 -- 172.21.15.45:0/1797942177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37380042c0 con 0x7f37441070e0 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f37437fe700 1 -- 172.21.15.45:0/1797942177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37380119b0 con 0x7f37441070e0 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f374acc6700 1 -- 172.21.15.45:0/1797942177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 msgr2=0x7f37441094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f374acc6700 1 --2- 172.21.15.45:0/1797942177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f37441094d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f37380098a0 tx=0x7f3738009bb0 comp rx=0 tx=0).stop 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f374acc6700 1 -- 172.21.15.45:0/1797942177 shutdown_connections 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f374acc6700 1 --2- 172.21.15.45:0/1797942177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f37441094d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f374acc6700 1 -- 172.21.15.45:0/1797942177 >> 172.21.15.45:0/1797942177 conn(0x7f3744100ad0 msgr2=0x7f3744102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f374acc6700 1 -- 172.21.15.45:0/1797942177 shutdown_connections 2023-09-23T09:22:01.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.590+0000 7f374acc6700 1 -- 172.21.15.45:0/1797942177 wait complete. 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.591+0000 7f374acc6700 1 Processor -- start 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.591+0000 7f374acc6700 1 -- start start 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.591+0000 7f374acc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f374419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.591+0000 7f374acc6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374419c920 con 0x7f37441070e0 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.591+0000 7f3748a62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f374419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.591+0000 7f3748a62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f374419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57806/0 (socket says 172.21.15.45:57806) 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.591+0000 7f3748a62700 1 -- 172.21.15.45:0/3068569062 learned_addr learned my addr 172.21.15.45:0/3068569062 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.592+0000 7f3748a62700 1 -- 172.21.15.45:0/3068569062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738009550 con 0x7f37441070e0 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.592+0000 7f3748a62700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f374419c3e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3738011980 tx=0x7f3738011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.592+0000 7f3741ffb700 1 -- 172.21.15.45:0/3068569062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3738003a60 con 0x7f37441070e0 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.592+0000 7f3741ffb700 1 -- 172.21.15.45:0/3068569062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3738003bc0 con 0x7f37441070e0 2023-09-23T09:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.592+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374419cb20 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.592+0000 7f3741ffb700 1 -- 172.21.15.45:0/3068569062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f373801a670 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.592+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374419cf40 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.593+0000 7f3741ffb700 1 -- 172.21.15.45:0/3068569062 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45655+0+0 (secure 0 0 0) 0x7f3738024440 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.593+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3744062450 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.593+0000 7f3741ffb700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f37340389e0 0x7f373403aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.594+0000 7f3741ffb700 1 -- 172.21.15.45:0/3068569062 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f373804cb90 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.594+0000 7f3743fff700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f37340389e0 0x7f373403aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.594+0000 7f3743fff700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f37340389e0 0x7f373403aea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3730006fd0 tx=0x7f3730006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.598+0000 7f3741ffb700 1 -- 172.21.15.45:0/3068569062 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3738018d90 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.960+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f374419f980 con 0x7f37441070e0 2023-09-23T09:22:01.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.960+0000 7f3741ffb700 1 -- 172.21.15.45:0/3068569062 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f3738018350 con 0x7f37441070e0 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f37340389e0 msgr2=0x7f373403aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f37340389e0 0x7f373403aea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3730006fd0 tx=0x7f3730006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 msgr2=0x7f374419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f374419c3e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3738011980 tx=0x7f3738011a60 comp rx=0 tx=0).stop 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 shutdown_connections 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f37340389e0 0x7f373403aea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 --2- 172.21.15.45:0/3068569062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37441070e0 0x7f374419c3e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 >> 172.21.15.45:0/3068569062 conn(0x7f3744100ad0 msgr2=0x7f37441017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 shutdown_connections 2023-09-23T09:22:01.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:00.963+0000 7f374acc6700 1 -- 172.21.15.45:0/3068569062 wait complete. 2023-09-23T09:22:02.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:02 smithi045 ceph-mon[98627]: mgrmap e13: x(active, since 6s) 2023-09-23T09:22:02.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.807+0000 7f0a03a44700 1 Processor -- start 2023-09-23T09:22:02.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.807+0000 7f0a03a44700 1 -- start start 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.807+0000 7f0a03a44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc104d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.807+0000 7f0a03a44700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc105340 con 0x7f09fc104950 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.807+0000 7f0a017e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc104d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.808+0000 7f0a017e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc104d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57810/0 (socket says 172.21.15.45:57810) 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.808+0000 7f0a017e0700 1 -- 172.21.15.45:0/2133441013 learned_addr learned my addr 172.21.15.45:0/2133441013 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.808+0000 7f0a017e0700 1 -- 172.21.15.45:0/2133441013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc1078e0 con 0x7f09fc104950 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.808+0000 7f0a017e0700 1 --2- 172.21.15.45:0/2133441013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc104d70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098a0 tx=0x7f09ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edde1f12ec1f7a9f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.808+0000 7f09f3fff700 1 -- 172.21.15.45:0/2133441013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09ec004160 con 0x7f09fc104950 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.808+0000 7f09f3fff700 1 -- 172.21.15.45:0/2133441013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ec0042c0 con 0x7f09fc104950 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.808+0000 7f09f3fff700 1 -- 172.21.15.45:0/2133441013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09ec0119b0 con 0x7f09fc104950 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.809+0000 7f0a03a44700 1 -- 172.21.15.45:0/2133441013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 msgr2=0x7f09fc104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:02.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.809+0000 7f0a03a44700 1 --2- 172.21.15.45:0/2133441013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc104d70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098a0 tx=0x7f09ec009bb0 comp rx=0 tx=0).stop 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.809+0000 7f0a03a44700 1 -- 172.21.15.45:0/2133441013 shutdown_connections 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.809+0000 7f0a03a44700 1 --2- 172.21.15.45:0/2133441013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc104d70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.809+0000 7f0a03a44700 1 -- 172.21.15.45:0/2133441013 >> 172.21.15.45:0/2133441013 conn(0x7f09fc1002c0 msgr2=0x7f09fc1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.809+0000 7f0a03a44700 1 -- 172.21.15.45:0/2133441013 shutdown_connections 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.809+0000 7f0a03a44700 1 -- 172.21.15.45:0/2133441013 wait complete. 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a03a44700 1 Processor -- start 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a03a44700 1 -- start start 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a03a44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a03a44700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc19a730 con 0x7f09fc104950 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a017e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a017e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57812/0 (socket says 172.21.15.45:57812) 2023-09-23T09:22:02.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a017e0700 1 -- 172.21.15.45:0/1755332075 learned_addr learned my addr 172.21.15.45:0/1755332075 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a017e0700 1 -- 172.21.15.45:0/1755332075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009550 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.810+0000 7f0a017e0700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc19a1f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.811+0000 7f09f27fc700 1 -- 172.21.15.45:0/1755332075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09ec0039f0 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.811+0000 7f09f27fc700 1 -- 172.21.15.45:0/1755332075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ec003b50 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.811+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc19a930 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.811+0000 7f09f27fc700 1 -- 172.21.15.45:0/1755332075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09ec01a780 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.811+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc19ad50 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.811+0000 7f09f27fc700 1 -- 172.21.15.45:0/1755332075 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f09ec01aa40 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.812+0000 7f09f27fc700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09e8038a30 0x7f09e803aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.812+0000 7f09f27fc700 1 -- 172.21.15.45:0/1755332075 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f09ec04ccf0 con 0x7f09fc104950 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.812+0000 7f0a00fdf700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09e8038a30 0x7f09e803aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:02.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.812+0000 7f0a00fdf700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09e8038a30 0x7f09e803aef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f09f8006fd0 tx=0x7f09f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.812+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc062450 con 0x7f09fc104950 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:01.817+0000 7f09f27fc700 1 -- 172.21.15.45:0/1755332075 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f09ec01ad40 con 0x7f09fc104950 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.398+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f09fc10ab10 con 0x7f09fc104950 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.401+0000 7f09f27fc700 1 -- 172.21.15.45:0/1755332075 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f09ec018350 con 0x7f09fc104950 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09e8038a30 msgr2=0x7f09e803aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09e8038a30 0x7f09e803aef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f09f8006fd0 tx=0x7f09f8006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 msgr2=0x7f09fc19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc19a1f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec011a30 comp rx=0 tx=0).stop 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 shutdown_connections 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09e8038a30 0x7f09e803aef0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 --2- 172.21.15.45:0/1755332075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc104950 0x7f09fc19a1f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 >> 172.21.15.45:0/1755332075 conn(0x7f09fc1002c0 msgr2=0x7f09fc100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 shutdown_connections 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-09-23T09:22:02.403+0000 7f0a03a44700 1 -- 172.21.15.45:0/1755332075 wait complete. 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-23T09:22:02.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.785 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2023-09-23T09:22:02.785 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.785 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2023-09-23T09:22:02.785 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.785 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-23T09:22:02.785 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:02.785 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2023-09-23T09:22:02.814 INFO:tasks.cephadm:Fetching config... 2023-09-23T09:22:02.814 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:22:02.814 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-23T09:22:02.831 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-23T09:22:02.831 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:22:02.831 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-23T09:22:02.889 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-23T09:22:02.889 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:22:02.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-09-23T09:22:02.957 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-23T09:22:02.957 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:22:02.957 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-23T09:22:03.014 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-23T09:22:03.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRVSoM1blfbQpO8vxOEuCdWE2czWM3sWTCA2cEeLIQjYadsFldfh2vsamfH5n+nmRfhllcptjPQjOrHZnIrWFmaHkadm066NhgGX0AbRh+tYwilLrJrSZtjufsK8WIbdSd8Gv3Lk5WRCvDHrACJp3zrxheyMYtfhMg1q/o/Cekrsy4rmq+o/Xgs1TgP0B5sEQCHxYjtQeCdu2VoaZmWY6uNY2XMw48obrYJVYxfk103uTvuqHcEjdl9ynZGzZWBpOQzzcSwfx4TmVRwoMIW1FapwbDEq3/CqG2egy8QrsrEi+ZLwUcT6h3bg/LuW57sPlAr2tGtGtHazecKzMKYMHkts7m1nXTZCsWM+mUuB/xnoseI+6HapbxpY0iw8v/KCCY1CBKut+fmdAR88u09dAN+Bjw1X7HdTosGhHkm0jKtQxqknQGhI1XFejR9UFiVjzMb+vmeA+Y9uTPQ/BkOZO/NvD/QnbmOgNbRYDJvXPfiM2/6t5cs871wpm89aLmiQs= ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-23T09:22:03.091 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRVSoM1blfbQpO8vxOEuCdWE2czWM3sWTCA2cEeLIQjYadsFldfh2vsamfH5n+nmRfhllcptjPQjOrHZnIrWFmaHkadm066NhgGX0AbRh+tYwilLrJrSZtjufsK8WIbdSd8Gv3Lk5WRCvDHrACJp3zrxheyMYtfhMg1q/o/Cekrsy4rmq+o/Xgs1TgP0B5sEQCHxYjtQeCdu2VoaZmWY6uNY2XMw48obrYJVYxfk103uTvuqHcEjdl9ynZGzZWBpOQzzcSwfx4TmVRwoMIW1FapwbDEq3/CqG2egy8QrsrEi+ZLwUcT6h3bg/LuW57sPlAr2tGtGtHazecKzMKYMHkts7m1nXTZCsWM+mUuB/xnoseI+6HapbxpY0iw8v/KCCY1CBKut+fmdAR88u09dAN+Bjw1X7HdTosGhHkm0jKtQxqknQGhI1XFejR9UFiVjzMb+vmeA+Y9uTPQ/BkOZO/NvD/QnbmOgNbRYDJvXPfiM2/6t5cs871wpm89aLmiQs= ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:03.103 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRVSoM1blfbQpO8vxOEuCdWE2czWM3sWTCA2cEeLIQjYadsFldfh2vsamfH5n+nmRfhllcptjPQjOrHZnIrWFmaHkadm066NhgGX0AbRh+tYwilLrJrSZtjufsK8WIbdSd8Gv3Lk5WRCvDHrACJp3zrxheyMYtfhMg1q/o/Cekrsy4rmq+o/Xgs1TgP0B5sEQCHxYjtQeCdu2VoaZmWY6uNY2XMw48obrYJVYxfk103uTvuqHcEjdl9ynZGzZWBpOQzzcSwfx4TmVRwoMIW1FapwbDEq3/CqG2egy8QrsrEi+ZLwUcT6h3bg/LuW57sPlAr2tGtGtHazecKzMKYMHkts7m1nXTZCsWM+mUuB/xnoseI+6HapbxpY0iw8v/KCCY1CBKut+fmdAR88u09dAN+Bjw1X7HdTosGhHkm0jKtQxqknQGhI1XFejR9UFiVjzMb+vmeA+Y9uTPQ/BkOZO/NvD/QnbmOgNbRYDJvXPfiM2/6t5cs871wpm89aLmiQs= ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-23T09:22:03.147 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRVSoM1blfbQpO8vxOEuCdWE2czWM3sWTCA2cEeLIQjYadsFldfh2vsamfH5n+nmRfhllcptjPQjOrHZnIrWFmaHkadm066NhgGX0AbRh+tYwilLrJrSZtjufsK8WIbdSd8Gv3Lk5WRCvDHrACJp3zrxheyMYtfhMg1q/o/Cekrsy4rmq+o/Xgs1TgP0B5sEQCHxYjtQeCdu2VoaZmWY6uNY2XMw48obrYJVYxfk103uTvuqHcEjdl9ynZGzZWBpOQzzcSwfx4TmVRwoMIW1FapwbDEq3/CqG2egy8QrsrEi+ZLwUcT6h3bg/LuW57sPlAr2tGtGtHazecKzMKYMHkts7m1nXTZCsWM+mUuB/xnoseI+6HapbxpY0iw8v/KCCY1CBKut+fmdAR88u09dAN+Bjw1X7HdTosGhHkm0jKtQxqknQGhI1XFejR9UFiVjzMb+vmeA+Y9uTPQ/BkOZO/NvD/QnbmOgNbRYDJvXPfiM2/6t5cs871wpm89aLmiQs= ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:03.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRVSoM1blfbQpO8vxOEuCdWE2czWM3sWTCA2cEeLIQjYadsFldfh2vsamfH5n+nmRfhllcptjPQjOrHZnIrWFmaHkadm066NhgGX0AbRh+tYwilLrJrSZtjufsK8WIbdSd8Gv3Lk5WRCvDHrACJp3zrxheyMYtfhMg1q/o/Cekrsy4rmq+o/Xgs1TgP0B5sEQCHxYjtQeCdu2VoaZmWY6uNY2XMw48obrYJVYxfk103uTvuqHcEjdl9ynZGzZWBpOQzzcSwfx4TmVRwoMIW1FapwbDEq3/CqG2egy8QrsrEi+ZLwUcT6h3bg/LuW57sPlAr2tGtGtHazecKzMKYMHkts7m1nXTZCsWM+mUuB/xnoseI+6HapbxpY0iw8v/KCCY1CBKut+fmdAR88u09dAN+Bjw1X7HdTosGhHkm0jKtQxqknQGhI1XFejR9UFiVjzMb+vmeA+Y9uTPQ/BkOZO/NvD/QnbmOgNbRYDJvXPfiM2/6t5cs871wpm89aLmiQs= ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-23T09:22:03.207 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRVSoM1blfbQpO8vxOEuCdWE2czWM3sWTCA2cEeLIQjYadsFldfh2vsamfH5n+nmRfhllcptjPQjOrHZnIrWFmaHkadm066NhgGX0AbRh+tYwilLrJrSZtjufsK8WIbdSd8Gv3Lk5WRCvDHrACJp3zrxheyMYtfhMg1q/o/Cekrsy4rmq+o/Xgs1TgP0B5sEQCHxYjtQeCdu2VoaZmWY6uNY2XMw48obrYJVYxfk103uTvuqHcEjdl9ynZGzZWBpOQzzcSwfx4TmVRwoMIW1FapwbDEq3/CqG2egy8QrsrEi+ZLwUcT6h3bg/LuW57sPlAr2tGtGtHazecKzMKYMHkts7m1nXTZCsWM+mUuB/xnoseI+6HapbxpY0iw8v/KCCY1CBKut+fmdAR88u09dAN+Bjw1X7HdTosGhHkm0jKtQxqknQGhI1XFejR9UFiVjzMb+vmeA+Y9uTPQ/BkOZO/NvD/QnbmOgNbRYDJvXPfiM2/6t5cs871wpm89aLmiQs= ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:03.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-23T09:22:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:03 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1755332075' entity='client.admin' 2023-09-23T09:22:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 -- 172.21.15.45:0/2709988706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 msgr2=0x7f767c0ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 --2- 172.21.15.45:0/2709988706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0ffc30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f766c009910 tx=0x7f766c009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 -- 172.21.15.45:0/2709988706 shutdown_connections 2023-09-23T09:22:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 --2- 172.21.15.45:0/2709988706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0ffc30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 -- 172.21.15.45:0/2709988706 >> 172.21.15.45:0/2709988706 conn(0x7f767c0fadb0 msgr2=0x7f767c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 -- 172.21.15.45:0/2709988706 shutdown_connections 2023-09-23T09:22:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 -- 172.21.15.45:0/2709988706 wait complete. 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.138+0000 7f7681c24700 1 Processor -- start 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f7681c24700 1 -- start start 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f7681c24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f7681c24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0fd290 con 0x7f767c0ff810 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f767b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f767b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57818/0 (socket says 172.21.15.45:57818) 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f767b7fe700 1 -- 172.21.15.45:0/3931657158 learned_addr learned my addr 172.21.15.45:0/3931657158 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f767b7fe700 1 -- 172.21.15.45:0/3931657158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0095f0 con 0x7f767c0ff810 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f767b7fe700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0fcd50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f7678ff9700 1 -- 172.21.15.45:0/3931657158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f766c01d070 con 0x7f767c0ff810 2023-09-23T09:22:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c0fb3a0 con 0x7f767c0ff810 2023-09-23T09:22:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f7678ff9700 1 -- 172.21.15.45:0/3931657158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f766c0044e0 con 0x7f767c0ff810 2023-09-23T09:22:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.139+0000 7f7678ff9700 1 -- 172.21.15.45:0/3931657158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f766c0039b0 con 0x7f767c0ff810 2023-09-23T09:22:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.140+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c0fb7e0 con 0x7f767c0ff810 2023-09-23T09:22:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.140+0000 7f7678ff9700 1 -- 172.21.15.45:0/3931657158 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f766c003c70 con 0x7f767c0ff810 2023-09-23T09:22:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.140+0000 7f7678ff9700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7664038920 0x7f766403ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.141+0000 7f7678ff9700 1 -- 172.21.15.45:0/3931657158 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f766c04ca90 con 0x7f767c0ff810 2023-09-23T09:22:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.141+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c062450 con 0x7f767c0ff810 2023-09-23T09:22:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.141+0000 7f767affd700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7664038920 0x7f766403ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.141+0000 7f767affd700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7664038920 0x7f766403ade0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7670006fd0 tx=0x7f7670006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.146+0000 7f7678ff9700 1 -- 172.21.15.45:0/3931657158 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f766c00f460 con 0x7f767c0ff810 2023-09-23T09:22:05.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.348+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f767c105650 con 0x7f767c0ff810 2023-09-23T09:22:05.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.351+0000 7f7678ff9700 1 -- 172.21.15.45:0/3931657158 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f766c02a360 con 0x7f767c0ff810 2023-09-23T09:22:05.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7664038920 msgr2=0x7f766403ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:05.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7664038920 0x7f766403ade0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7670006fd0 tx=0x7f7670006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:05.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 msgr2=0x7f767c0fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:05.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0fcd50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c0057c0 comp rx=0 tx=0).stop 2023-09-23T09:22:05.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 shutdown_connections 2023-09-23T09:22:05.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7664038920 0x7f766403ade0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:05.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 --2- 172.21.15.45:0/3931657158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767c0ff810 0x7f767c0fcd50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:05.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 >> 172.21.15.45:0/3931657158 conn(0x7f767c0fadb0 msgr2=0x7f767c0688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:05.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 shutdown_connections 2023-09-23T09:22:05.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:05.353+0000 7f7681c24700 1 -- 172.21.15.45:0/3931657158 wait complete. 2023-09-23T09:22:05.759 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-23T09:22:05.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-23T09:22:06.454 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:06 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3931657158' entity='client.admin' 2023-09-23T09:22:06.533 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:07.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:07.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:07.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:22:07.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:07.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:07.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.807+0000 7f1e492a5700 1 -- 172.21.15.45:0/3914965456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 msgr2=0x7f1e440ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.807+0000 7f1e492a5700 1 --2- 172.21.15.45:0/3914965456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e440ffc40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009910 tx=0x7f1e38009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.807+0000 7f1e492a5700 1 -- 172.21.15.45:0/3914965456 shutdown_connections 2023-09-23T09:22:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.807+0000 7f1e492a5700 1 --2- 172.21.15.45:0/3914965456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e440ffc40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.807+0000 7f1e492a5700 1 -- 172.21.15.45:0/3914965456 >> 172.21.15.45:0/3914965456 conn(0x7f1e440fadc0 msgr2=0x7f1e440fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.807+0000 7f1e492a5700 1 -- 172.21.15.45:0/3914965456 shutdown_connections 2023-09-23T09:22:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.807+0000 7f1e492a5700 1 -- 172.21.15.45:0/3914965456 wait complete. 2023-09-23T09:22:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.808+0000 7f1e492a5700 1 Processor -- start 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.808+0000 7f1e492a5700 1 -- start start 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e492a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e44197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e492a5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44198180 con 0x7f1e440ff820 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e42ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e44197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e42ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e44197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57824/0 (socket says 172.21.15.45:57824) 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e42ffd700 1 -- 172.21.15.45:0/2640703202 learned_addr learned my addr 172.21.15.45:0/2640703202 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e42ffd700 1 -- 172.21.15.45:0/2640703202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380095f0 con 0x7f1e440ff820 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e42ffd700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e44197c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e38004f40 tx=0x7f1e380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e2bfff700 1 -- 172.21.15.45:0/2640703202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e3801d070 con 0x7f1e440ff820 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e2bfff700 1 -- 172.21.15.45:0/2640703202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e3800bc10 con 0x7f1e440ff820 2023-09-23T09:22:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e2bfff700 1 -- 172.21.15.45:0/2640703202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e38003a70 con 0x7f1e440ff820 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e44198380 con 0x7f1e440ff820 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.809+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e44198820 con 0x7f1e440ff820 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.810+0000 7f1e2bfff700 1 -- 172.21.15.45:0/2640703202 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f1e38003d30 con 0x7f1e440ff820 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.810+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e44105570 con 0x7f1e440ff820 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.811+0000 7f1e2bfff700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1e2c038920 0x7f1e2c03ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.811+0000 7f1e2bfff700 1 -- 172.21.15.45:0/2640703202 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1e3804c700 con 0x7f1e440ff820 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.811+0000 7f1e427fc700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1e2c038920 0x7f1e2c03ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.811+0000 7f1e427fc700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1e2c038920 0x7f1e2c03ade0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e34006fd0 tx=0x7f1e34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:07.815+0000 7f1e2bfff700 1 -- 172.21.15.45:0/2640703202 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1e3802ae20 con 0x7f1e440ff820 2023-09-23T09:22:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.019+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e44107330 con 0x7f1e2c038920 2023-09-23T09:22:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.022+0000 7f1e2bfff700 1 -- 172.21.15.45:0/2640703202 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1e44107330 con 0x7f1e2c038920 2023-09-23T09:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1e2c038920 msgr2=0x7f1e2c03ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1e2c038920 0x7f1e2c03ade0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e34006fd0 tx=0x7f1e34006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 msgr2=0x7f1e44197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e44197c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e38004f40 tx=0x7f1e380057c0 comp rx=0 tx=0).stop 2023-09-23T09:22:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 shutdown_connections 2023-09-23T09:22:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1e2c038920 0x7f1e2c03ade0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 --2- 172.21.15.45:0/2640703202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440ff820 0x7f1e44197c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 >> 172.21.15.45:0/2640703202 conn(0x7f1e440fadc0 msgr2=0x7f1e440fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 shutdown_connections 2023-09-23T09:22:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:08.024+0000 7f1e492a5700 1 -- 172.21.15.45:0/2640703202 wait complete. 2023-09-23T09:22:08.392 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2023-09-23T09:22:08.392 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:22:08.392 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2023-09-23T09:22:08.413 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:22:08.413 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:08.472 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2023-09-23T09:22:08.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch host add smithi098 2023-09-23T09:22:09.041 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:09.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:09 smithi045 ceph-mon[98627]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:09.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:09 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:09.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:09 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:09.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:09 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:09.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:09 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:09.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:09 smithi045 ceph-mon[98627]: Updating smithi045:/etc/ceph/ceph.conf 2023-09-23T09:22:10.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.245+0000 7fe21f4d3700 1 -- 172.21.15.45:0/201288397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 msgr2=0x7fe218106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:10.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.245+0000 7fe21f4d3700 1 --2- 172.21.15.45:0/201288397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe218106c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe214009910 tx=0x7fe214009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.245+0000 7fe21f4d3700 1 -- 172.21.15.45:0/201288397 shutdown_connections 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.245+0000 7fe21f4d3700 1 --2- 172.21.15.45:0/201288397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe218106c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.245+0000 7fe21f4d3700 1 -- 172.21.15.45:0/201288397 >> 172.21.15.45:0/201288397 conn(0x7fe218075320 msgr2=0x7fe218075730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.245+0000 7fe21f4d3700 1 -- 172.21.15.45:0/201288397 shutdown_connections 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.245+0000 7fe21f4d3700 1 -- 172.21.15.45:0/201288397 wait complete. 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21f4d3700 1 Processor -- start 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21f4d3700 1 -- start start 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21f4d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe2180ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21f4d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe218101860 con 0x7fe2181068b0 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21d26f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe2180ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:10.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21d26f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe2180ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57830/0 (socket says 172.21.15.45:57830) 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21d26f700 1 -- 172.21.15.45:0/3297836588 learned_addr learned my addr 172.21.15.45:0/3297836588 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21d26f700 1 -- 172.21.15.45:0/3297836588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140095f0 con 0x7fe2181068b0 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21d26f700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe2180ff890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe214006010 tx=0x7fe214004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe20e7fc700 1 -- 172.21.15.45:0/3297836588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe21401d070 con 0x7fe2181068b0 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe20e7fc700 1 -- 172.21.15.45:0/3297836588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21400bc10 con 0x7fe2181068b0 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe20e7fc700 1 -- 172.21.15.45:0/3297836588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe214003910 con 0x7fe2181068b0 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.246+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2180ffe90 con 0x7fe2181068b0 2023-09-23T09:22:10.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.247+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe218100310 con 0x7fe2181068b0 2023-09-23T09:22:10.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.247+0000 7fe20e7fc700 1 -- 172.21.15.45:0/3297836588 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7fe21400bd80 con 0x7fe2181068b0 2023-09-23T09:22:10.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.248+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe218105b60 con 0x7fe2181068b0 2023-09-23T09:22:10.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.248+0000 7fe20e7fc700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe204038970 0x7fe20403ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:10.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.248+0000 7fe20e7fc700 1 -- 172.21.15.45:0/3297836588 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe21404c680 con 0x7fe2181068b0 2023-09-23T09:22:10.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.248+0000 7fe21ca6e700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe204038970 0x7fe20403ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:10.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.248+0000 7fe21ca6e700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe204038970 0x7fe20403ae30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:10.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.253+0000 7fe20e7fc700 1 -- 172.21.15.45:0/3297836588 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe21402ad00 con 0x7fe2181068b0 2023-09-23T09:22:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:10 smithi045 ceph-mon[98627]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:10 smithi045 ceph-mon[98627]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:10 smithi045 ceph-mon[98627]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:22:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:10 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:10 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:10 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:10.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:10.456+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}) v1 -- 0x7fe218100560 con 0x7fe204038970 2023-09-23T09:22:11.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:11 smithi045 ceph-mon[98627]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:11.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:11 smithi045 ceph-mon[98627]: Deploying cephadm binary to smithi098 2023-09-23T09:22:13.339 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi098' with addr '172.21.15.98' 2023-09-23T09:22:13.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.337+0000 7fe20e7fc700 1 -- 172.21.15.45:0/3297836588 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fe218100560 con 0x7fe204038970 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe204038970 msgr2=0x7fe20403ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe204038970 0x7fe20403ae30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 msgr2=0x7fe2180ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe2180ff890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe214006010 tx=0x7fe214004ff0 comp rx=0 tx=0).stop 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 shutdown_connections 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe204038970 0x7fe20403ae30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 --2- 172.21.15.45:0/3297836588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2181068b0 0x7fe2180ff890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 >> 172.21.15.45:0/3297836588 conn(0x7fe218075320 msgr2=0x7fe2180fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 shutdown_connections 2023-09-23T09:22:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:13.340+0000 7fe21f4d3700 1 -- 172.21.15.45:0/3297836588 wait complete. 2023-09-23T09:22:13.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch host ls --format=json 2023-09-23T09:22:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:14.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:14 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:14.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:14 smithi045 ceph-mon[98627]: Added host smithi098 2023-09-23T09:22:14.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:14 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:14.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:14 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.473+0000 7f00106ee700 1 -- 172.21.15.45:0/3743924323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 msgr2=0x7f0008106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.473+0000 7f00106ee700 1 --2- 172.21.15.45:0/3743924323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008106c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efff8009910 tx=0x7efff8009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.473+0000 7f00106ee700 1 -- 172.21.15.45:0/3743924323 shutdown_connections 2023-09-23T09:22:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.473+0000 7f00106ee700 1 --2- 172.21.15.45:0/3743924323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008106c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.473+0000 7f00106ee700 1 -- 172.21.15.45:0/3743924323 >> 172.21.15.45:0/3743924323 conn(0x7f00080fc3b0 msgr2=0x7f00080fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.473+0000 7f00106ee700 1 -- 172.21.15.45:0/3743924323 shutdown_connections 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.473+0000 7f00106ee700 1 -- 172.21.15.45:0/3743924323 wait complete. 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f00106ee700 1 Processor -- start 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f00106ee700 1 -- start start 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f00106ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f00106ee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0008196280 con 0x7f00081068a0 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f000e48a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f000e48a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57856/0 (socket says 172.21.15.45:57856) 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f000e48a700 1 -- 172.21.15.45:0/91435785 learned_addr learned my addr 172.21.15.45:0/91435785 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f000e48a700 1 -- 172.21.15.45:0/91435785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff80095f0 con 0x7f00081068a0 2023-09-23T09:22:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.474+0000 7f000e48a700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008195d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efff8004f40 tx=0x7efff8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:15.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.475+0000 7effff7fe700 1 -- 172.21.15.45:0/91435785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff801d070 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.475+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00081964e0 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.475+0000 7effff7fe700 1 -- 172.21.15.45:0/91435785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efff8004370 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.475+0000 7effff7fe700 1 -- 172.21.15.45:0/91435785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff8003970 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.475+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0008196960 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.475+0000 7effff7fe700 1 -- 172.21.15.45:0/91435785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7efff8003c30 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.476+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0008062450 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.476+0000 7effff7fe700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efff4038920 0x7efff403ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.476+0000 7effff7fe700 1 -- 172.21.15.45:0/91435785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efff804c6e0 con 0x7f00081068a0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.476+0000 7f000dc89700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efff4038920 0x7efff403ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.476+0000 7f000dc89700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efff4038920 0x7efff403ade0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0004006fd0 tx=0x7f0004006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:15.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.481+0000 7effff7fe700 1 -- 172.21.15.45:0/91435785 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7efff802a590 con 0x7f00081068a0 2023-09-23T09:22:15.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:15 smithi045 ceph-mon[98627]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:15.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.683+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f000810ad20 con 0x7efff4038920 2023-09-23T09:22:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.684+0000 7effff7fe700 1 -- 172.21.15.45:0/91435785 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f000810ad20 con 0x7efff4038920 2023-09-23T09:22:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:15.686 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2023-09-23T09:22:15.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efff4038920 msgr2=0x7efff403ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:15.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efff4038920 0x7efff403ade0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0004006fd0 tx=0x7f0004006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:15.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 msgr2=0x7f0008195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:15.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008195d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efff8004f40 tx=0x7efff8005020 comp rx=0 tx=0).stop 2023-09-23T09:22:15.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 shutdown_connections 2023-09-23T09:22:15.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efff4038920 0x7efff403ade0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:15.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 --2- 172.21.15.45:0/91435785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00081068a0 0x7f0008195d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:15.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 >> 172.21.15.45:0/91435785 conn(0x7f00080fc3b0 msgr2=0x7f0008108d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:15.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 shutdown_connections 2023-09-23T09:22:15.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:15.686+0000 7f00106ee700 1 -- 172.21.15.45:0/91435785 wait complete. 2023-09-23T09:22:16.029 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2023-09-23T09:22:16.029 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:22:16.029 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2023-09-23T09:22:16.050 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:22:16.050 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:16.111 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2023-09-23T09:22:16.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch host add smithi123 2023-09-23T09:22:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:17.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:16 smithi045 ceph-mon[98627]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:22:17.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:16 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:17.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 -- 172.21.15.45:0/1046643615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 msgr2=0x7f32a0105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 --2- 172.21.15.45:0/1046643615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a0105d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3294009910 tx=0x7f3294009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 -- 172.21.15.45:0/1046643615 shutdown_connections 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 --2- 172.21.15.45:0/1046643615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a0105d30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 -- 172.21.15.45:0/1046643615 >> 172.21.15.45:0/1046643615 conn(0x7f32a0075f10 msgr2=0x7f32a0076320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 -- 172.21.15.45:0/1046643615 shutdown_connections 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 -- 172.21.15.45:0/1046643615 wait complete. 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.835+0000 7f32a5895700 1 Processor -- start 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f32a5895700 1 -- start start 2023-09-23T09:22:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f32a5895700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:17.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f32a5895700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a019afa0 con 0x7f32a0105950 2023-09-23T09:22:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f329effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f329effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57862/0 (socket says 172.21.15.45:57862) 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f329effd700 1 -- 172.21.15.45:0/1435879178 learned_addr learned my addr 172.21.15.45:0/1435879178 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f329effd700 1 -- 172.21.15.45:0/1435879178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940095f0 con 0x7f32a0105950 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.836+0000 7f329effd700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a019aa60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3294004f40 tx=0x7f3294005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.837+0000 7f32a4893700 1 -- 172.21.15.45:0/1435879178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f329401d070 con 0x7f32a0105950 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.837+0000 7f32a4893700 1 -- 172.21.15.45:0/1435879178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3294004370 con 0x7f32a0105950 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.837+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a019b200 con 0x7f32a0105950 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.837+0000 7f32a4893700 1 -- 172.21.15.45:0/1435879178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32940039b0 con 0x7f32a0105950 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.837+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a0194a50 con 0x7f32a0105950 2023-09-23T09:22:17.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.837+0000 7f32a4893700 1 -- 172.21.15.45:0/1435879178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f3294003c20 con 0x7f32a0105950 2023-09-23T09:22:17.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.838+0000 7f32a4893700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3288038970 0x7f328803ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:17.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.838+0000 7f32a4893700 1 -- 172.21.15.45:0/1435879178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f329404c710 con 0x7f32a0105950 2023-09-23T09:22:17.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.838+0000 7f329e7fc700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3288038970 0x7f328803ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:17.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.838+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a0062450 con 0x7f32a0105950 2023-09-23T09:22:17.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.838+0000 7f329e7fc700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3288038970 0x7f328803ae30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3290006fd0 tx=0x7f3290006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:17.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:17.843+0000 7f32a4893700 1 -- 172.21.15.45:0/1435879178 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3294026020 con 0x7f32a0105950 2023-09-23T09:22:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:18.045+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) v1 -- 0x7f32a0109b60 con 0x7f3288038970 2023-09-23T09:22:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:17 smithi045 ceph-mon[98627]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:22:19.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:19.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:19 smithi045 ceph-mon[98627]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:19 smithi045 ceph-mon[98627]: Deploying cephadm binary to smithi123 2023-09-23T09:22:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:19 smithi045 ceph-mon[98627]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:22:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:19 smithi045 ceph-mon[98627]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:20.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.953+0000 7f32a4893700 1 -- 172.21.15.45:0/1435879178 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f32a0109b60 con 0x7f3288038970 2023-09-23T09:22:20.955 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi123' with addr '172.21.15.123' 2023-09-23T09:22:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3288038970 msgr2=0x7f328803ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3288038970 0x7f328803ae30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3290006fd0 tx=0x7f3290006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 msgr2=0x7f32a019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a019aa60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3294004f40 tx=0x7f3294005020 comp rx=0 tx=0).stop 2023-09-23T09:22:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 shutdown_connections 2023-09-23T09:22:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3288038970 0x7f328803ae30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 --2- 172.21.15.45:0/1435879178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a0105950 0x7f32a019aa60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 >> 172.21.15.45:0/1435879178 conn(0x7f32a0075f10 msgr2=0x7f32a00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 shutdown_connections 2023-09-23T09:22:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:20.955+0000 7f32a5895700 1 -- 172.21.15.45:0/1435879178 wait complete. 2023-09-23T09:22:21.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch host ls --format=json 2023-09-23T09:22:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:22.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:22.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:22.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:22:22.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:22.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:22.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:22.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:22.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: Added host smithi123 2023-09-23T09:22:22.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:22.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.109+0000 7f76ce249700 1 -- 172.21.15.45:0/1145662615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 msgr2=0x7f76c8105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.109+0000 7f76ce249700 1 --2- 172.21.15.45:0/1145662615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8105bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009910 tx=0x7f76b8009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.109+0000 7f76ce249700 1 -- 172.21.15.45:0/1145662615 shutdown_connections 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.109+0000 7f76ce249700 1 --2- 172.21.15.45:0/1145662615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8105bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.109+0000 7f76ce249700 1 -- 172.21.15.45:0/1145662615 >> 172.21.15.45:0/1145662615 conn(0x7f76c8075320 msgr2=0x7f76c8075730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.109+0000 7f76ce249700 1 -- 172.21.15.45:0/1145662615 shutdown_connections 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.109+0000 7f76ce249700 1 -- 172.21.15.45:0/1145662615 wait complete. 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.110+0000 7f76ce249700 1 Processor -- start 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.110+0000 7f76ce249700 1 -- start start 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.110+0000 7f76ce249700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.110+0000 7f76ce249700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81984e0 con 0x7f76c8105810 2023-09-23T09:22:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57888/0 (socket says 172.21.15.45:57888) 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c77fe700 1 -- 172.21.15.45:0/3587545304 learned_addr learned my addr 172.21.15.45:0/3587545304 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c77fe700 1 -- 172.21.15.45:0/3587545304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80095f0 con 0x7f76c8105810 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c77fe700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8197fa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005850 tx=0x7f76b8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c4ff9700 1 -- 172.21.15.45:0/3587545304 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76b801d070 con 0x7f76c8105810 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c4ff9700 1 -- 172.21.15.45:0/3587545304 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76b8004370 con 0x7f76c8105810 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c81986e0 con 0x7f76c8105810 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76c4ff9700 1 -- 172.21.15.45:0/3587545304 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76b8003ab0 con 0x7f76c8105810 2023-09-23T09:22:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.111+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c8198b00 con 0x7f76c8105810 2023-09-23T09:22:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.112+0000 7f76c4ff9700 1 -- 172.21.15.45:0/3587545304 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f76b800f460 con 0x7f76c8105810 2023-09-23T09:22:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.112+0000 7f76c4ff9700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f76b0038920 0x7f76b003ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.112+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c8062450 con 0x7f76c8105810 2023-09-23T09:22:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.112+0000 7f76c4ff9700 1 -- 172.21.15.45:0/3587545304 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f76b804c8b0 con 0x7f76c8105810 2023-09-23T09:22:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.112+0000 7f76c6ffd700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f76b0038920 0x7f76b003ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.113+0000 7f76c6ffd700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f76b0038920 0x7f76b003ade0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006fd0 tx=0x7f76bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.117+0000 7f76c4ff9700 1 -- 172.21.15.45:0/3587545304 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f76b8026080 con 0x7f76c8105810 2023-09-23T09:22:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.323+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76c810cfb0 con 0x7f76b0038920 2023-09-23T09:22:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.324+0000 7f76c4ff9700 1 -- 172.21.15.45:0/3587545304 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f76c810cfb0 con 0x7f76b0038920 2023-09-23T09:22:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:22:23.326 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2023-09-23T09:22:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f76b0038920 msgr2=0x7f76b003ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f76b0038920 0x7f76b003ade0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006fd0 tx=0x7f76bc006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 msgr2=0x7f76c8197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8197fa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005850 tx=0x7f76b8004ff0 comp rx=0 tx=0).stop 2023-09-23T09:22:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 shutdown_connections 2023-09-23T09:22:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f76b0038920 0x7f76b003ade0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 --2- 172.21.15.45:0/3587545304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105810 0x7f76c8197fa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 >> 172.21.15.45:0/3587545304 conn(0x7f76c8075320 msgr2=0x7f76c80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 shutdown_connections 2023-09-23T09:22:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:23.327+0000 7f76ce249700 1 -- 172.21.15.45:0/3587545304 wait complete. 2023-09-23T09:22:23.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:23.664 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-23T09:22:23.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd crush tunables default 2023-09-23T09:22:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:24.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:23 smithi045 ceph-mon[98627]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:24.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:25.250 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:24 smithi045 ceph-mon[98627]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:22:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.396+0000 7fddc68e3700 1 -- 172.21.15.45:0/3466975245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 msgr2=0x7fddc0105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.396+0000 7fddc68e3700 1 --2- 172.21.15.45:0/3466975245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0105bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009910 tx=0x7fddb0009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.396+0000 7fddc68e3700 1 -- 172.21.15.45:0/3466975245 shutdown_connections 2023-09-23T09:22:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.396+0000 7fddc68e3700 1 --2- 172.21.15.45:0/3466975245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0105bf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.396+0000 7fddc68e3700 1 -- 172.21.15.45:0/3466975245 >> 172.21.15.45:0/3466975245 conn(0x7fddc0075320 msgr2=0x7fddc0075730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.396+0000 7fddc68e3700 1 -- 172.21.15.45:0/3466975245 shutdown_connections 2023-09-23T09:22:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.396+0000 7fddc68e3700 1 -- 172.21.15.45:0/3466975245 wait complete. 2023-09-23T09:22:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.397+0000 7fddc68e3700 1 Processor -- start 2023-09-23T09:22:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.397+0000 7fddc68e3700 1 -- start start 2023-09-23T09:22:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.397+0000 7fddc68e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.397+0000 7fddc68e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0195210 con 0x7fddc0105810 2023-09-23T09:22:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0194cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57894/0 (socket says 172.21.15.45:57894) 2023-09-23T09:22:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbffff700 1 -- 172.21.15.45:0/3760656198 learned_addr learned my addr 172.21.15.45:0/3760656198 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbffff700 1 -- 172.21.15.45:0/3760656198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb00095f0 con 0x7fddc0105810 2023-09-23T09:22:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbffff700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0194cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006010 tx=0x7fddb0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbd7fa700 1 -- 172.21.15.45:0/3760656198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddb001d070 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbd7fa700 1 -- 172.21.15.45:0/3760656198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddb00044a0 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc0195410 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddbd7fa700 1 -- 172.21.15.45:0/3760656198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddb0003a70 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.398+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc0195830 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.399+0000 7fddbd7fa700 1 -- 172.21.15.45:0/3760656198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7fddb0003cb0 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.399+0000 7fddbd7fa700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdda8038920 0x7fdda803ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.399+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc0062450 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.399+0000 7fddbd7fa700 1 -- 172.21.15.45:0/3760656198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fddb004c950 con 0x7fddc0105810 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.399+0000 7fddbf7fe700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdda8038920 0x7fdda803ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.400+0000 7fddbf7fe700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdda8038920 0x7fdda803ade0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006fd0 tx=0x7fddb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:25.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.404+0000 7fddbd7fa700 1 -- 172.21.15.45:0/3760656198 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fddb0026070 con 0x7fddc0105810 2023-09-23T09:22:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:25.613+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fddc0195ff0 con 0x7fddc0105810 2023-09-23T09:22:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.075+0000 7fddbd7fa700 1 -- 172.21.15.45:0/3760656198 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fddb002b440 con 0x7fddc0105810 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdda8038920 msgr2=0x7fdda803ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdda8038920 0x7fdda803ade0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006fd0 tx=0x7fddb4006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 msgr2=0x7fddc0194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0194cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006010 tx=0x7fddb0004ff0 comp rx=0 tx=0).stop 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 shutdown_connections 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdda8038920 0x7fdda803ade0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 --2- 172.21.15.45:0/3760656198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddc0105810 0x7fddc0194cd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 >> 172.21.15.45:0/3760656198 conn(0x7fddc0075320 msgr2=0x7fddc00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 shutdown_connections 2023-09-23T09:22:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:26.078+0000 7fddc68e3700 1 -- 172.21.15.45:0/3760656198 wait complete. 2023-09-23T09:22:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2023-09-23T09:22:26.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:26 smithi045 ceph-mon[98627]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:26.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:26 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3760656198' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-23T09:22:26.580 INFO:tasks.cephadm:Adding mon.a on smithi045 2023-09-23T09:22:26.580 INFO:tasks.cephadm:Adding mon.b on smithi098 2023-09-23T09:22:26.581 INFO:tasks.cephadm:Adding mon.c on smithi123 2023-09-23T09:22:26.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c' 2023-09-23T09:22:27.180 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3760656198' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-23T09:22:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:22:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:22:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:22:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:28 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:28.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.375+0000 7f3939454700 1 -- 172.21.15.123:0/3551420304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 msgr2=0x7f3934105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:28.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.375+0000 7f3939454700 1 --2- 172.21.15.123:0/3551420304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934105b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3924009910 tx=0x7f3924009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:28.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.375+0000 7f3939454700 1 -- 172.21.15.123:0/3551420304 shutdown_connections 2023-09-23T09:22:28.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.375+0000 7f3939454700 1 --2- 172.21.15.123:0/3551420304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934105b70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:28.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.375+0000 7f3939454700 1 -- 172.21.15.123:0/3551420304 >> 172.21.15.123:0/3551420304 conn(0x7f39340fb2a0 msgr2=0x7f39340fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:28.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.375+0000 7f3939454700 1 -- 172.21.15.123:0/3551420304 shutdown_connections 2023-09-23T09:22:28.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.375+0000 7f3939454700 1 -- 172.21.15.123:0/3551420304 wait complete. 2023-09-23T09:22:28.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.376+0000 7f3939454700 1 Processor -- start 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.376+0000 7f3939454700 1 -- start start 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.376+0000 7f3939454700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.376+0000 7f3939454700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39341970f0 con 0x7f3934105790 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.376+0000 7f3932ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.376+0000 7f3932ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.123:36454/0 (socket says 172.21.15.123:36454) 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.376+0000 7f3932ffd700 1 -- 172.21.15.123:0/1165422646 learned_addr learned my addr 172.21.15.123:0/1165422646 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.377+0000 7f3932ffd700 1 -- 172.21.15.123:0/1165422646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39240095f0 con 0x7f3934105790 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.377+0000 7f3932ffd700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934196bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3924004f40 tx=0x7f3924005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.377+0000 7f391bfff700 1 -- 172.21.15.123:0/1165422646 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f392401d070 con 0x7f3934105790 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.377+0000 7f391bfff700 1 -- 172.21.15.123:0/1165422646 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3924004370 con 0x7f3934105790 2023-09-23T09:22:28.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.377+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3934197350 con 0x7f3934105790 2023-09-23T09:22:28.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.377+0000 7f391bfff700 1 -- 172.21.15.123:0/1165422646 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3924003970 con 0x7f3934105790 2023-09-23T09:22:28.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.377+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39341977d0 con 0x7f3934105790 2023-09-23T09:22:28.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.378+0000 7f391bfff700 1 -- 172.21.15.123:0/1165422646 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f3924003c30 con 0x7f3934105790 2023-09-23T09:22:28.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.378+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3934197a10 con 0x7f3934105790 2023-09-23T09:22:28.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.378+0000 7f391bfff700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f391c038920 0x7f391c03ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:28.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.378+0000 7f391bfff700 1 -- 172.21.15.123:0/1165422646 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f392404c710 con 0x7f3934105790 2023-09-23T09:22:28.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.378+0000 7f39327fc700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f391c038920 0x7f391c03ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:28.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.379+0000 7f39327fc700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f391c038920 0x7f391c03ade0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:28.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.384+0000 7f391bfff700 1 -- 172.21.15.123:0/1165422646 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f392402a590 con 0x7f3934105790 2023-09-23T09:22:28.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.580+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f3934109a90 con 0x7f391c038920 2023-09-23T09:22:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.587+0000 7f391bfff700 1 -- 172.21.15.123:0/1165422646 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3934109a90 con 0x7f391c038920 2023-09-23T09:22:28.589 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mon update... 2023-09-23T09:22:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f391c038920 msgr2=0x7f391c03ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f391c038920 0x7f391c03ade0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 msgr2=0x7f3934196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934196bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3924004f40 tx=0x7f3924005020 comp rx=0 tx=0).stop 2023-09-23T09:22:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 shutdown_connections 2023-09-23T09:22:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f391c038920 0x7f391c03ade0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 --2- 172.21.15.123:0/1165422646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3934105790 0x7f3934196bb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 >> 172.21.15.123:0/1165422646 conn(0x7f39340fb2a0 msgr2=0x7f3934107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 shutdown_connections 2023-09-23T09:22:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:28.589+0000 7f3939454700 1 -- 172.21.15.123:0/1165422646 wait complete. 2023-09-23T09:22:29.049 DEBUG:teuthology.orchestra.run.smithi098:mon.b> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.b.service 2023-09-23T09:22:29.131 DEBUG:teuthology.orchestra.run.smithi123:mon.c> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.c.service 2023-09-23T09:22:29.135 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-23T09:22:29.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph mon dump -f json 2023-09-23T09:22:29.166 INFO:journalctl@ceph.mon.c.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:22:29.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:22:30.055 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c;count:3 2023-09-23T09:22:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:30.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:30.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:30.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:30.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:30.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:30.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:29 smithi045 ceph-mon[98627]: Deploying daemon mon.c on smithi123 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 -- 172.21.15.123:0/1535118492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 msgr2=0x7fc108105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 --2- 172.21.15.123:0/1535118492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108105bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009910 tx=0x7fc0f8009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 -- 172.21.15.123:0/1535118492 shutdown_connections 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 --2- 172.21.15.123:0/1535118492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108105bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 -- 172.21.15.123:0/1535118492 >> 172.21.15.123:0/1535118492 conn(0x7fc108075320 msgr2=0x7fc108075730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 -- 172.21.15.123:0/1535118492 shutdown_connections 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 -- 172.21.15.123:0/1535118492 wait complete. 2023-09-23T09:22:31.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.854+0000 7fc10d634700 1 Processor -- start 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.855+0000 7fc10d634700 1 -- start start 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.855+0000 7fc10d634700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.855+0000 7fc10d634700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081962f0 con 0x7fc108105810 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.855+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.855+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.123:36460/0 (socket says 172.21.15.123:36460) 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.855+0000 7fc106ffd700 1 -- 172.21.15.123:0/2575177171 learned_addr learned my addr 172.21.15.123:0/2575177171 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.855+0000 7fc106ffd700 1 -- 172.21.15.123:0/2575177171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80095f0 con 0x7fc108105810 2023-09-23T09:22:31.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.856+0000 7fc106ffd700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108195db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8006010 tx=0x7fc0f8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:31.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.856+0000 7fc0effff700 1 -- 172.21.15.123:0/2575177171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0f801d070 con 0x7fc108105810 2023-09-23T09:22:31.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.856+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1081964f0 con 0x7fc108105810 2023-09-23T09:22:31.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.856+0000 7fc0effff700 1 -- 172.21.15.123:0/2575177171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f80044a0 con 0x7fc108105810 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.856+0000 7fc0effff700 1 -- 172.21.15.123:0/2575177171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0f8003a70 con 0x7fc108105810 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.856+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc108196910 con 0x7fc108105810 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.857+0000 7fc0effff700 1 -- 172.21.15.123:0/2575177171 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7fc0f8003cb0 con 0x7fc108105810 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.857+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc108062450 con 0x7fc108105810 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.857+0000 7fc0effff700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc0f0038920 0x7fc0f003ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.857+0000 7fc0effff700 1 -- 172.21.15.123:0/2575177171 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc0f804c800 con 0x7fc108105810 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.857+0000 7fc1067fc700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc0f0038920 0x7fc0f003ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:31.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.858+0000 7fc1067fc700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc0f0038920 0x7fc0f003ade0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc006fd0 tx=0x7fc0fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:31.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:31.863+0000 7fc0effff700 1 -- 172.21.15.123:0/2575177171 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc0f8029770 con 0x7fc108105810 2023-09-23T09:22:32.159 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.157+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc108109c30 con 0x7fc108105810 2023-09-23T09:22:32.159 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.158+0000 7fc0effff700 1 -- 172.21.15.123:0/2575177171 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fc0f8026070 con 0x7fc108105810 2023-09-23T09:22:32.160 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:22:32.160 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","modified":"2023-09-23T09:20:51.489504Z","created":"2023-09-23T09:20:51.489504Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-23T09:22:32.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc0f0038920 msgr2=0x7fc0f003ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:32.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc0f0038920 0x7fc0f003ade0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc006fd0 tx=0x7fc0fc006e40 comp rx=0 tx=0).stop 2023-09-23T09:22:32.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 msgr2=0x7fc108195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:32.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108195db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8006010 tx=0x7fc0f8004ff0 comp rx=0 tx=0).stop 2023-09-23T09:22:32.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 shutdown_connections 2023-09-23T09:22:32.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc0f0038920 0x7fc0f003ade0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:32.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 --2- 172.21.15.123:0/2575177171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc108105810 0x7fc108195db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:32.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 >> 172.21.15.123:0/2575177171 conn(0x7fc108075320 msgr2=0x7fc108068020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:32.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 shutdown_connections 2023-09-23T09:22:32.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:32.160+0000 7fc10d634700 1 -- 172.21.15.123:0/2575177171 wait complete. 2023-09-23T09:22:32.163 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2023-09-23T09:22:32.200 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:31 smithi123 systemd[1]: Starting Ceph mon.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:22:32.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:32 smithi045 ceph-mon[98627]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:32.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 podman[103512]: 2023-09-23T09:22:33.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 bash[103512]: fdc165c42e56f95cc09fc82f1683ddcd96565fe027c5e9c0729396a905f1ebec 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: set uid:gid to 167:167 (ceph:ceph) 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev), process ceph-mon, pid 7 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pidfile_write: ignore empty --pid-file 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 systemd[1]: Started Ceph mon.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: load: jerasure load: lrc 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: RocksDB version: 7.9.2 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Git sha 0 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Compile date 2023-09-22 16:46:33 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: DB SUMMARY 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: DB Session ID: OVUE5XC8N4FTANL6I1DG 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: CURRENT file: CURRENT 2023-09-23T09:22:33.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: IDENTITY file: IDENTITY 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.error_if_exists: 0 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.create_if_missing: 0 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.paranoid_checks: 1 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.env: 0x562ef2332da0 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.fs: PosixFileSystem 2023-09-23T09:22:33.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.info_log: 0x562ef43cf5a0 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_file_opening_threads: 16 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.statistics: (nil) 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.use_fsync: 0 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_log_file_size: 0 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.keep_log_file_num: 1000 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.recycle_log_file_num: 0 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.allow_fallocate: 1 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.allow_mmap_reads: 0 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.allow_mmap_writes: 0 2023-09-23T09:22:33.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.use_direct_reads: 0 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.create_missing_column_families: 0 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.db_log_dir: 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.wal_dir: 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.advise_random_on_open: 1 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.db_write_buffer_size: 0 2023-09-23T09:22:33.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.write_buffer_manager: 0x562ef43874a0 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.rate_limiter: (nil) 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.wal_recovery_mode: 2 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.enable_thread_tracking: 0 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.enable_pipelined_write: 0 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.unordered_write: 0 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-23T09:22:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.row_cache: None 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.wal_filter: None 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.allow_ingest_behind: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.two_write_queues: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.manual_wal_flush: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.wal_compression: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.atomic_flush: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-23T09:22:33.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.log_readahead_size: 0 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.best_efforts_recovery: 0 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.allow_data_in_errors: 0 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.db_host_id: __hostname__ 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_background_jobs: 2 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_background_compactions: -1 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_subcompactions: 1 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-23T09:22:33.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_total_wal_size: 0 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_open_files: -1 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bytes_per_sync: 0 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_readahead_size: 0 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_background_flushes: -1 2023-09-23T09:22:33.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Compression algorithms supported: 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kZSTD supported: 0 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kXpressCompression supported: 0 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kLZ4HCCompression supported: 1 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kZlibCompression supported: 1 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kSnappyCompression supported: 1 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kLZ4Compression supported: 1 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: kBZip2Compression supported: 0 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-09-23T09:22:33.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.merge_operator: 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_filter: None 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_filter_factory: None 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.sst_partitioner_factory: None 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562ef43cf6a0) 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks: 1 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-23T09:22:33.099 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_top_level_index_and_filter: 1 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_type: 0 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_index_type: 0 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_shortening: 1 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: checksum: 4 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: no_block_cache: 0 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache: 0x562ef44251f0 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_name: BinnedLRUCache 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_options: 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: capacity : 536870912 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: num_shard_bits : 4 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: strict_capacity_limit : 0 2023-09-23T09:22:33.100 INFO:journalctl@ceph.mon.c.smithi123.stdout: high_pri_pool_ratio: 0.000 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_compressed: (nil) 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: persistent_cache: (nil) 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size: 4096 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size_deviation: 10 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_restart_interval: 16 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_block_restart_interval: 1 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: metadata_block_size: 4096 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: partition_filters: 0 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: use_delta_encoding: 1 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: filter_policy: bloomfilter 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: whole_key_filtering: 1 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: verify_compression: 0 2023-09-23T09:22:33.101 INFO:journalctl@ceph.mon.c.smithi123.stdout: read_amp_bytes_per_bit: 0 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout: format_version: 5 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_index_compression: 1 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_align: 0 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout: max_auto_readahead_size: 262144 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout: prepopulate_block_cache: 0 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout: initial_auto_readahead_size: 8192 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.write_buffer_size: 33554432 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_write_buffer_number: 2 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression: NoCompression 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression: Disabled 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.prefix_extractor: nullptr 2023-09-23T09:22:33.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.num_levels: 7 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-23T09:22:33.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.level: 32767 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.strategy: 0 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.enabled: false 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-23T09:22:33.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.target_file_size_base: 67108864 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-23T09:22:33.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-23T09:22:33.106 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-23T09:22:33.106 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-23T09:22:33.106 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.arena_block_size: 1048576 2023-09-23T09:22:33.106 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-23T09:22:33.106 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-23T09:22:33.106 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.disable_auto_compactions: 0 2023-09-23T09:22:33.107 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-23T09:22:33.107 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-23T09:22:33.107 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-23T09:22:33.107 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-23T09:22:33.107 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-23T09:22:33.107 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-23T09:22:33.107 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.table_properties_collectors: 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.inplace_update_support: 0 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.bloom_locality: 0 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.max_successive_merges: 0 2023-09-23T09:22:33.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.paranoid_file_checks: 0 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.force_consistency_checks: 1 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.report_bg_io_stats: 0 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.ttl: 2592000 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.enable_blob_files: false 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.min_blob_size: 0 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.blob_file_size: 268435456 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-23T09:22:33.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.blob_file_starting_level: 0 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f37fd61c-6112-4d0d-b57b-a207cb519ebf 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460952782261, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460952782715, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695460952, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f37fd61c-6112-4d0d-b57b-a207cb519ebf", "db_session_id": "OVUE5XC8N4FTANL6I1DG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-23T09:22:33.110 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460952782812, "job": 1, "event": "recovery_finished"} 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562ef44c2000 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: DB pointer 0x562ef44ae000 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** DB Stats ** 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-23T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T09:22:33.112 INFO:journalctl@ceph.mon.c.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T09:22:33.113 INFO:journalctl@ceph.mon.c.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout: Block cache BinnedLRUCache@0x562ef44251f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: using public_addr v2:172.21.15.123:0/0 -> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: starting mon.c rank -1 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(???) e0 preinit fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).mds e1 new map 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout: e1 2023-09-23T09:22:33.114 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout: legacy client fscid: -1 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout: No filesystems configured 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:22:33.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Deploying cephadm binary to smithi123 2023-09-23T09:22:33.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Added host smithi123 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.117 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3760656198' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3760656198' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.119 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c;count:3 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: Deploying daemon mon.c on smithi123 2023-09-23T09:22:33.120 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expand map: {default=false} 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta from 'false' to 'false' 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expanded map: {default=false} 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expand map: {default=info} 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta from 'info' to 'info' 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expanded map: {default=info} 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expand map: {default=daemon} 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expanded map: {default=daemon} 2023-09-23T09:22:33.121 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expand map: {default=debug} 2023-09-23T09:22:33.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta from 'debug' to 'debug' 2023-09-23T09:22:33.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:32 smithi123 ceph-mon[103577]: expand_channel_meta expanded map: {default=debug} 2023-09-23T09:22:33.701 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-23T09:22:33.702 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph mon dump -f json 2023-09-23T09:22:34.267 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:22:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: mon.a calling monitor election 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: mon.c calling monitor election 2023-09-23T09:22:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: fsmap 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: mgrmap e13: x(active, since 43s) 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: overall HEALTH_OK 2023-09-23T09:22:38.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:38.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:37 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: mon.a calling monitor election 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: mon.c calling monitor election 2023-09-23T09:22:38.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: fsmap 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: mgrmap e13: x(active, since 43s) 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: overall HEALTH_OK 2023-09-23T09:22:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:38.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:37 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:38.479 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.477+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1622834049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9a80035c0 msgr2=0x7fd9a8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:38.479 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.477+0000 7fd9c1c87700 1 --2- 172.21.15.123:0/1622834049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9a80035c0 0x7fd9a8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0ff820 tx=0x7fd9b4009e80 comp rx=0 tx=0).stop 2023-09-23T09:22:38.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.477+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1622834049 shutdown_connections 2023-09-23T09:22:38.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.477+0000 7fd9c1c87700 1 --2- 172.21.15.123:0/1622834049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9a80035c0 0x7fd9a8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:38.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.477+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1622834049 >> 172.21.15.123:0/1622834049 conn(0x7fd9bc0fb420 msgr2=0x7fd9bc0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:38.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.477+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1622834049 shutdown_connections 2023-09-23T09:22:38.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.477+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1622834049 wait complete. 2023-09-23T09:22:38.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.479+0000 7fd9c1c87700 1 Processor -- start 2023-09-23T09:22:38.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.479+0000 7fd9c1c87700 1 -- start start 2023-09-23T09:22:38.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9c1c87700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 0x7fd9bc195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9c1c87700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc196390 0x7fd9bc19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9c1c87700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc196920 con 0x7fd9bc196390 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9c1c87700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc196a90 con 0x7fd9a80035c0 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9bb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 0x7fd9bc195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9bb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 0x7fd9bc195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48048/0 (socket says 172.21.15.123:48048) 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9bb7fe700 1 -- 172.21.15.123:0/1997338909 learned_addr learned my addr 172.21.15.123:0/1997338909 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9bb7fe700 1 -- 172.21.15.123:0/1997338909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 msgr2=0x7fd9bc195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9bb7fe700 1 -- 172.21.15.123:0/1997338909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 msgr2=0x7fd9bc195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9bb7fe700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 0x7fd9bc195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9bb7fe700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 0x7fd9bc195e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:22:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9baffd700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc196390 0x7fd9bc19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9baffd700 1 -- 172.21.15.123:0/1997338909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 msgr2=0x7fd9bc195e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9baffd700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 0x7fd9bc195e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.480+0000 7fd9baffd700 1 -- 172.21.15.123:0/1997338909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40098d0 con 0x7fd9bc196390 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.481+0000 7fd9baffd700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc196390 0x7fd9bc19a800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b4d0 tx=0x7fd9a400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.481+0000 7fd9b8ff9700 1 -- 172.21.15.123:0/1997338909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd9a4011630 con 0x7fd9bc196390 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.481+0000 7fd9b8ff9700 1 -- 172.21.15.123:0/1997338909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9a4011c50 con 0x7fd9bc196390 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.481+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc19ada0 con 0x7fd9bc196390 2023-09-23T09:22:38.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.481+0000 7fd9b8ff9700 1 -- 172.21.15.123:0/1997338909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd9a4010d90 con 0x7fd9bc196390 2023-09-23T09:22:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.481+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc19b2f0 con 0x7fd9bc196390 2023-09-23T09:22:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.482+0000 7fd9b8ff9700 1 -- 172.21.15.123:0/1997338909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7fd9a401a400 con 0x7fd9bc196390 2023-09-23T09:22:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.482+0000 7fd9b8ff9700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd99c038750 0x7fd99c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.482+0000 7fd9b8ff9700 1 -- 172.21.15.123:0/1997338909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd9a404bf90 con 0x7fd9bc196390 2023-09-23T09:22:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.482+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc19b620 con 0x7fd9bc196390 2023-09-23T09:22:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.482+0000 7fd9bb7fe700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd99c038750 0x7fd99c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.483+0000 7fd9bb7fe700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd99c038750 0x7fd99c03ac10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4007590 tx=0x7fd9b40074a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.488+0000 7fd9b8ff9700 1 -- 172.21.15.123:0/1997338909 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd9a40193b0 con 0x7fd9bc196390 2023-09-23T09:22:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.758+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd9bc062450 con 0x7fd9bc196390 2023-09-23T09:22:38.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.758+0000 7fd9b8ff9700 1 -- 172.21.15.123:0/1997338909 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fd9a40172c0 con 0x7fd9bc196390 2023-09-23T09:22:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:22:38.760 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","modified":"2023-09-23T09:22:32.812217Z","created":"2023-09-23T09:20:51.489504Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-23T09:22:38.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd99c038750 msgr2=0x7fd99c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:38.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd99c038750 0x7fd99c03ac10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4007590 tx=0x7fd9b40074a0 comp rx=0 tx=0).stop 2023-09-23T09:22:38.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc196390 msgr2=0x7fd9bc19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc196390 0x7fd9bc19a800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b4d0 tx=0x7fd9a400b890 comp rx=0 tx=0).stop 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 shutdown_connections 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd99c038750 0x7fd99c03ac10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9a80035c0 0x7fd9bc195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 --2- 172.21.15.123:0/1997338909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc196390 0x7fd9bc19a800 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.760+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 >> 172.21.15.123:0/1997338909 conn(0x7fd9bc0fb420 msgr2=0x7fd9bc103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.761+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 shutdown_connections 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:38.761+0000 7fd9c1c87700 1 -- 172.21.15.123:0/1997338909 wait complete. 2023-09-23T09:22:38.763 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 2 2023-09-23T09:22:39.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:39.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:39.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:39.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:38 smithi123 ceph-mon[103577]: Deploying daemon mon.b on smithi098 2023-09-23T09:22:39.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:38 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/1997338909' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T09:22:39.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:39.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:39.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:39.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:39.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:38 smithi045 ceph-mon[98627]: Deploying daemon mon.b on smithi098 2023-09-23T09:22:39.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:38 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/1997338909' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T09:22:39.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:40.128 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-23T09:22:40.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph mon dump -f json 2023-09-23T09:22:40.161 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:39 smithi123 ceph-mon[103577]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:40.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:39 smithi045 ceph-mon[98627]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 systemd[1]: Starting Ceph mon.b for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:22:40.698 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:22:41.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 podman[102346]: 2023-09-23T09:22:41.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: set uid:gid to 167:167 (ceph:ceph) 2023-09-23T09:22:41.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev), process ceph-mon, pid 7 2023-09-23T09:22:41.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pidfile_write: ignore empty --pid-file 2023-09-23T09:22:41.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 bash[102346]: 9be13296a478587020a3bc1d673d872b386cc161bd077b63fe68b29bfb6ab5ec 2023-09-23T09:22:41.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: load: jerasure load: lrc 2023-09-23T09:22:41.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: RocksDB version: 7.9.2 2023-09-23T09:22:41.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Git sha 0 2023-09-23T09:22:41.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Compile date 2023-09-22 16:46:33 2023-09-23T09:22:41.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: DB SUMMARY 2023-09-23T09:22:41.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: DB Session ID: BYLJKAKHHL7K4EUYJIVF 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: CURRENT file: CURRENT 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: IDENTITY file: IDENTITY 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.error_if_exists: 0 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.create_if_missing: 0 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.paranoid_checks: 1 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-23T09:22:41.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.env: 0x55973d788da0 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.fs: PosixFileSystem 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.info_log: 0x55973e0f75a0 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_file_opening_threads: 16 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.statistics: (nil) 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.use_fsync: 0 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_log_file_size: 0 2023-09-23T09:22:41.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.keep_log_file_num: 1000 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.recycle_log_file_num: 0 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.allow_fallocate: 1 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.allow_mmap_reads: 0 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.allow_mmap_writes: 0 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.use_direct_reads: 0 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.create_missing_column_families: 0 2023-09-23T09:22:41.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.db_log_dir: 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.wal_dir: 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.advise_random_on_open: 1 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.db_write_buffer_size: 0 2023-09-23T09:22:41.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.write_buffer_manager: 0x55973e0af4a0 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.rate_limiter: (nil) 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.wal_recovery_mode: 2 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.enable_thread_tracking: 0 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.enable_pipelined_write: 0 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.unordered_write: 0 2023-09-23T09:22:41.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.row_cache: None 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.wal_filter: None 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.allow_ingest_behind: 0 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.two_write_queues: 0 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.manual_wal_flush: 0 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.wal_compression: 0 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.atomic_flush: 0 2023-09-23T09:22:41.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-23T09:22:41.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-23T09:22:41.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-23T09:22:41.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.log_readahead_size: 0 2023-09-23T09:22:41.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-23T09:22:41.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.best_efforts_recovery: 0 2023-09-23T09:22:41.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-23T09:22:41.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-23T09:22:41.053 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.allow_data_in_errors: 0 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.db_host_id: __hostname__ 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_background_jobs: 2 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_background_compactions: -1 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_subcompactions: 1 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_total_wal_size: 0 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-23T09:22:41.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_open_files: -1 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bytes_per_sync: 0 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_readahead_size: 0 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_background_flushes: -1 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Compression algorithms supported: 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kZSTD supported: 0 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kXpressCompression supported: 0 2023-09-23T09:22:41.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kLZ4HCCompression supported: 1 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kZlibCompression supported: 1 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kSnappyCompression supported: 1 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kLZ4Compression supported: 1 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: kBZip2Compression supported: 0 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.merge_operator: 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_filter: None 2023-09-23T09:22:41.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_filter_factory: None 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.sst_partitioner_factory: None 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55973e0f76a0) 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks: 1 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_top_level_index_and_filter: 1 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_type: 0 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_index_type: 0 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_shortening: 1 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-23T09:22:41.057 INFO:journalctl@ceph.mon.b.smithi098.stdout: checksum: 4 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: no_block_cache: 0 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache: 0x55973e14d1f0 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_name: BinnedLRUCache 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_options: 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: capacity : 536870912 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_shard_bits : 4 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: strict_capacity_limit : 0 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: high_pri_pool_ratio: 0.000 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_compressed: (nil) 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: persistent_cache: (nil) 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size: 4096 2023-09-23T09:22:41.058 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size_deviation: 10 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_restart_interval: 16 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_block_restart_interval: 1 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: metadata_block_size: 4096 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: partition_filters: 0 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: use_delta_encoding: 1 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: filter_policy: bloomfilter 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: whole_key_filtering: 1 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: verify_compression: 0 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: read_amp_bytes_per_bit: 0 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: format_version: 5 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_index_compression: 1 2023-09-23T09:22:41.059 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_align: 0 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout: max_auto_readahead_size: 262144 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout: prepopulate_block_cache: 0 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout: initial_auto_readahead_size: 8192 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.write_buffer_size: 33554432 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_write_buffer_number: 2 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression: NoCompression 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression: Disabled 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.prefix_extractor: nullptr 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.num_levels: 7 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-23T09:22:41.060 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-23T09:22:41.061 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.level: 32767 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.strategy: 0 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.enabled: false 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.target_file_size_base: 67108864 2023-09-23T09:22:41.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-23T09:22:41.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.arena_block_size: 1048576 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.disable_auto_compactions: 0 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-23T09:22:41.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.table_properties_collectors: 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.inplace_update_support: 0 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-23T09:22:41.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.bloom_locality: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.max_successive_merges: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.paranoid_file_checks: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.force_consistency_checks: 1 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.report_bg_io_stats: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.ttl: 2592000 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-23T09:22:41.066 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.enable_blob_files: false 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.min_blob_size: 0 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.blob_file_size: 268435456 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.blob_file_starting_level: 0 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-23T09:22:41.067 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: afc710a7-f78c-49d5-b803-35370afbd065 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460960933120, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460960933531, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695460960, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "afc710a7-f78c-49d5-b803-35370afbd065", "db_session_id": "BYLJKAKHHL7K4EUYJIVF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695460960933604, "job": 1, "event": "recovery_finished"} 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55973e1ea000 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: DB pointer 0x55973e1d6000 2023-09-23T09:22:41.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** DB Stats ** 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T09:22:41.069 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2023-09-23T09:22:41.070 INFO:journalctl@ceph.mon.b.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-23T09:22:41.071 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache BinnedLRUCache@0x55973e14d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-23T09:22:41.072 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: starting mon.b rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(???) e0 preinit fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 systemd[1]: Started Ceph mon.b for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).mds e1 new map 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout: e1 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout: legacy client fscid: -1 2023-09-23T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout: No filesystems configured 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:22:41.074 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:22:41.075 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Deploying cephadm binary to smithi123 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Added host smithi123 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3760656198' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-23T09:22:41.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3760656198' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:22:41.078 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:41.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c;count:3 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Deploying daemon mon.c on smithi123 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:22:41.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.a calling monitor election 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.c calling monitor election 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: fsmap 2023-09-23T09:22:41.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mgrmap e13: x(active, since 43s) 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: overall HEALTH_OK 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: Deploying daemon mon.b on smithi098 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/1997338909' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:41.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expand map: {default=false} 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta from 'false' to 'false' 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expanded map: {default=false} 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expand map: {default=info} 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta from 'info' to 'info' 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expanded map: {default=info} 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expand map: {default=daemon} 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expanded map: {default=daemon} 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expand map: {default=debug} 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta from 'debug' to 'debug' 2023-09-23T09:22:41.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:40 smithi098 ceph-mon[102386]: expand_channel_meta expanded map: {default=debug} 2023-09-23T09:22:46.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.288+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2640811479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 msgr2=0x7f5e98005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:46.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.288+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2640811479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5e98005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0076810 tx=0x7f5ea000fe60 comp rx=0 tx=0).stop 2023-09-23T09:22:46.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.288+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2640811479 shutdown_connections 2023-09-23T09:22:46.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.288+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2640811479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5e98005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:46.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.288+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2640811479 >> 172.21.15.123:0/2640811479 conn(0x7f5eb00753d0 msgr2=0x7f5eb00757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:46.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.288+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2640811479 shutdown_connections 2023-09-23T09:22:46.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.288+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2640811479 wait complete. 2023-09-23T09:22:46.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.289+0000 7f5eb67c2700 1 Processor -- start 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.289+0000 7f5eb67c2700 1 -- start start 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb67c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5eb018cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb67c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 0x7f5eb018d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb67c2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 0x7f5eb01a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb67c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01a90a0 con 0x7f5e980035c0 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb67c2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01a91e0 con 0x7f5eb01a6770 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb67c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01a9320 con 0x7f5eb018d2c0 2023-09-23T09:22:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 0x7f5eb018d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5eb018cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 0x7f5eb018d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48078/0 (socket says 172.21.15.123:48078) 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaf7fe700 1 -- 172.21.15.123:0/2007739545 learned_addr learned my addr 172.21.15.123:0/2007739545 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaffff700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5eb018cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.123:36518/0 (socket says 172.21.15.123:36518) 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaf7fe700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 msgr2=0x7f5eb018d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaf7fe700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 msgr2=0x7f5eb018d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaf7fe700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 0x7f5eb018d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-23T09:22:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb4d5f700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 0x7f5eb01a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaf7fe700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 0x7f5eb018d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb4d5f700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 msgr2=0x7f5eb01a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb4d5f700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 msgr2=0x7f5eb01a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb4d5f700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 0x7f5eb01a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eb4d5f700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 0x7f5eb01a8b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaffff700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 msgr2=0x7f5eb018d740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaffff700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 0x7f5eb018d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaffff700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 msgr2=0x7f5eb01a8b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:22:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.290+0000 7f5eaffff700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 0x7f5eb01a8b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.291+0000 7f5eaffff700 1 -- 172.21.15.123:0/2007739545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea000f8b0 con 0x7f5e980035c0 2023-09-23T09:22:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.291+0000 7f5eaffff700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5eb018cd80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0020040 tx=0x7f5ea0012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.291+0000 7f5ead7fa700 1 -- 172.21.15.123:0/2007739545 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00040b0 con 0x7f5e980035c0 2023-09-23T09:22:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.291+0000 7f5ead7fa700 1 -- 172.21.15.123:0/2007739545 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea0004210 con 0x7f5e980035c0 2023-09-23T09:22:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.291+0000 7f5ead7fa700 1 -- 172.21.15.123:0/2007739545 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0027790 con 0x7f5e980035c0 2023-09-23T09:22:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.291+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb01a95a0 con 0x7f5e980035c0 2023-09-23T09:22:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.291+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb01a9aa0 con 0x7f5e980035c0 2023-09-23T09:22:46.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.292+0000 7f5ead7fa700 1 -- 172.21.15.123:0/2007739545 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f5ea0024050 con 0x7f5e980035c0 2023-09-23T09:22:46.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.292+0000 7f5ead7fa700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5e9c038df0 0x7f5e9c03b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:46.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.292+0000 7f5ead7fa700 1 -- 172.21.15.123:0/2007739545 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ea0000e40 con 0x7f5e980035c0 2023-09-23T09:22:46.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.293+0000 7f5eaf7fe700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5e9c038df0 0x7f5e9c03b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:46.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.293+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb0062450 con 0x7f5e980035c0 2023-09-23T09:22:46.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.293+0000 7f5eaf7fe700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5e9c038df0 0x7f5e9c03b2b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009180 tx=0x7f5ea4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.298+0000 7f5ead7fa700 1 -- 172.21.15.123:0/2007739545 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5ea002b020 con 0x7f5e980035c0 2023-09-23T09:22:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: mon.c calling monitor election 2023-09-23T09:22:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: mon.a calling monitor election 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: fsmap 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: mgrmap e13: x(active, since 51s) 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:46.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: mon.c calling monitor election 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: mon.a calling monitor election 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: fsmap 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: mgrmap e13: x(active, since 51s) 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:46.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.588+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5eb01075d0 con 0x7f5e980035c0 2023-09-23T09:22:46.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.589+0000 7f5ead7fa700 1 -- 172.21.15.123:0/2007739545 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f5ea0022070 con 0x7f5e980035c0 2023-09-23T09:22:46.591 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-23T09:22:46.591 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","modified":"2023-09-23T09:22:40.955715Z","created":"2023-09-23T09:20:51.489504Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-23T09:22:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5e9c038df0 msgr2=0x7f5e9c03b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5e9c038df0 0x7f5e9c03b2b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009180 tx=0x7f5ea4008040 comp rx=0 tx=0).stop 2023-09-23T09:22:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 msgr2=0x7f5eb018cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5eb018cd80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0020040 tx=0x7f5ea0012d50 comp rx=0 tx=0).stop 2023-09-23T09:22:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 shutdown_connections 2023-09-23T09:22:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5eb01a6770 0x7f5eb01a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e980035c0 0x7f5eb018cd80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5e9c038df0 0x7f5e9c03b2b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 --2- 172.21.15.123:0/2007739545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb018d2c0 0x7f5eb018d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 >> 172.21.15.123:0/2007739545 conn(0x7f5eb00753d0 msgr2=0x7f5eb01033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 shutdown_connections 2023-09-23T09:22:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:46.591+0000 7f5eb67c2700 1 -- 172.21.15.123:0/2007739545 wait complete. 2023-09-23T09:22:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 3 2023-09-23T09:22:46.963 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-23T09:22:46.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph config generate-minimal-conf 2023-09-23T09:22:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:47 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/2007739545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T09:22:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:47.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:47.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:47 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/2007739545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T09:22:47.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:47.545 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: mon.c calling monitor election 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: mon.a calling monitor election 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: fsmap 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: mgrmap e13: x(active, since 51s) 2023-09-23T09:22:48.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/2007739545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T09:22:48.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.940+0000 7f411d8fc700 1 -- 172.21.15.45:0/1361184660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41180ff840 msgr2=0x7f41180ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.940+0000 7f411d8fc700 1 --2- 172.21.15.45:0/1361184660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41180ff840 0x7f41180ffc20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4108009910 tx=0x7f4108009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.940+0000 7f411d8fc700 1 -- 172.21.15.45:0/1361184660 shutdown_connections 2023-09-23T09:22:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.940+0000 7f411d8fc700 1 --2- 172.21.15.45:0/1361184660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41180ff840 0x7f41180ffc20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.940+0000 7f411d8fc700 1 -- 172.21.15.45:0/1361184660 >> 172.21.15.45:0/1361184660 conn(0x7f41180fb2a0 msgr2=0x7f41180fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.940+0000 7f411d8fc700 1 -- 172.21.15.45:0/1361184660 shutdown_connections 2023-09-23T09:22:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.940+0000 7f411d8fc700 1 -- 172.21.15.45:0/1361184660 wait complete. 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.941+0000 7f411d8fc700 1 Processor -- start 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.941+0000 7f411d8fc700 1 -- start start 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f411d8fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41180ff840 0x7f411819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f411d8fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411819cac0 0x7f41181a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f411d8fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41181a1420 0x7f41181a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f411d8fc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411819d1b0 con 0x7f411819cac0 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f411d8fc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181a9190 con 0x7f41181a1420 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f411d8fc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181a92d0 con 0x7f41180ff840 2023-09-23T09:22:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411819cac0 0x7f41181a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411819cac0 0x7f41181a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57902/0 (socket says 172.21.15.45:57902) 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 -- 172.21.15.45:0/2452753879 learned_addr learned my addr 172.21.15.45:0/2452753879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f4116ffd700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41180ff840 0x7f411819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 -- 172.21.15.45:0/2452753879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41180ff840 msgr2=0x7f411819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41180ff840 0x7f411819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41177fe700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41181a1420 0x7f41181a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 -- 172.21.15.45:0/2452753879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41181a1420 msgr2=0x7f41181a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41181a1420 0x7f41181a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 -- 172.21.15.45:0/2452753879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080095f0 con 0x7f411819cac0 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41177fe700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41181a1420 0x7f41181a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.942+0000 7f41167fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411819cac0 0x7f41181a0ee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f410c00b780 tx=0x7f410c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.943+0000 7f411c8fa700 1 -- 172.21.15.45:0/2452753879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c004230 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.943+0000 7f411c8fa700 1 -- 172.21.15.45:0/2452753879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f410c004390 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.943+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41181a95b0 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.943+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41181a9b00 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.943+0000 7f411c8fa700 1 -- 172.21.15.45:0/2452753879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c011b30 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.944+0000 7f411c8fa700 1 -- 172.21.15.45:0/2452753879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f410c01a400 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.944+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4118062450 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.944+0000 7f411c8fa700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4100038da0 0x7f410003b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.944+0000 7f411c8fa700 1 -- 172.21.15.45:0/2452753879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f410c04b000 con 0x7f411819cac0 2023-09-23T09:22:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.944+0000 7f4116ffd700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4100038da0 0x7f410003b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:48.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.945+0000 7f4116ffd700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4100038da0 0x7f410003b260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4108006010 tx=0x7f410800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:48.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:48.949+0000 7f411c8fa700 1 -- 172.21.15.45:0/2452753879 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f410c011d90 con 0x7f411819cac0 2023-09-23T09:22:49.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: mon.b calling monitor election 2023-09-23T09:22:49.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Updating smithi045:/etc/ceph/ceph.conf 2023-09-23T09:22:49.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:22:49.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:22:49.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: mon.b calling monitor election 2023-09-23T09:22:49.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: mon.c calling monitor election 2023-09-23T09:22:49.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: mon.a calling monitor election 2023-09-23T09:22:49.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-23T09:22:49.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:49.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: fsmap 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: mgrmap e13: x(active, since 53s) 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Cluster is now healthy 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: overall HEALTH_OK 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:49.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:49.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: mon.b calling monitor election 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Updating smithi045:/etc/ceph/ceph.conf 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: mon.b calling monitor election 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: mon.c calling monitor election 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: mon.a calling monitor election 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:49.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: fsmap 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: mgrmap e13: x(active, since 53s) 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Cluster is now healthy 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: overall HEALTH_OK 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:49.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:49.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.160+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f41181112e0 con 0x7f411819cac0 2023-09-23T09:22:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.161+0000 7f411c8fa700 1 -- 172.21.15.45:0/2452753879 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f410c00f2e0 con 0x7f411819cac0 2023-09-23T09:22:49.163 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:49.163 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2023-09-23T09:22:49.163 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:22:49.163 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4100038da0 msgr2=0x7f410003b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4100038da0 0x7f410003b260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4108006010 tx=0x7f410800b540 comp rx=0 tx=0).stop 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411819cac0 msgr2=0x7f41181a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411819cac0 0x7f41181a0ee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f410c00b780 tx=0x7f410c00bb40 comp rx=0 tx=0).stop 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 shutdown_connections 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41181a1420 0x7f41181a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4100038da0 0x7f410003b260 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:49.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41180ff840 0x7f411819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:49.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 --2- 172.21.15.45:0/2452753879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411819cac0 0x7f41181a0ee0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:49.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 >> 172.21.15.45:0/2452753879 conn(0x7f41180fb2a0 msgr2=0x7f41180fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:49.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 shutdown_connections 2023-09-23T09:22:49.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:49.164+0000 7f411d8fc700 1 -- 172.21.15.45:0/2452753879 wait complete. 2023-09-23T09:22:49.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: mon.b calling monitor election 2023-09-23T09:22:49.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Updating smithi045:/etc/ceph/ceph.conf 2023-09-23T09:22:49.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:22:49.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:22:49.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: mon.b calling monitor election 2023-09-23T09:22:49.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: mon.c calling monitor election 2023-09-23T09:22:49.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: mon.a calling monitor election 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: fsmap 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: mgrmap e13: x(active, since 53s) 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Cluster is now healthy 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: overall HEALTH_OK 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:49.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:49.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:49.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:49.595 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-23T09:22:49.595 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:22:49.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2023-09-23T09:22:49.626 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:22:49.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:49.693 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:22:49.693 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2023-09-23T09:22:49.729 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:22:49.729 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:49.800 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:22:49.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2023-09-23T09:22:49.837 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:22:49.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:22:49.909 INFO:tasks.cephadm:Adding mgr.x on smithi045 2023-09-23T09:22:49.910 INFO:tasks.cephadm:Adding mgr.y on smithi098 2023-09-23T09:22:49.910 INFO:tasks.cephadm:Adding mgr.z on smithi123 2023-09-23T09:22:49.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch apply mgr '3;smithi045=x;smithi098=y;smithi123=z' 2023-09-23T09:22:50.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:49 smithi098 ceph-mon[102386]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:50.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:49 smithi098 ceph-mon[102386]: Reconfiguring mon.a (unknown last config time)... 2023-09-23T09:22:50.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:49 smithi098 ceph-mon[102386]: Reconfiguring daemon mon.a on smithi045 2023-09-23T09:22:50.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:49 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:50.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:49 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2452753879' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:50.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:49 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:49 smithi045 ceph-mon[98627]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:49 smithi045 ceph-mon[98627]: Reconfiguring mon.a (unknown last config time)... 2023-09-23T09:22:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:49 smithi045 ceph-mon[98627]: Reconfiguring daemon mon.a on smithi045 2023-09-23T09:22:50.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:49 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:50.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:49 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2452753879' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:50.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:49 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:49 smithi123 ceph-mon[103577]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:49 smithi123 ceph-mon[103577]: Reconfiguring mon.a (unknown last config time)... 2023-09-23T09:22:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:49 smithi123 ceph-mon[103577]: Reconfiguring daemon mon.a on smithi045 2023-09-23T09:22:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:49 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:22:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:49 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2452753879' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:49 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:50.505 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:22:51.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:50 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:51.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:50 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:51.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:50 smithi098 ceph-mon[102386]: Reconfiguring mon.b (monmap changed)... 2023-09-23T09:22:51.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:50 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:51.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:50 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:51.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:50 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:51.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:50 smithi098 ceph-mon[102386]: Reconfiguring daemon mon.b on smithi098 2023-09-23T09:22:51.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:50 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:51.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:50 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:51.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:50 smithi045 ceph-mon[98627]: Reconfiguring mon.b (monmap changed)... 2023-09-23T09:22:51.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:50 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:51.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:50 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:51.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:50 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:51.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:50 smithi045 ceph-mon[98627]: Reconfiguring daemon mon.b on smithi098 2023-09-23T09:22:51.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:50 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:51.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:50 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:51.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:50 smithi123 ceph-mon[103577]: Reconfiguring mon.b (monmap changed)... 2023-09-23T09:22:51.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:50 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:51.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:50 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:51.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:50 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:51.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:50 smithi123 ceph-mon[103577]: Reconfiguring daemon mon.b on smithi098 2023-09-23T09:22:51.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.672+0000 7fd730787700 1 -- 172.21.15.123:0/1458676739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280ff940 msgr2=0x7fd7280ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:51.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.672+0000 7fd730787700 1 --2- 172.21.15.123:0/1458676739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280ff940 0x7fd7280ffd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd718009910 tx=0x7fd718009c20 comp rx=0 tx=0).stop 2023-09-23T09:22:51.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.672+0000 7fd730787700 1 -- 172.21.15.123:0/1458676739 shutdown_connections 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.672+0000 7fd730787700 1 --2- 172.21.15.123:0/1458676739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280ff940 0x7fd7280ffd60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.672+0000 7fd730787700 1 -- 172.21.15.123:0/1458676739 >> 172.21.15.123:0/1458676739 conn(0x7fd7280faec0 msgr2=0x7fd7280fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 -- 172.21.15.123:0/1458676739 shutdown_connections 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 -- 172.21.15.123:0/1458676739 wait complete. 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 Processor -- start 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 -- start start 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7280ff940 0x7fd728069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd728198a00 0x7fd72819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72819b380 0x7fd72819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72819dfb0 con 0x7fd728198a00 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72819e120 con 0x7fd72819b380 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.673+0000 7fd730787700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72819e2e0 con 0x7fd7280ff940 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7280ff940 0x7fd728069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7280ff940 0x7fd728069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48086/0 (socket says 172.21.15.123:48086) 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 -- 172.21.15.123:0/1657902214 learned_addr learned my addr 172.21.15.123:0/1657902214 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72ed24700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72819b380 0x7fd72819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 -- 172.21.15.123:0/1657902214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72819b380 msgr2=0x7fd72819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72819b380 0x7fd72819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 -- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd728198a00 msgr2=0x7fd72819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72dd22700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd728198a00 0x7fd72819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd728198a00 0x7fd72819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 -- 172.21.15.123:0/1657902214 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180095f0 con 0x7fd7280ff940 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72ed24700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72819b380 0x7fd72819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72dd22700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd728198a00 0x7fd72819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd72e523700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7280ff940 0x7fd728069970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd718000c00 tx=0x7fd718004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd71f7fe700 1 -- 172.21.15.123:0/1657902214 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71801d070 con 0x7fd7280ff940 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72819e560 con 0x7fd7280ff940 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd71f7fe700 1 -- 172.21.15.123:0/1657902214 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71800be20 con 0x7fd7280ff940 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd71f7fe700 1 -- 172.21.15.123:0/1657902214 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800f460 con 0x7fd7280ff940 2023-09-23T09:22:51.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.674+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72819ea80 con 0x7fd7280ff940 2023-09-23T09:22:51.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.675+0000 7fd71f7fe700 1 -- 172.21.15.123:0/1657902214 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7fd71800f5c0 con 0x7fd7280ff940 2023-09-23T09:22:51.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.675+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72818e400 con 0x7fd7280ff940 2023-09-23T09:22:51.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.675+0000 7fd71f7fe700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd714038da0 0x7fd71403b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:51.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.675+0000 7fd71f7fe700 1 -- 172.21.15.123:0/1657902214 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd71804cd90 con 0x7fd7280ff940 2023-09-23T09:22:51.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.676+0000 7fd72dd22700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd714038da0 0x7fd71403b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:51.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.676+0000 7fd72dd22700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd714038da0 0x7fd71403b260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd728069ef0 tx=0x7fd724008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:51.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.680+0000 7fd71f7fe700 1 -- 172.21.15.123:0/1657902214 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd718020900 con 0x7fd7280ff940 2023-09-23T09:22:51.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.892+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi098=y;smithi123=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fd72802d1f0 con 0x7fd714038da0 2023-09-23T09:22:51.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.899+0000 7fd71f7fe700 1 -- 172.21.15.123:0/1657902214 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd72802d1f0 con 0x7fd714038da0 2023-09-23T09:22:51.901 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mgr update... 2023-09-23T09:22:51.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd714038da0 msgr2=0x7fd71403b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:51.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd714038da0 0x7fd71403b260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd728069ef0 tx=0x7fd724008040 comp rx=0 tx=0).stop 2023-09-23T09:22:51.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7280ff940 msgr2=0x7fd728069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7280ff940 0x7fd728069970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd718000c00 tx=0x7fd718004d60 comp rx=0 tx=0).stop 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 shutdown_connections 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72819b380 0x7fd72819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd714038da0 0x7fd71403b260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7280ff940 0x7fd728069970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 --2- 172.21.15.123:0/1657902214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd728198a00 0x7fd72819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.901+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 >> 172.21.15.123:0/1657902214 conn(0x7fd7280faec0 msgr2=0x7fd7280fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.902+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 shutdown_connections 2023-09-23T09:22:51.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:22:51.902+0000 7fd730787700 1 -- 172.21.15.123:0/1657902214 wait complete. 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: Reconfiguring mon.c (monmap changed)... 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:52.336 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: Reconfiguring mon.c (monmap changed)... 2023-09-23T09:22:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.577 DEBUG:teuthology.orchestra.run.smithi098:mgr.y> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.y.service 2023-09-23T09:22:52.580 DEBUG:teuthology.orchestra.run.smithi123:mgr.z> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.z.service 2023-09-23T09:22:52.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: Reconfiguring mon.c (monmap changed)... 2023-09-23T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:52.583 INFO:tasks.cephadm:Deploying OSDs... 2023-09-23T09:22:52.583 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:22:52.583 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T09:22:52.605 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T09:22:52.605 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2023-09-23T09:22:52.612 INFO:journalctl@ceph.mgr.z.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:22:52.613 INFO:journalctl@ceph.mgr.y.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:22:52.665 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 380727 Links: 1 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-09-23 09:22:05.667900800 +0000 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-09-23 09:16:58.680377773 +0000 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-09-23 09:16:58.680377773 +0000 2023-09-23T09:22:52.666 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-09-23T09:22:52.666 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T09:22:52.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-09-23T09:22:52.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-09-23T09:22:52.737 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000131465 s, 3.9 MB/s 2023-09-23T09:22:52.738 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T09:22:52.799 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2023-09-23T09:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T09:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 392348 Links: 1 2023-09-23T09:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-09-23 09:22:05.727899909 +0000 2023-09-23T09:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-09-23 09:16:59.185370204 +0000 2023-09-23T09:22:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-09-23 09:16:59.185370204 +0000 2023-09-23T09:22:52.860 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-09-23T09:22:52.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T09:22:52.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-09-23T09:22:52.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-09-23T09:22:52.928 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000122938 s, 4.2 MB/s 2023-09-23T09:22:52.929 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T09:22:52.989 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2023-09-23T09:22:53.048 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T09:22:53.048 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:53.048 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 392577 Links: 1 2023-09-23T09:22:53.049 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:53.049 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:53.049 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-09-23 09:22:05.790898973 +0000 2023-09-23T09:22:53.049 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-09-23 09:16:59.711362320 +0000 2023-09-23T09:22:53.049 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-09-23 09:16:59.711362320 +0000 2023-09-23T09:22:53.049 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-09-23T09:22:53.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T09:22:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-09-23T09:22:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-09-23T09:22:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.00012021 s, 4.3 MB/s 2023-09-23T09:22:53.118 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T09:22:53.177 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2023-09-23T09:22:53.234 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 392727 Links: 1 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-09-23 09:22:05.852898052 +0000 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-09-23 09:17:00.234354481 +0000 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-09-23 09:17:00.234354481 +0000 2023-09-23T09:22:53.235 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-09-23T09:22:53.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: Reconfiguring daemon mon.c on smithi123 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi098=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: Saving service mgr spec with placement smithi045=x;smithi098=y;smithi123=z;count:3 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:53.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:53.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:53.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T09:22:53.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:53.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:53 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:53.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-09-23T09:22:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-09-23T09:22:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000117085 s, 4.4 MB/s 2023-09-23T09:22:53.302 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T09:22:53.360 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-23T09:22:53.360 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T09:22:53.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T09:22:53.380 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 381767 Links: 1 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-23 09:22:40.409931641 +0000 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-23 09:16:49.492318943 +0000 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-23 09:16:49.492318943 +0000 2023-09-23T09:22:53.440 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-23T09:22:53.440 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T09:22:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: Reconfiguring daemon mon.c on smithi123 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi098=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: Saving service mgr spec with placement smithi045=x;smithi098=y;smithi123=z;count:3 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T09:22:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:53.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:53 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:53.509 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-23T09:22:53.509 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-23T09:22:53.509 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000228891 s, 2.2 MB/s 2023-09-23T09:22:53.510 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T09:22:53.569 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2023-09-23T09:22:53.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: Reconfiguring daemon mon.c on smithi123 2023-09-23T09:22:53.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi098=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:22:53.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: Saving service mgr spec with placement smithi045=x;smithi098=y;smithi123=z;count:3 2023-09-23T09:22:53.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:53.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:53 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 386871 Links: 1 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-23 09:22:40.409931641 +0000 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-23 09:16:49.985313061 +0000 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-23 09:16:49.985313061 +0000 2023-09-23T09:22:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-23T09:22:53.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T09:22:53.697 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-23T09:22:53.697 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-23T09:22:53.697 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000251964 s, 2.0 MB/s 2023-09-23T09:22:53.698 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T09:22:53.757 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2023-09-23T09:22:53.815 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T09:22:53.815 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:53.815 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 385277 Links: 1 2023-09-23T09:22:53.815 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:53.815 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:53.815 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-23 09:22:40.409931641 +0000 2023-09-23T09:22:53.816 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-23 09:16:50.482307132 +0000 2023-09-23T09:22:53.816 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-23 09:16:50.482307132 +0000 2023-09-23T09:22:53.816 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-23T09:22:53.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T09:22:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-23T09:22:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-23T09:22:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000222986 s, 2.3 MB/s 2023-09-23T09:22:53.882 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T09:22:53.940 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2023-09-23T09:22:53.997 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T09:22:53.997 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:53.997 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 385381 Links: 1 2023-09-23T09:22:53.997 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:53.998 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:53.998 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-23 09:22:40.409931641 +0000 2023-09-23T09:22:53.998 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-23 09:16:50.994301024 +0000 2023-09-23T09:22:53.998 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-23 09:16:50.994301024 +0000 2023-09-23T09:22:53.998 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-23T09:22:53.998 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T09:22:54.037 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:53 smithi123 systemd[1]: Starting Ceph mgr.z for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:22:54.062 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-23T09:22:54.062 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-23T09:22:54.062 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000180284 s, 2.8 MB/s 2023-09-23T09:22:54.063 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T09:22:54.120 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-23T09:22:54.120 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T09:22:54.139 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T09:22:54.139 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2023-09-23T09:22:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T09:22:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:54.197 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 384594 Links: 1 2023-09-23T09:22:54.197 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:54.197 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:54.197 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-23 09:22:53.690063738 +0000 2023-09-23T09:22:54.197 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-23 09:16:49.195625608 +0000 2023-09-23T09:22:54.197 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-23 09:16:49.195625608 +0000 2023-09-23T09:22:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-23T09:22:54.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T09:22:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-23T09:22:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-23T09:22:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00025152 s, 2.0 MB/s 2023-09-23T09:22:54.266 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T09:22:54.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:54 smithi098 ceph-mon[102386]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:54.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:54 smithi098 ceph-mon[102386]: Deploying daemon mgr.z on smithi123 2023-09-23T09:22:54.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-mon[103577]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:54.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-mon[103577]: Deploying daemon mgr.z on smithi123 2023-09-23T09:22:54.298 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 podman[105421]: 2023-09-23T09:22:54.316 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2023-09-23T09:22:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T09:22:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:54.375 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 388854 Links: 1 2023-09-23T09:22:54.375 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:54.375 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:54.376 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-23 09:22:53.691063715 +0000 2023-09-23T09:22:54.376 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-23 09:16:49.709613618 +0000 2023-09-23T09:22:54.376 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-23 09:16:49.709613618 +0000 2023-09-23T09:22:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-23T09:22:54.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T09:22:54.443 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-23T09:22:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-23T09:22:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000228656 s, 2.2 MB/s 2023-09-23T09:22:54.445 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T09:22:54.503 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2023-09-23T09:22:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T09:22:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:54.560 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 387008 Links: 1 2023-09-23T09:22:54.560 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:54.560 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:54.561 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-23 09:22:53.691063715 +0000 2023-09-23T09:22:54.561 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-23 09:16:50.210601930 +0000 2023-09-23T09:22:54.561 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-23 09:16:50.210601930 +0000 2023-09-23T09:22:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-23T09:22:54.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T09:22:54.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:54 smithi045 ceph-mon[98627]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:54.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:54 smithi045 ceph-mon[98627]: Deploying daemon mgr.z on smithi123 2023-09-23T09:22:54.588 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 bash[105421]: 8df500528090a102b2745c88dcc8db5906c13c45c2802cb85aae14b636f77a13 2023-09-23T09:22:54.588 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-mgr[105520]: -- 172.21.15.123:0/3581962543 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fc1e47f1e0 con 0x55fc1e454800 2023-09-23T09:22:54.588 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 systemd[1]: Started Ceph mgr.z for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:22:54.588 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:54.429+0000 7fae9c693200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:22:54.589 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:54.515+0000 7fae9c693200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:22:54.613 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-23T09:22:54.613 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-23T09:22:54.613 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000208889 s, 2.5 MB/s 2023-09-23T09:22:54.614 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T09:22:54.672 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2023-09-23T09:22:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T09:22:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T09:22:54.730 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 391183 Links: 1 2023-09-23T09:22:54.730 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T09:22:54.730 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T09:22:54.730 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-23 09:22:53.691063715 +0000 2023-09-23T09:22:54.730 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-23 09:16:50.739589590 +0000 2023-09-23T09:22:54.730 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-23 09:16:50.739589590 +0000 2023-09-23T09:22:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-23T09:22:54.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T09:22:54.794 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-23T09:22:54.794 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-23T09:22:54.794 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000196427 s, 2.6 MB/s 2023-09-23T09:22:54.795 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T09:22:54.851 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2023-09-23T09:22:54.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-23T09:22:54.892 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:54.613+0000 7fae9c693200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:22:54.892 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:54.711+0000 7fae9c693200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:22:55.255 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:54.890+0000 7fae9c693200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:22:55.425 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:55.565 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:55.254+0000 7fae9c693200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:22:55.565 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:55.382+0000 7fae9c693200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:22:55.565 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:55.470+0000 7fae9c693200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:55.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:55.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-mon[103577]: Deploying daemon mgr.y on smithi098 2023-09-23T09:22:55.645 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:55.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:55.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T09:22:55.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:55.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:55.647 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:55 smithi098 ceph-mon[102386]: Deploying daemon mgr.y on smithi098 2023-09-23T09:22:55.670 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:55.670 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.670 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.670 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:55.670 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:55.670 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T09:22:55.670 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:55.671 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:55.671 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:55 smithi045 ceph-mon[98627]: Deploying daemon mgr.y on smithi098 2023-09-23T09:22:56.044 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:55 smithi098 systemd[1]: Starting Ceph mgr.y for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:22:56.338 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:55.563+0000 7fae9c693200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:22:56.338 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:55.980+0000 7fae9c693200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:22:56.448 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 podman[103820]: 2023-09-23T09:22:56.448 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 bash[103820]: 642e98be66eb238bec9d49d501de7613e20bbce49f16c254a721e739c9bd6c0f 2023-09-23T09:22:56.448 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 ceph-mgr[103860]: -- 172.21.15.98:0/459035576 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0036b91e0 con 0x55a00368e800 2023-09-23T09:22:56.764 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 systemd[1]: Started Ceph mgr.y for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:22:56.764 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:56.489+0000 7faee7f05200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:22:56.764 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:56.572+0000 7faee7f05200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:22:56.764 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:56.664+0000 7faee7f05200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:22:56.764 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:56.761+0000 7faee7f05200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:22:56.933 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-23T09:22:56.934 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-23T09:22:56.934 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-09-23T09:22:56.934 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-09-23T09:22:56.934 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137558 s, 762 MB/s 2023-09-23T09:22:56.934 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-09-23T09:22:57.043 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:56 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:56.941+0000 7faee7f05200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:22:57.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:56.642+0000 7fae9c693200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:22:57.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:56.726+0000 7fae9c693200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:22:57.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2023-09-23T09:22:57.652 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:57.319+0000 7faee7f05200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:22:57.652 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:57.456+0000 7faee7f05200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:22:57.652 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:57.552+0000 7faee7f05200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:22:57.652 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:57.649+0000 7faee7f05200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:22:57.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-mon[102386]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:57.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:57.824 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:57.543+0000 7fae9c693200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:22:57.824 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:57.634+0000 7fae9c693200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:22:57.824 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-mon[103577]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:57.825 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.825 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.825 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.825 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.825 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:57 smithi045 ceph-mon[98627]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:22:58.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:57.821+0000 7fae9c693200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:22:58.510 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:58.127+0000 7fae9c693200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:22:58.510 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:58.357+0000 7fae9c693200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:22:58.543 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:58 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:58.101+0000 7faee7f05200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:22:58.774 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:58.507+0000 7fae9c693200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 -- 172.21.15.45:0/2458898498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d80ffdb0 msgr2=0x7f38d81001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2458898498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d80ffdb0 0x7f38d81001d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38c80099c0 tx=0x7f38c8009cd0 comp rx=0 tx=0).stop 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 -- 172.21.15.45:0/2458898498 shutdown_connections 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2458898498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d8101950 0x7f38d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2458898498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d80ffdb0 0x7f38d81001d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2458898498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d8100fb0 0x7f38d8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 -- 172.21.15.45:0/2458898498 >> 172.21.15.45:0/2458898498 conn(0x7f38d80fb330 msgr2=0x7f38d80fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 -- 172.21.15.45:0/2458898498 shutdown_connections 2023-09-23T09:22:58.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.996+0000 7f38dcf43700 1 -- 172.21.15.45:0/2458898498 wait complete. 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 Processor -- start 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 -- start start 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d80ffdb0 0x7f38d8069d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d8100fb0 0x7f38d819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d8101950 0x7f38d819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d819e6f0 con 0x7f38d8100fb0 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d819e860 con 0x7f38d80ffdb0 2023-09-23T09:22:59.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38dcf43700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d819ea20 con 0x7f38d8101950 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d8100fb0 0x7f38d819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d8100fb0 0x7f38d819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57912/0 (socket says 172.21.15.45:57912) 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d80ffdb0 0x7f38d8069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d5d9b700 1 -- 172.21.15.45:0/2679412684 learned_addr learned my addr 172.21.15.45:0/2679412684 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d6d9d700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d8101950 0x7f38d819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d659c700 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d8101950 msgr2=0x7f38d819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d659c700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d8101950 0x7f38d819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d659c700 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d8100fb0 msgr2=0x7f38d819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d659c700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d8100fb0 0x7f38d819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.997+0000 7f38d659c700 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0009c40 con 0x7f38d80ffdb0 2023-09-23T09:22:59.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.998+0000 7f38d659c700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d80ffdb0 0x7f38d8069d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38c80058e0 tx=0x7f38c800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:59.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.998+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c801d070 con 0x7f38d80ffdb0 2023-09-23T09:22:59.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.998+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38c80054e0 con 0x7f38d80ffdb0 2023-09-23T09:22:59.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.998+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c80095f0 con 0x7f38d80ffdb0 2023-09-23T09:22:59.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.998+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004a40 con 0x7f38d80ffdb0 2023-09-23T09:22:59.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.998+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d819ef20 con 0x7f38d80ffdb0 2023-09-23T09:22:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.999+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 13) v1 ==== 45698+0+0 (secure 0 0 0) 0x7f38c8004ba0 con 0x7f38d80ffdb0 2023-09-23T09:22:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.999+0000 7f38c77fe700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38c0038b50 0x7f38c003b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:22:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.999+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f38c804c150 con 0x7f38d80ffdb0 2023-09-23T09:22:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.999+0000 7f38d5d9b700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38c0038b50 0x7f38c003b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:22:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:58.999+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d81073a0 con 0x7f38d80ffdb0 2023-09-23T09:22:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:59.000+0000 7f38d5d9b700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38c0038b50 0x7f38c003b010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009d20 tx=0x7f38cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:22:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:59.005+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f38c8004e50 con 0x7f38d80ffdb0 2023-09-23T09:22:59.043 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:58 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:58.770+0000 7faee7f05200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:22:59.043 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:58 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:58.850+0000 7faee7f05200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:22:59.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:58.771+0000 7fae9c693200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:22:59.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:22:59.218+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f38d802d040 con 0x7f38c0038b50 2023-09-23T09:22:59.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: Reconfiguring mgr.x (unknown last config time)... 2023-09-23T09:22:59.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:59.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:59.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: Reconfiguring daemon mgr.x on smithi045 2023-09-23T09:22:59.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:22:59.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:22:59.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:22:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: Reconfiguring mgr.x (unknown last config time)... 2023-09-23T09:22:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:59.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:59.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: Reconfiguring daemon mgr.x on smithi045 2023-09-23T09:22:59.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:22:59.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:22:59.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.588 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:59.473+0000 7fae9c693200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:22:59.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: Reconfiguring mgr.x (unknown last config time)... 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T09:22:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: Reconfiguring daemon mgr.x on smithi045 2023-09-23T09:22:59.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:22:59.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:22:59.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:22:59.887 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:59.618+0000 7faee7f05200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:22:59.887 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:59.712+0000 7faee7f05200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:22:59.917 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:59.607+0000 7fae9c693200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:22:59.917 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:59.802+0000 7fae9c693200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:23:00.165 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:22:59 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:22:59.884+0000 7faee7f05200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:23:00.321 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:22:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:22:59.915+0000 7fae9c693200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:23:00.322 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:23:00.133+0000 7fae9c693200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:23:00.534 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:00.163+0000 7faee7f05200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:00.386+0000 7faee7f05200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:00.532+0000 7faee7f05200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:00.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:00.536 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:00.536 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:00.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:01.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:00 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:00.793+0000 7faee7f05200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:23:01.323 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:23:00 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:23:00.997+0000 7fae9c693200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:23:01.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:01.330+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 14) v1 ==== 90917+0+0 (secure 0 0 0) 0x7f38c8005200 con 0x7f38d80ffdb0 2023-09-23T09:23:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:01 smithi045 ceph-mon[98627]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:01 smithi045 ceph-mon[98627]: Standby manager daemon z started 2023-09-23T09:23:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:01 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T09:23:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:01 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:23:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:01 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T09:23:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:01 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:23:01.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:01 smithi123 ceph-mon[103577]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:01 smithi123 ceph-mon[103577]: Standby manager daemon z started 2023-09-23T09:23:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:01 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T09:23:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:01 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:23:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:01 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T09:23:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:01 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:23:01.623 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:01.490+0000 7faee7f05200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:23:01.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-mon[102386]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:01.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-mon[102386]: Standby manager daemon z started 2023-09-23T09:23:01.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T09:23:01.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:23:01.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T09:23:01.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/653630469' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:23:01.926 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:01.620+0000 7faee7f05200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:23:01.926 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:01.811+0000 7faee7f05200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:23:01.926 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:01 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:01.924+0000 7faee7f05200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:23:02.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:02.139+0000 7faee7f05200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:23:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:02 smithi123 ceph-mon[103577]: mgrmap e14: x(active, since 67s), standbys: z 2023-09-23T09:23:02.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T09:23:02.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:02 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2214483330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8136dd58-335a-4231-97c9-99ae3b5e6f8f"}]: dispatch 2023-09-23T09:23:02.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:02 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2214483330' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8136dd58-335a-4231-97c9-99ae3b5e6f8f"}]': finished 2023-09-23T09:23:02.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:02 smithi123 ceph-mon[103577]: osdmap e5: 1 total, 0 up, 1 in 2023-09-23T09:23:02.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:02.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:02 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3952522159' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:02.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-mon[102386]: mgrmap e14: x(active, since 67s), standbys: z 2023-09-23T09:23:02.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T09:23:02.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2214483330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8136dd58-335a-4231-97c9-99ae3b5e6f8f"}]: dispatch 2023-09-23T09:23:02.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2214483330' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8136dd58-335a-4231-97c9-99ae3b5e6f8f"}]': finished 2023-09-23T09:23:02.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-mon[102386]: osdmap e5: 1 total, 0 up, 1 in 2023-09-23T09:23:02.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:02.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3952522159' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:02.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:02 smithi045 ceph-mon[98627]: mgrmap e14: x(active, since 67s), standbys: z 2023-09-23T09:23:02.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T09:23:02.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:02 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2214483330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8136dd58-335a-4231-97c9-99ae3b5e6f8f"}]: dispatch 2023-09-23T09:23:02.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:02 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2214483330' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8136dd58-335a-4231-97c9-99ae3b5e6f8f"}]': finished 2023-09-23T09:23:02.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:02 smithi045 ceph-mon[98627]: osdmap e5: 1 total, 0 up, 1 in 2023-09-23T09:23:02.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:02.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:02 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3952522159' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:03.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:23:02 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:23:02.992+0000 7faee7f05200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:23:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:03.573+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mon.2 v2:172.21.15.98:3300/0 8 ==== mgrmap(e 15) v1 ==== 136136+0+0 (secure 0 0 0) 0x7f38c802aca0 con 0x7f38d80ffdb0 2023-09-23T09:23:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:03 smithi045 ceph-mon[98627]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:03 smithi045 ceph-mon[98627]: Standby manager daemon y started 2023-09-23T09:23:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:03 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T09:23:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:03 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:23:03.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:03 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T09:23:03.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:03 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:23:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:03 smithi123 ceph-mon[103577]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:03 smithi123 ceph-mon[103577]: Standby manager daemon y started 2023-09-23T09:23:03.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:03 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T09:23:03.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:03 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:23:03.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:03 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T09:23:03.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:03 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:23:04.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:03 smithi098 ceph-mon[102386]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:04.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:03 smithi098 ceph-mon[102386]: Standby manager daemon y started 2023-09-23T09:23:04.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:03 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T09:23:04.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:03 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:23:04.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:03 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T09:23:04.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:03 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1078068494' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:23:04.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:04 smithi045 ceph-mon[98627]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-09-23T09:23:04.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T09:23:04.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:04 smithi123 ceph-mon[103577]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-09-23T09:23:04.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T09:23:05.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:04 smithi098 ceph-mon[102386]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-09-23T09:23:05.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T09:23:06.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:06 smithi123 ceph-mon[103577]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:06.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:06 smithi045 ceph-mon[98627]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:06.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:06 smithi098 ceph-mon[102386]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:08.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:08 smithi123 ceph-mon[103577]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:08.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-23T09:23:08.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:08.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:08 smithi045 ceph-mon[98627]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:08.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-23T09:23:08.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:08.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:08 smithi098 ceph-mon[102386]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:08.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-23T09:23:08.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:09.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:09 smithi123 ceph-mon[103577]: Deploying daemon osd.0 on smithi045 2023-09-23T09:23:09.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:09 smithi045 ceph-mon[98627]: Deploying daemon osd.0 on smithi045 2023-09-23T09:23:09.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:09 smithi098 ceph-mon[102386]: Deploying daemon osd.0 on smithi045 2023-09-23T09:23:10.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:10 smithi045 ceph-mon[98627]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:10.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:10 smithi098 ceph-mon[102386]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:10.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:10 smithi123 ceph-mon[103577]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:12.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:12 smithi045 ceph-mon[98627]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:12.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:12 smithi098 ceph-mon[102386]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:12.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:12 smithi123 ceph-mon[103577]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:13.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:13 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:13.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:13 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:13.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:13 smithi045 ceph-mon[98627]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:13.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:13 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:13.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:13 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:13.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:13 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:13.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:13 smithi098 ceph-mon[102386]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:13.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:13 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:13.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:13 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:13.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:13 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:13.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:13 smithi123 ceph-mon[103577]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:13.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:13 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.669+0000 7f38c77fe700 1 -- 172.21.15.45:0/2679412684 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f38d802d040 con 0x7f38c0038b50 2023-09-23T09:23:14.671 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38c0038b50 msgr2=0x7f38c003b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38c0038b50 0x7f38c003b010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009d20 tx=0x7f38cc009260 comp rx=0 tx=0).stop 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d80ffdb0 msgr2=0x7f38d8069d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d80ffdb0 0x7f38d8069d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38c80058e0 tx=0x7f38c800f690 comp rx=0 tx=0).stop 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 shutdown_connections 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38c0038b50 0x7f38c003b010 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d8101950 0x7f38d819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d80ffdb0 0x7f38d8069d30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 --2- 172.21.15.45:0/2679412684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d8100fb0 0x7f38d819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 >> 172.21.15.45:0/2679412684 conn(0x7f38d80fb330 msgr2=0x7f38d81044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:23:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 shutdown_connections 2023-09-23T09:23:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:23:14.671+0000 7f38dcf43700 1 -- 172.21.15.45:0/2679412684 wait complete. 2023-09-23T09:23:15.024 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.0.service 2023-09-23T09:23:15.026 INFO:tasks.cephadm:Deploying osd.1 on smithi098 with /dev/vg_nvme/lv_4... 2023-09-23T09:23:15.026 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-23T09:23:15.058 INFO:journalctl@ceph.osd.0.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:15.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:15.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:15.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:15.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:15.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:15.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:23:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:16 smithi045 ceph-mon[98627]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T09:23:16.331 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:23:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-0[118000]: 2023-09-23T09:23:16.201+0000 7fdd891d9640 -1 osd.0 0 log_to_monitors true 2023-09-23T09:23:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:16 smithi098 ceph-mon[102386]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T09:23:16.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:16 smithi123 ceph-mon[103577]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T09:23:17.108 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-23T09:23:17.108 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-23T09:23:17.108 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-23T09:23:17.109 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-23T09:23:17.109 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110014 s, 953 MB/s 2023-09-23T09:23:17.109 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 2023-09-23T09:23:17.109 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-23T09:23:17.407 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:23:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-0[118000]: 2023-09-23T09:23:17.271+0000 7fdd7e851700 -1 osd.0 0 waiting for initial osdmap 2023-09-23T09:23:17.407 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:23:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-0[118000]: 2023-09-23T09:23:17.274+0000 7fdd7663e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:23:17.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:17 smithi045 ceph-mon[98627]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:17.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:17 smithi045 ceph-mon[98627]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-23T09:23:17.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:17 smithi045 ceph-mon[98627]: osdmap e6: 1 total, 0 up, 1 in 2023-09-23T09:23:17.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:17.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:17 smithi045 ceph-mon[98627]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:23:17.482 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:17 smithi098 ceph-mon[102386]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:17.483 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:17 smithi098 ceph-mon[102386]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-23T09:23:17.483 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:17 smithi098 ceph-mon[102386]: osdmap e6: 1 total, 0 up, 1 in 2023-09-23T09:23:17.483 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:17.483 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:17 smithi098 ceph-mon[102386]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:23:17.530 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2023-09-23T09:23:17.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:17 smithi123 ceph-mon[103577]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:17.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:17 smithi123 ceph-mon[103577]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-23T09:23:17.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:17 smithi123 ceph-mon[103577]: osdmap e6: 1 total, 0 up, 1 in 2023-09-23T09:23:17.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:17.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:17 smithi123 ceph-mon[103577]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:23:18.086 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: osdmap e7: 1 total, 0 up, 1 in 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi045 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:23:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi045 to 17173M 2023-09-23T09:23:18.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:18.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:18.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:23:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: osdmap e7: 1 total, 0 up, 1 in 2023-09-23T09:23:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi045 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi045 to 17173M 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:23:18.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: osdmap e7: 1 total, 0 up, 1 in 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi045 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi045 to 17173M 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:18.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:18.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:19.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- 172.21.15.98:0/81849812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102210 msgr2=0x7f11b4102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:19.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 --2- 172.21.15.98:0/81849812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102210 0x7f11b4102670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009990 tx=0x7f11a0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:23:19.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- 172.21.15.98:0/81849812 shutdown_connections 2023-09-23T09:23:19.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 --2- 172.21.15.98:0/81849812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11b4102bb0 0x7f11b4109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:19.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 --2- 172.21.15.98:0/81849812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b40ff190 0x7f11b40ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:19.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 --2- 172.21.15.98:0/81849812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102210 0x7f11b4102670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:19.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- 172.21.15.98:0/81849812 >> 172.21.15.98:0/81849812 conn(0x7f11b40faad0 msgr2=0x7f11b40fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:23:19.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- 172.21.15.98:0/81849812 shutdown_connections 2023-09-23T09:23:19.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- 172.21.15.98:0/81849812 wait complete. 2023-09-23T09:23:19.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 Processor -- start 2023-09-23T09:23:19.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- start start 2023-09-23T09:23:19.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11b40ff190 0x7f11b4196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:19.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b4102210 0x7f11b41970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:19.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102bb0 0x7f11b419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:19.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b419e700 con 0x7f11b40ff190 2023-09-23T09:23:19.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b419e840 con 0x7f11b4102bb0 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.346+0000 7f11babac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b419e9d0 con 0x7f11b4102210 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102bb0 0x7f11b419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102bb0 0x7f11b419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46504/0 (socket says 172.21.15.98:46504) 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 -- 172.21.15.98:0/2269143443 learned_addr learned my addr 172.21.15.98:0/2269143443 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b4102210 0x7f11b41970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b8948700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11b40ff190 0x7f11b4196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 -- 172.21.15.98:0/2269143443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b4102210 msgr2=0x7f11b41970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b4102210 0x7f11b41970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 -- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11b40ff190 msgr2=0x7f11b4196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11b40ff190 0x7f11b4196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:19.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 -- 172.21.15.98:0/2269143443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00095f0 con 0x7f11b4102bb0 2023-09-23T09:23:19.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b9149700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102bb0 0x7f11b419e130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11ac007ee0 tx=0x7f11ac00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:23:19.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b1ffb700 1 -- 172.21.15.98:0/2269143443 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac017070 con 0x7f11b4102bb0 2023-09-23T09:23:19.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b1ffb700 1 -- 172.21.15.98:0/2269143443 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ac00f040 con 0x7f11b4102bb0 2023-09-23T09:23:19.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b419ecb0 con 0x7f11b4102bb0 2023-09-23T09:23:19.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11b1ffb700 1 -- 172.21.15.98:0/2269143443 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac005610 con 0x7f11b4102bb0 2023-09-23T09:23:19.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.347+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b419f160 con 0x7f11b4102bb0 2023-09-23T09:23:19.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.348+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b410c6c0 con 0x7f11b4102bb0 2023-09-23T09:23:19.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.349+0000 7f11b1ffb700 1 -- 172.21.15.98:0/2269143443 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) v1 ==== 136136+0+0 (secure 0 0 0) 0x7f11ac004c40 con 0x7f11b4102bb0 2023-09-23T09:23:19.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.350+0000 7f11b1ffb700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f11a40a1510 0x7f11a40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:19.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.350+0000 7f11b1ffb700 1 -- 172.21.15.98:0/2269143443 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f11ac0cab60 con 0x7f11b4102bb0 2023-09-23T09:23:19.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.350+0000 7f11b8948700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f11a40a1510 0x7f11a40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:19.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.350+0000 7f11b8948700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f11a40a1510 0x7f11a40a39d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11a8005950 tx=0x7f11a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:23:19.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.354+0000 7f11b1ffb700 1 -- 172.21.15.98:0/2269143443 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f11ac095bc0 con 0x7f11b4102bb0 2023-09-23T09:23:19.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:19 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:23:19.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:19 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:23:19.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:19 smithi098 ceph-mon[102386]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:19.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:19 smithi098 ceph-mon[102386]: osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] boot 2023-09-23T09:23:19.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:19 smithi098 ceph-mon[102386]: osdmap e8: 1 total, 1 up, 1 in 2023-09-23T09:23:19.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:19 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:19.550+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b4061260 con 0x7f11a40a1510 2023-09-23T09:23:19.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:19 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:23:19.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:19 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:23:19.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:19 smithi045 ceph-mon[98627]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:19.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:19 smithi045 ceph-mon[98627]: osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] boot 2023-09-23T09:23:19.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:19 smithi045 ceph-mon[98627]: osdmap e8: 1 total, 1 up, 1 in 2023-09-23T09:23:19.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:19 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:19.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:19 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:23:19.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:19 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:23:19.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:19 smithi123 ceph-mon[103577]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T09:23:19.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:19 smithi123 ceph-mon[103577]: osd.0 [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] boot 2023-09-23T09:23:19.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:19 smithi123 ceph-mon[103577]: osdmap e8: 1 total, 1 up, 1 in 2023-09-23T09:23:19.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:19 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:23:20.678 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:20 smithi098 ceph-mon[102386]: osdmap e9: 1 total, 1 up, 1 in 2023-09-23T09:23:20.678 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:20 smithi098 ceph-mon[102386]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:20.679 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:23:20.679 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:23:20.679 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:20.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:20 smithi045 ceph-mon[98627]: osdmap e9: 1 total, 1 up, 1 in 2023-09-23T09:23:20.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:20 smithi045 ceph-mon[98627]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:20.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:23:20.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:23:20.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:20.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:20 smithi123 ceph-mon[103577]: osdmap e9: 1 total, 1 up, 1 in 2023-09-23T09:23:20.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:20 smithi123 ceph-mon[103577]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:20.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:23:20.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:23:20.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:22.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:22 smithi123 ceph-mon[103577]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:22.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:22 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/3646132360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]: dispatch 2023-09-23T09:23:22.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:22 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]: dispatch 2023-09-23T09:23:22.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:22 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]': finished 2023-09-23T09:23:22.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:22 smithi123 ceph-mon[103577]: osdmap e10: 2 total, 1 up, 2 in 2023-09-23T09:23:22.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:22 smithi098 ceph-mon[102386]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:22 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/3646132360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]: dispatch 2023-09-23T09:23:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:22 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]: dispatch 2023-09-23T09:23:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:22 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]': finished 2023-09-23T09:23:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:22 smithi098 ceph-mon[102386]: osdmap e10: 2 total, 1 up, 2 in 2023-09-23T09:23:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:22 smithi045 ceph-mon[98627]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:22 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/3646132360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]: dispatch 2023-09-23T09:23:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:22 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]: dispatch 2023-09-23T09:23:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:22 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "934fb488-aa5c-4a75-acf1-4f1a42c9a428"}]': finished 2023-09-23T09:23:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:22 smithi045 ceph-mon[98627]: osdmap e10: 2 total, 1 up, 2 in 2023-09-23T09:23:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:23.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:23 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/113020209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:23.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:23 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/113020209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:23.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:23 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/113020209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:24.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:24 smithi098 ceph-mon[102386]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:24 smithi045 ceph-mon[98627]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:24.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:24 smithi123 ceph-mon[103577]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:26.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:26 smithi098 ceph-mon[102386]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:26.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:26 smithi045 ceph-mon[98627]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:26.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:26 smithi123 ceph-mon[103577]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:28.496 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:28 smithi098 ceph-mon[102386]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:28.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:28 smithi045 ceph-mon[98627]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:28.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:28 smithi123 ceph-mon[103577]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:29.469 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-23T09:23:29.470 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-23T09:23:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:29.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-23T09:23:29.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:30.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:30 smithi098 ceph-mon[102386]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:30.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:30 smithi098 ceph-mon[102386]: Deploying daemon osd.1 on smithi098 2023-09-23T09:23:30.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:30 smithi045 ceph-mon[98627]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:30.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:30 smithi045 ceph-mon[98627]: Deploying daemon osd.1 on smithi098 2023-09-23T09:23:30.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:30 smithi123 ceph-mon[103577]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:30.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:30 smithi123 ceph-mon[103577]: Deploying daemon osd.1 on smithi098 2023-09-23T09:23:32.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:32 smithi098 ceph-mon[102386]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:32 smithi045 ceph-mon[98627]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:32.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:32 smithi123 ceph-mon[103577]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:34 smithi098 ceph-mon[102386]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:34 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:34 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:34 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:34.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:34 smithi045 ceph-mon[98627]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:34.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:34 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:34.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:34 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:34.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:34 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:34.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:34 smithi123 ceph-mon[103577]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:34.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:34 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:34.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:34 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:34.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:34 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:35.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.722+0000 7f11b1ffb700 1 -- 172.21.15.98:0/2269143443 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f11b4061260 con 0x7f11a40a1510 2023-09-23T09:23:35.723 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 1 on host 'smithi098' 2023-09-23T09:23:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f11a40a1510 msgr2=0x7f11a40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f11a40a1510 0x7f11a40a39d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11a8005950 tx=0x7f11a800b410 comp rx=0 tx=0).stop 2023-09-23T09:23:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102bb0 msgr2=0x7f11b419e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102bb0 0x7f11b419e130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11ac007ee0 tx=0x7f11ac00d3b0 comp rx=0 tx=0).stop 2023-09-23T09:23:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 shutdown_connections 2023-09-23T09:23:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11b4102bb0 0x7f11b419e130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11b40ff190 0x7f11b4196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:35.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f11a40a1510 0x7f11a40a39d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:35.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 --2- 172.21.15.98:0/2269143443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b4102210 0x7f11b41970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:35.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 >> 172.21.15.98:0/2269143443 conn(0x7f11b40faad0 msgr2=0x7f11b40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:23:35.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 shutdown_connections 2023-09-23T09:23:35.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:23:35.724+0000 7f11babac700 1 -- 172.21.15.98:0/2269143443 wait complete. 2023-09-23T09:23:36.171 DEBUG:teuthology.orchestra.run.smithi098:osd.1> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.1.service 2023-09-23T09:23:36.174 INFO:tasks.cephadm:Deploying osd.2 on smithi123 with /dev/vg_nvme/lv_4... 2023-09-23T09:23:36.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-23T09:23:36.206 INFO:journalctl@ceph.osd.1.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:23:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:36 smithi098 ceph-mon[102386]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:36 smithi045 ceph-mon[98627]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:36.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:36 smithi123 ceph-mon[103577]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:36.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:36.738 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:23:37.225 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:23:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-1[107309]: 2023-09-23T09:23:36.996+0000 7fcb393f0640 -1 osd.1 0 log_to_monitors true 2023-09-23T09:23:37.225 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:37 smithi098 ceph-mon[102386]: from='osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T09:23:37.225 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:37 smithi098 ceph-mon[102386]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T09:23:37.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:37 smithi123 ceph-mon[103577]: from='osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T09:23:37.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:37 smithi123 ceph-mon[103577]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T09:23:37.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:37 smithi045 ceph-mon[98627]: from='osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T09:23:37.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:37 smithi045 ceph-mon[98627]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T09:23:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-23T09:23:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-23T09:23:38.210 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-23T09:23:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-23T09:23:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0144933 s, 723 MB/s 2023-09-23T09:23:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-23T09:23:38.442 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-1[107309]: 2023-09-23T09:23:38.118+0000 7fcb2ea68700 -1 osd.1 0 waiting for initial osdmap 2023-09-23T09:23:38.443 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-1[107309]: 2023-09-23T09:23:38.122+0000 7fcb26855700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:23:38.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-mon[102386]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:38.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-mon[102386]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-23T09:23:38.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-mon[102386]: from='osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:23:38.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-mon[102386]: osdmap e11: 2 total, 1 up, 2 in 2023-09-23T09:23:38.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:38.444 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:38 smithi098 ceph-mon[102386]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:23:38.577 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:38 smithi123 ceph-mon[103577]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:38.577 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:38 smithi123 ceph-mon[103577]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-23T09:23:38.578 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:38 smithi123 ceph-mon[103577]: from='osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:23:38.578 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:38 smithi123 ceph-mon[103577]: osdmap e11: 2 total, 1 up, 2 in 2023-09-23T09:23:38.578 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:38.578 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:38 smithi123 ceph-mon[103577]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:23:38.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2023-09-23T09:23:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:38 smithi045 ceph-mon[98627]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:38 smithi045 ceph-mon[98627]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-23T09:23:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:38 smithi045 ceph-mon[98627]: from='osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:23:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:38 smithi045 ceph-mon[98627]: osdmap e11: 2 total, 1 up, 2 in 2023-09-23T09:23:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:38 smithi045 ceph-mon[98627]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:23:39.135 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:23:39.385 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:23:39.385 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: osdmap e12: 2 total, 1 up, 2 in 2023-09-23T09:23:39.385 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:39.385 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:39.385 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.385 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:39.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: osdmap e12: 2 total, 1 up, 2 in 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:39.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:39.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:39.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:39.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:23:39.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: osdmap e12: 2 total, 1 up, 2 in 2023-09-23T09:23:39.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:39.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:39.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:39.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.320+0000 7fcb336ed700 1 -- 172.21.15.123:0/354835139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 msgr2=0x7fcb2c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.320+0000 7fcb336ed700 1 --2- 172.21.15.123:0/354835139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c108200 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b0a0 tx=0x7fcb2400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.320+0000 7fcb336ed700 1 -- 172.21.15.123:0/354835139 shutdown_connections 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.320+0000 7fcb336ed700 1 --2- 172.21.15.123:0/354835139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c108200 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.320+0000 7fcb336ed700 1 --2- 172.21.15.123:0/354835139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb2c0ffdb0 0x7fcb2c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.320+0000 7fcb336ed700 1 --2- 172.21.15.123:0/354835139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb2c100fb0 0x7fcb2c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.320+0000 7fcb336ed700 1 -- 172.21.15.123:0/354835139 >> 172.21.15.123:0/354835139 conn(0x7fcb2c0fb330 msgr2=0x7fcb2c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 -- 172.21.15.123:0/354835139 shutdown_connections 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 -- 172.21.15.123:0/354835139 wait complete. 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 Processor -- start 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 -- start start 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb2c0ffdb0 0x7fcb2c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb2c100fb0 0x7fcb2c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19e7a0 con 0x7fcb2c100fb0 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19e8e0 con 0x7fcb2c0ffdb0 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb336ed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19eaa0 con 0x7fcb2c101950 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb31c8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb31c8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c19e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48118/0 (socket says 172.21.15.123:48118) 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.321+0000 7fcb31c8a700 1 -- 172.21.15.123:0/3036801074 learned_addr learned my addr 172.21.15.123:0/3036801074 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb30c88700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb2c100fb0 0x7fcb2c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31489700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb2c0ffdb0 0x7fcb2c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31c8a700 1 -- 172.21.15.123:0/3036801074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb2c0ffdb0 msgr2=0x7fcb2c196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:40.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31c8a700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb2c0ffdb0 0x7fcb2c196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31c8a700 1 -- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb2c100fb0 msgr2=0x7fcb2c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31c8a700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb2c100fb0 0x7fcb2c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31c8a700 1 -- 172.21.15.123:0/3036801074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24009d00 con 0x7fcb2c101950 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31489700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb2c0ffdb0 0x7fcb2c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb31c8a700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c19e1d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb240062a0 tx=0x7fcb24004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb227fc700 1 -- 172.21.15.123:0/3036801074 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400e040 con 0x7fcb2c101950 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb227fc700 1 -- 172.21.15.123:0/3036801074 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2401db50 con 0x7fcb2c101950 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb227fc700 1 -- 172.21.15.123:0/3036801074 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2401bcb0 con 0x7fcb2c101950 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c19ed20 con 0x7fcb2c101950 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.322+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c19f140 con 0x7fcb2c101950 2023-09-23T09:23:40.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.323+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c10b5c0 con 0x7fcb2c101950 2023-09-23T09:23:40.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.324+0000 7fcb227fc700 1 -- 172.21.15.123:0/3036801074 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) v1 ==== 136136+0+0 (secure 0 0 0) 0x7fcb2401d690 con 0x7fcb2c101950 2023-09-23T09:23:40.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.324+0000 7fcb227fc700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb180a1590 0x7fcb180a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:23:40.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.325+0000 7fcb227fc700 1 -- 172.21.15.123:0/3036801074 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fcb240cd2e0 con 0x7fcb2c101950 2023-09-23T09:23:40.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.325+0000 7fcb31489700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb180a1590 0x7fcb180a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:23:40.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.325+0000 7fcb31489700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb180a1590 0x7fcb180a3a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c005fd0 tx=0x7fcb1c005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:23:40.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.329+0000 7fcb227fc700 1 -- 172.21.15.123:0/3036801074 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fcb24098110 con 0x7fcb2c101950 2023-09-23T09:23:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:40.524+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb2c069980 con 0x7fcb180a1590 2023-09-23T09:23:40.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:23:40.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:23:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi098 2023-09-23T09:23:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi098 to 17168M 2023-09-23T09:23:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] boot 2023-09-23T09:23:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: osdmap e13: 2 total, 2 up, 2 in 2023-09-23T09:23:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:23:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:23:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi098 2023-09-23T09:23:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi098 to 17168M 2023-09-23T09:23:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] boot 2023-09-23T09:23:40.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: osdmap e13: 2 total, 2 up, 2 in 2023-09-23T09:23:40.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:40.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:23:40.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:23:40.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T09:23:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi098 2023-09-23T09:23:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi098 to 17168M 2023-09-23T09:23:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: osd.1 [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] boot 2023-09-23T09:23:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: osdmap e13: 2 total, 2 up, 2 in 2023-09-23T09:23:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:23:41.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:41 smithi123 ceph-mon[103577]: osdmap e14: 2 total, 2 up, 2 in 2023-09-23T09:23:41.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:23:41.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:23:41.378 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:41 smithi098 ceph-mon[102386]: osdmap e14: 2 total, 2 up, 2 in 2023-09-23T09:23:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:23:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:23:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:41.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:41 smithi045 ceph-mon[98627]: osdmap e14: 2 total, 2 up, 2 in 2023-09-23T09:23:41.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:23:41.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:23:41.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:42.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:42 smithi098 ceph-mon[102386]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:42.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:42 smithi098 ceph-mon[102386]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:42 smithi045 ceph-mon[98627]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:42 smithi045 ceph-mon[98627]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:42.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:42 smithi123 ceph-mon[103577]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:42.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:42 smithi123 ceph-mon[103577]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:23:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:43 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/1720423149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]: dispatch 2023-09-23T09:23:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:43 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]: dispatch 2023-09-23T09:23:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:43 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]': finished 2023-09-23T09:23:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:43 smithi098 ceph-mon[102386]: osdmap e15: 3 total, 2 up, 3 in 2023-09-23T09:23:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:43 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:23:43.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:43 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/1720423149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]: dispatch 2023-09-23T09:23:43.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:43 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]: dispatch 2023-09-23T09:23:43.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:43 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]': finished 2023-09-23T09:23:43.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:43 smithi045 ceph-mon[98627]: osdmap e15: 3 total, 2 up, 3 in 2023-09-23T09:23:43.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:43 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:23:43.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:43 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/1720423149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]: dispatch 2023-09-23T09:23:43.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:43 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]: dispatch 2023-09-23T09:23:43.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:43 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "752d7ddf-4714-4f48-b2ce-5ad5e04e6426"}]': finished 2023-09-23T09:23:43.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:43 smithi123 ceph-mon[103577]: osdmap e15: 3 total, 2 up, 3 in 2023-09-23T09:23:43.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:43 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:23:44.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:44 smithi098 ceph-mon[102386]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:44.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:44 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/482849275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:44 smithi045 ceph-mon[98627]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:44 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/482849275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:44.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:44 smithi123 ceph-mon[103577]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:44.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:44 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/482849275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:23:46.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:46 smithi123 ceph-mon[103577]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:46.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:46 smithi098 ceph-mon[102386]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:46.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:46 smithi045 ceph-mon[98627]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:48 smithi123 ceph-mon[103577]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:48.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:48 smithi098 ceph-mon[102386]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:48 smithi045 ceph-mon[98627]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:50.191 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:50 smithi123 ceph-mon[103577]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:50.191 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:50 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-23T09:23:50.191 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:50 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:50.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:50 smithi098 ceph-mon[102386]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:50.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:50 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-23T09:23:50.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:50 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:50 smithi045 ceph-mon[98627]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:50 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-23T09:23:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:50 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:51.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:51 smithi123 ceph-mon[103577]: Deploying daemon osd.2 on smithi123 2023-09-23T09:23:51.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:51 smithi098 ceph-mon[102386]: Deploying daemon osd.2 on smithi123 2023-09-23T09:23:51.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:51 smithi045 ceph-mon[98627]: Deploying daemon osd.2 on smithi123 2023-09-23T09:23:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:52 smithi098 ceph-mon[102386]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:52.572 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:52 smithi123 ceph-mon[103577]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:52 smithi045 ceph-mon[98627]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:54 smithi123 ceph-mon[103577]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:54.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:54 smithi098 ceph-mon[102386]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:54.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:54.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:54.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:54.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:54 smithi045 ceph-mon[98627]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:54.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:54.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:54.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:56.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.187+0000 7fcb227fc700 1 -- 172.21.15.123:0/3036801074 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb2c069980 con 0x7fcb180a1590 2023-09-23T09:23:56.190 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 2 on host 'smithi123' 2023-09-23T09:23:56.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb180a1590 msgr2=0x7fcb180a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb180a1590 0x7fcb180a3a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c005fd0 tx=0x7fcb1c005de0 comp rx=0 tx=0).stop 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 msgr2=0x7fcb2c19e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c19e1d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb240062a0 tx=0x7fcb24004db0 comp rx=0 tx=0).stop 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 shutdown_connections 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb180a1590 0x7fcb180a3a50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c101950 0x7fcb2c19e1d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb2c0ffdb0 0x7fcb2c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 --2- 172.21.15.123:0/3036801074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb2c100fb0 0x7fcb2c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:23:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 >> 172.21.15.123:0/3036801074 conn(0x7fcb2c0fb330 msgr2=0x7fcb2c069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:23:56.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 shutdown_connections 2023-09-23T09:23:56.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:23:56.189+0000 7fcb336ed700 1 -- 172.21.15.123:0/3036801074 wait complete. 2023-09-23T09:23:56.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:56 smithi098 ceph-mon[102386]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:56.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:56 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:56 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:56 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:56.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:56 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:56.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:56 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:56 smithi045 ceph-mon[98627]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:56.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:56 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:56 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:56 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:56.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:56 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:56.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:56 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.586 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:56 smithi123 ceph-mon[103577]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:56.586 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:56 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.586 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:56 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.586 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:56 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:23:56.586 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:56 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:23:56.586 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:56 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:56.610 DEBUG:teuthology.orchestra.run.smithi123:osd.2> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.2.service 2023-09-23T09:23:56.612 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2023-09-23T09:23:56.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-23T09:23:56.641 INFO:journalctl@ceph.osd.2.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:23:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:23:57.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:57.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:57.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:57.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:57.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:57.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:57.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:57.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:23:57.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:23:57.840 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:23:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-2[109353]: 2023-09-23T09:23:57.509+0000 7f75b5f63640 -1 osd.2 0 log_to_monitors true 2023-09-23T09:23:58.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:58 smithi098 ceph-mon[102386]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:58.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:58 smithi098 ceph-mon[102386]: from='osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T09:23:58.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:58 smithi098 ceph-mon[102386]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T09:23:58.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:58 smithi045 ceph-mon[98627]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:58.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:58 smithi045 ceph-mon[98627]: from='osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T09:23:58.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:58 smithi045 ceph-mon[98627]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T09:23:58.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:58 smithi123 ceph-mon[103577]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:58.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:58 smithi123 ceph-mon[103577]: from='osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T09:23:58.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:58 smithi123 ceph-mon[103577]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T09:23:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-23T09:23:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-23T09:23:58.667 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-09-23T09:23:58.667 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-09-23T09:23:58.667 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0080591 s, 1.3 GB/s 2023-09-23T09:23:58.667 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-09-23T09:23:59.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2023-09-23T09:23:59.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:59 smithi045 ceph-mon[98627]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-23T09:23:59.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:59 smithi045 ceph-mon[98627]: osdmap e16: 3 total, 2 up, 3 in 2023-09-23T09:23:59.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:23:59.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:59 smithi045 ceph-mon[98627]: from='osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:23:59.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:59 smithi045 ceph-mon[98627]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:23:59.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:23:59 smithi045 ceph-mon[98627]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:59 smithi098 ceph-mon[102386]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-23T09:23:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:59 smithi098 ceph-mon[102386]: osdmap e16: 3 total, 2 up, 3 in 2023-09-23T09:23:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:23:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:59 smithi098 ceph-mon[102386]: from='osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:23:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:59 smithi098 ceph-mon[102386]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:23:59.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:23:59 smithi098 ceph-mon[102386]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:59.588 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-2[109353]: 2023-09-23T09:23:59.181+0000 7f75ab5db700 -1 osd.2 0 waiting for initial osdmap 2023-09-23T09:23:59.588 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-2[109353]: 2023-09-23T09:23:59.185+0000 7f75a53cc700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:23:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-mon[103577]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-23T09:23:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-mon[103577]: osdmap e16: 3 total, 2 up, 3 in 2023-09-23T09:23:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:23:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-mon[103577]: from='osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:23:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-mon[103577]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:23:59.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:23:59 smithi123 ceph-mon[103577]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:23:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: osdmap e17: 3 total, 2 up, 3 in 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi123 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:00.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi123 to 17168M 2023-09-23T09:24:00.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:00.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:00.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:24:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: osdmap e17: 3 total, 2 up, 3 in 2023-09-23T09:24:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi123 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi123 to 17168M 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:24:00.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: osdmap e17: 3 total, 2 up, 3 in 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi123 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi123 to 17168M 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 -- 172.21.15.45:0/2295499562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 msgr2=0x7f802c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 --2- 172.21.15.45:0/2295499562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c1001d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80200099c0 tx=0x7f8020009cd0 comp rx=0 tx=0).stop 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 -- 172.21.15.45:0/2295499562 shutdown_connections 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 --2- 172.21.15.45:0/2295499562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f802c101950 0x7f802c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 --2- 172.21.15.45:0/2295499562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c1001d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 --2- 172.21.15.45:0/2295499562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802c100fb0 0x7f802c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 -- 172.21.15.45:0/2295499562 >> 172.21.15.45:0/2295499562 conn(0x7f802c0fb330 msgr2=0x7f802c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 -- 172.21.15.45:0/2295499562 shutdown_connections 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 -- 172.21.15.45:0/2295499562 wait complete. 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.867+0000 7f80313ee700 1 Processor -- start 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f80313ee700 1 -- start start 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f80313ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f80313ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f802c100fb0 0x7f802c195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f80313ee700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802c101950 0x7f802c19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f80313ee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19d630 con 0x7f802c0ffdb0 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f80313ee700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19d770 con 0x7f802c101950 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f80313ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19d900 con 0x7f802c100fb0 2023-09-23T09:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57982/0 (socket says 172.21.15.45:57982) 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 -- 172.21.15.45:0/1882809909 learned_addr learned my addr 172.21.15.45:0/1882809909 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802a7fc700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f802c100fb0 0x7f802c195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802b7fe700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802c101950 0x7f802c19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 -- 172.21.15.45:0/1882809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f802c100fb0 msgr2=0x7f802c195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f802c100fb0 0x7f802c195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 -- 172.21.15.45:0/1882809909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802c101950 msgr2=0x7f802c19d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802c101950 0x7f802c19d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 -- 172.21.15.45:0/1882809909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200095f0 con 0x7f802c0ffdb0 2023-09-23T09:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802b7fe700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802c101950 0x7f802c19d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.868+0000 7f802affd700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c195aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f802000c010 tx=0x7f8020009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.869+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802001d070 con 0x7f802c0ffdb0 2023-09-23T09:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.869+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8020004cd0 con 0x7f802c0ffdb0 2023-09-23T09:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.869+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c19db80 con 0x7f802c0ffdb0 2023-09-23T09:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.869+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020005290 con 0x7f802c0ffdb0 2023-09-23T09:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.869+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c19df90 con 0x7f802c0ffdb0 2023-09-23T09:24:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.870+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c1073d0 con 0x7f802c0ffdb0 2023-09-23T09:24:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.871+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) v1 ==== 136136+0+0 (secure 0 0 0) 0x7f8020003b50 con 0x7f802c0ffdb0 2023-09-23T09:24:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.871+0000 7f8013fff700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f80140a1590 0x7f80140a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.872+0000 7f802a7fc700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f80140a1590 0x7f80140a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.872+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f80200ccbc0 con 0x7f802c0ffdb0 2023-09-23T09:24:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.872+0000 7f802a7fc700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f80140a1590 0x7f80140a3a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f801c005950 tx=0x7f801c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:24:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:00.875+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f80200977d0 con 0x7f802c0ffdb0 2023-09-23T09:24:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:01.087+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f802c02d1d0 con 0x7f80140a1590 2023-09-23T09:24:01.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] boot 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: osdmap e18: 3 total, 3 up, 3 in 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:24:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:01.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:01.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:01.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:01 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:24:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:24:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] boot 2023-09-23T09:24:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: osdmap e18: 3 total, 3 up, 3 in 2023-09-23T09:24:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:24:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:24:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:01 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: osd.2 [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] boot 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: osdmap e18: 3 total, 3 up, 3 in 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:24:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:01.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:01.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:01.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:01 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-23T09:24:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:02 smithi098 ceph-mon[102386]: osdmap e19: 3 total, 3 up, 3 in 2023-09-23T09:24:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:24:02.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-23T09:24:02.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:02 smithi045 ceph-mon[98627]: osdmap e19: 3 total, 3 up, 3 in 2023-09-23T09:24:02.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:24:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-23T09:24:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:02 smithi123 ceph-mon[103577]: osdmap e19: 3 total, 3 up, 3 in 2023-09-23T09:24:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:24:03.081 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:02 smithi045 sudo[123543]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T09:24:03.081 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:02 smithi045 sudo[123543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:03.081 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:02 smithi045 sudo[123543]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:03.081 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:02 smithi045 sudo[123543]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:03.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:03.229+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f8020097ba0 con 0x7f802c0ffdb0 2023-09-23T09:24:03.498 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:03 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-23T09:24:03.498 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:03 smithi098 ceph-mon[102386]: osdmap e20: 3 total, 3 up, 3 in 2023-09-23T09:24:03.499 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:03 smithi098 ceph-mon[102386]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:03 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-23T09:24:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:03 smithi045 ceph-mon[98627]: osdmap e20: 3 total, 3 up, 3 in 2023-09-23T09:24:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:03 smithi045 ceph-mon[98627]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:03.580 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:03 smithi045 sudo[123568]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T09:24:03.580 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:03 smithi045 sudo[123568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:03.581 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:03 smithi045 sudo[123568]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:03.581 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:24:03 smithi045 sudo[123568]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:03 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-23T09:24:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:03 smithi123 ceph-mon[103577]: osdmap e20: 3 total, 3 up, 3 in 2023-09-23T09:24:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:03 smithi123 ceph-mon[103577]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:03.751 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112108]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T09:24:03.751 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112108]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:03.751 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112108]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:03.751 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112108]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:04.044 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112111]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T09:24:04.044 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:04.044 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112111]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:04.044 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:24:03 smithi098 sudo[112111]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: osdmap e21: 3 total, 3 up, 3 in 2023-09-23T09:24:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-23T09:24:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1946883258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20e42b6b-8c74-4414-a64d-e76bb61de1aa"}]: dispatch 2023-09-23T09:24:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1946883258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20e42b6b-8c74-4414-a64d-e76bb61de1aa"}]': finished 2023-09-23T09:24:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: osdmap e22: 4 total, 3 up, 4 in 2023-09-23T09:24:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:04 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1674416248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:04.276 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114164]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T09:24:04.276 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:04.277 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114164]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:04.277 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114164]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: osdmap e21: 3 total, 3 up, 3 in 2023-09-23T09:24:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-23T09:24:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1946883258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20e42b6b-8c74-4414-a64d-e76bb61de1aa"}]: dispatch 2023-09-23T09:24:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1946883258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20e42b6b-8c74-4414-a64d-e76bb61de1aa"}]': finished 2023-09-23T09:24:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: osdmap e22: 4 total, 3 up, 4 in 2023-09-23T09:24:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:04.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:04.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:04 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1674416248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: osdmap e21: 3 total, 3 up, 3 in 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1946883258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20e42b6b-8c74-4414-a64d-e76bb61de1aa"}]: dispatch 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1946883258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20e42b6b-8c74-4414-a64d-e76bb61de1aa"}]': finished 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: osdmap e22: 4 total, 3 up, 4 in 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:04.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1674416248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:04.588 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114167]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T09:24:04.588 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:04.588 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114167]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:04.589 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:24:04 smithi123 sudo[114167]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:05.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 sudo[123678]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T09:24:05.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 sudo[123678]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:05.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 sudo[123678]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:05.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:04 smithi045 sudo[123678]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:05.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:05 smithi045 sudo[124049]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T09:24:05.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:05 smithi045 sudo[124049]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:05.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:05 smithi045 sudo[124049]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:05.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:05 smithi045 sudo[124049]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:05.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114170]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T09:24:05.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:05.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114170]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:05.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114170]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:06.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114173]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T09:24:06.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114173]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:06.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114173]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:06.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:05 smithi123 sudo[114173]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:06.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:05 smithi098 sudo[112114]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T09:24:06.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:05 smithi098 sudo[112114]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:06.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:05 smithi098 sudo[112114]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:06.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:05 smithi098 sudo[112114]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 sudo[112117]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 sudo[112117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 sudo[112117]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 sudo[112117]: pam_unix(sudo:session): session closed for user root 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:06.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:06.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:06.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:07 smithi098 ceph-mon[102386]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:07 smithi045 ceph-mon[98627]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T09:24:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:24:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:24:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:24:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T09:24:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:07 smithi123 ceph-mon[103577]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:09.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:09 smithi045 ceph-mon[98627]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:09.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:09 smithi123 ceph-mon[103577]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:09 smithi098 ceph-mon[102386]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:11.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:11 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-23T09:24:11.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:11 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:11.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:11 smithi098 ceph-mon[102386]: Deploying daemon osd.3 on smithi045 2023-09-23T09:24:11.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:11 smithi098 ceph-mon[102386]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-09-23T09:24:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:11 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-23T09:24:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:11 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:11 smithi045 ceph-mon[98627]: Deploying daemon osd.3 on smithi045 2023-09-23T09:24:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:11 smithi045 ceph-mon[98627]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-09-23T09:24:11.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:11 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-23T09:24:11.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:11 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:11.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:11 smithi123 ceph-mon[103577]: Deploying daemon osd.3 on smithi045 2023-09-23T09:24:11.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:11 smithi123 ceph-mon[103577]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-09-23T09:24:13.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:13 smithi045 ceph-mon[98627]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-23T09:24:13.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:13 smithi123 ceph-mon[103577]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-23T09:24:13.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:13 smithi098 ceph-mon[102386]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-23T09:24:15.487 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:15 smithi045 ceph-mon[98627]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-23T09:24:15.487 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:15.488 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:15.488 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:15 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:15.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:15 smithi098 ceph-mon[102386]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-23T09:24:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:15 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:15 smithi123 ceph-mon[103577]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-23T09:24:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:15 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:16.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.935+0000 7f8013fff700 1 -- 172.21.15.45:0/1882809909 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f802c02d1d0 con 0x7f80140a1590 2023-09-23T09:24:16.938 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2023-09-23T09:24:16.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f80140a1590 msgr2=0x7f80140a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:16.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f80140a1590 0x7f80140a3a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f801c005950 tx=0x7f801c00a300 comp rx=0 tx=0).stop 2023-09-23T09:24:16.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 msgr2=0x7f802c195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:16.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c195aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f802000c010 tx=0x7f8020009580 comp rx=0 tx=0).stop 2023-09-23T09:24:16.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 shutdown_connections 2023-09-23T09:24:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802c101950 0x7f802c19d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c0ffdb0 0x7f802c195aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f80140a1590 0x7f80140a3a50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 --2- 172.21.15.45:0/1882809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f802c100fb0 0x7f802c195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.937+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 >> 172.21.15.45:0/1882809909 conn(0x7f802c0fb330 msgr2=0x7f802c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:24:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.938+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 shutdown_connections 2023-09-23T09:24:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:24:16.938+0000 7f80313ee700 1 -- 172.21.15.45:0/1882809909 wait complete. 2023-09-23T09:24:17.334 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.3.service 2023-09-23T09:24:17.337 INFO:tasks.cephadm:Deploying osd.4 on smithi098 with /dev/vg_nvme/lv_3... 2023-09-23T09:24:17.337 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-23T09:24:17.366 INFO:journalctl@ceph.osd.3.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:24:17.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:17 smithi045 ceph-mon[98627]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-23T09:24:17.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:17 smithi098 ceph-mon[102386]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-23T09:24:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:17 smithi123 ceph-mon[103577]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-23T09:24:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:17.904 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:24:18.535 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:24:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-3[125835]: 2023-09-23T09:24:18.335+0000 7ff1804c7640 -1 osd.3 0 log_to_monitors true 2023-09-23T09:24:18.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:18 smithi098 ceph-mon[102386]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-23T09:24:18.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:18 smithi045 ceph-mon[98627]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-23T09:24:18.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:18 smithi123 ceph-mon[103577]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-23T09:24:19.408 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-23T09:24:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-23T09:24:19.409 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-23T09:24:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-23T09:24:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129008 s, 813 MB/s 2023-09-23T09:24:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-23T09:24:19.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:19 smithi098 ceph-mon[102386]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T09:24:19.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:19 smithi098 ceph-mon[102386]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-23T09:24:19.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:19 smithi098 ceph-mon[102386]: osdmap e23: 4 total, 3 up, 4 in 2023-09-23T09:24:19.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:19 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:19.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:19 smithi098 ceph-mon[102386]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:24:19.830 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:24:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-3[125835]: 2023-09-23T09:24:19.542+0000 7ff175b3f700 -1 osd.3 0 waiting for initial osdmap 2023-09-23T09:24:19.830 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:24:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-3[125835]: 2023-09-23T09:24:19.547+0000 7ff16f12f700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:24:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:19 smithi045 ceph-mon[98627]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T09:24:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:19 smithi045 ceph-mon[98627]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-23T09:24:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:19 smithi045 ceph-mon[98627]: osdmap e23: 4 total, 3 up, 4 in 2023-09-23T09:24:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:19 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:19 smithi045 ceph-mon[98627]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:24:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:19 smithi123 ceph-mon[103577]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T09:24:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:19 smithi123 ceph-mon[103577]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-23T09:24:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:19 smithi123 ceph-mon[103577]: osdmap e23: 4 total, 3 up, 4 in 2023-09-23T09:24:19.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:19 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:19.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:19 smithi123 ceph-mon[103577]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:24:19.849 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2023-09-23T09:24:20.408 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:24:20.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: osdmap e24: 4 total, 3 up, 4 in 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi045 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:20.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi045 to 8586M 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] boot 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: osdmap e25: 4 total, 4 up, 4 in 2023-09-23T09:24:20.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:24:20.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: osdmap e24: 4 total, 3 up, 4 in 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi045 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi045 to 8586M 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:20.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:20.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:20.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:20.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] boot 2023-09-23T09:24:20.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: osdmap e25: 4 total, 4 up, 4 in 2023-09-23T09:24:20.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: osdmap e24: 4 total, 3 up, 4 in 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi045 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi045 to 8586M 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: osd.3 [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] boot 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: osdmap e25: 4 total, 4 up, 4 in 2023-09-23T09:24:20.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:24:21.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 -- 172.21.15.98:0/1205631736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34106bb0 msgr2=0x7f9d34109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:21.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 --2- 172.21.15.98:0/1205631736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34106bb0 0x7f9d34109020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00b0a0 tx=0x7f9d2c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 -- 172.21.15.98:0/1205631736 shutdown_connections 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 --2- 172.21.15.98:0/1205631736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34106bb0 0x7f9d34109020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 --2- 172.21.15.98:0/1205631736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d34101950 0x7f9d34103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 --2- 172.21.15.98:0/1205631736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d34104280 0x7f9d34106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 -- 172.21.15.98:0/1205631736 >> 172.21.15.98:0/1205631736 conn(0x7f9d340fb360 msgr2=0x7f9d340fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 -- 172.21.15.98:0/1205631736 shutdown_connections 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.732+0000 7f9d3988c700 1 -- 172.21.15.98:0/1205631736 wait complete. 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 Processor -- start 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 -- start start 2023-09-23T09:24:21.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d34101950 0x7f9d34196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34104280 0x7f9d34197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d34106bb0 0x7f9d3419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3419e760 con 0x7f9d34101950 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3419e8a0 con 0x7f9d34104280 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d3988c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3419ea60 con 0x7f9d34106bb0 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d327fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34104280 0x7f9d34197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d327fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34104280 0x7f9d34197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46580/0 (socket says 172.21.15.98:46580) 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d327fc700 1 -- 172.21.15.98:0/2232523970 learned_addr learned my addr 172.21.15.98:0/2232523970 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d32ffd700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d34101950 0x7f9d34196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d327fc700 1 -- 172.21.15.98:0/2232523970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d34106bb0 msgr2=0x7f9d3419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d327fc700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d34106bb0 0x7f9d3419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:21.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d327fc700 1 -- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d34101950 msgr2=0x7f9d34196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.733+0000 7f9d327fc700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d34101950 0x7f9d34196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d327fc700 1 -- 172.21.15.98:0/2232523970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009d00 con 0x7f9d34104280 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d32ffd700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d34101950 0x7f9d34196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d327fc700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34104280 0x7f9d34197150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400d980 tx=0x7f9d2400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d3888a700 1 -- 172.21.15.98:0/2232523970 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400bb60 con 0x7f9d34104280 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3419ed40 con 0x7f9d34104280 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d3888a700 1 -- 172.21.15.98:0/2232523970 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d24007640 con 0x7f9d34104280 2023-09-23T09:24:21.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d3888a700 1 -- 172.21.15.98:0/2232523970 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24004e90 con 0x7f9d34104280 2023-09-23T09:24:21.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.734+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3419f1e0 con 0x7f9d34104280 2023-09-23T09:24:21.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.735+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d340fcee0 con 0x7f9d34104280 2023-09-23T09:24:21.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.736+0000 7f9d3888a700 1 -- 172.21.15.98:0/2232523970 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f9d2400bcc0 con 0x7f9d34104280 2023-09-23T09:24:21.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.736+0000 7f9d3888a700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9d1c0a15a0 0x7f9d1c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:21.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.737+0000 7f9d3888a700 1 -- 172.21.15.98:0/2232523970 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f9d240cc0d0 con 0x7f9d34104280 2023-09-23T09:24:21.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.737+0000 7f9d32ffd700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9d1c0a15a0 0x7f9d1c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:21.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.738+0000 7f9d32ffd700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9d1c0a15a0 0x7f9d1c0a3a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d341980f0 tx=0x7f9d2800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:24:21.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.740+0000 7f9d3888a700 1 -- 172.21.15.98:0/2232523970 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9d2409a350 con 0x7f9d34104280 2023-09-23T09:24:21.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:21 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:24:21.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:21 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:24:21.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:21 smithi098 ceph-mon[102386]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:21.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:21 smithi098 ceph-mon[102386]: osdmap e26: 4 total, 4 up, 4 in 2023-09-23T09:24:21.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:21 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:24:21.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:21 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:24:21.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:21 smithi045 ceph-mon[98627]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:21.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:21 smithi045 ceph-mon[98627]: osdmap e26: 4 total, 4 up, 4 in 2023-09-23T09:24:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:21 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:24:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:21 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:24:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:21 smithi123 ceph-mon[103577]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T09:24:21.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:21 smithi123 ceph-mon[103577]: osdmap e26: 4 total, 4 up, 4 in 2023-09-23T09:24:21.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:21.936+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d3402d350 con 0x7f9d1c0a15a0 2023-09-23T09:24:22.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:22 smithi098 ceph-mon[102386]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:22.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:22.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:22.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:22.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:22 smithi098 ceph-mon[102386]: osdmap e27: 4 total, 4 up, 4 in 2023-09-23T09:24:22.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:22 smithi045 ceph-mon[98627]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:22.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:22.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:22.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:22.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:22 smithi045 ceph-mon[98627]: osdmap e27: 4 total, 4 up, 4 in 2023-09-23T09:24:22.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:22 smithi123 ceph-mon[103577]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:22.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:22.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:22.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:22.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:22 smithi123 ceph-mon[103577]: osdmap e27: 4 total, 4 up, 4 in 2023-09-23T09:24:23.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:23 smithi045 ceph-mon[98627]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:23.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:23 smithi045 ceph-mon[98627]: osdmap e28: 4 total, 4 up, 4 in 2023-09-23T09:24:23.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:23 smithi123 ceph-mon[103577]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:23.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:23 smithi123 ceph-mon[103577]: osdmap e28: 4 total, 4 up, 4 in 2023-09-23T09:24:24.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:23 smithi098 ceph-mon[102386]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:24.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:23 smithi098 ceph-mon[102386]: osdmap e28: 4 total, 4 up, 4 in 2023-09-23T09:24:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: osdmap e29: 4 total, 4 up, 4 in 2023-09-23T09:24:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/3020999696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]: dispatch 2023-09-23T09:24:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]: dispatch 2023-09-23T09:24:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]': finished 2023-09-23T09:24:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: osdmap e30: 5 total, 4 up, 5 in 2023-09-23T09:24:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:25 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/1317942929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: osdmap e29: 4 total, 4 up, 4 in 2023-09-23T09:24:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/3020999696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]: dispatch 2023-09-23T09:24:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]: dispatch 2023-09-23T09:24:25.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]': finished 2023-09-23T09:24:25.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: osdmap e30: 5 total, 4 up, 5 in 2023-09-23T09:24:25.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:25.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:25 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/1317942929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: osdmap e29: 4 total, 4 up, 4 in 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/3020999696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]: dispatch 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]: dispatch 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecfe4f49-dd41-46e6-b948-c396c14fe25a"}]': finished 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: osdmap e30: 5 total, 4 up, 5 in 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:25 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/1317942929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:27.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:27 smithi123 ceph-mon[103577]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2023-09-23T09:24:27.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:27 smithi098 ceph-mon[102386]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2023-09-23T09:24:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:27 smithi045 ceph-mon[98627]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2023-09-23T09:24:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:29 smithi123 ceph-mon[103577]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-23T09:24:29.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:29 smithi098 ceph-mon[102386]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-23T09:24:29.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:29 smithi045 ceph-mon[98627]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-23T09:24:31.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:31 smithi098 ceph-mon[102386]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-09-23T09:24:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-23T09:24:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:31 smithi098 ceph-mon[102386]: Deploying daemon osd.4 on smithi098 2023-09-23T09:24:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:31 smithi045 ceph-mon[98627]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-09-23T09:24:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-23T09:24:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:31 smithi045 ceph-mon[98627]: Deploying daemon osd.4 on smithi098 2023-09-23T09:24:31.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:31 smithi123 ceph-mon[103577]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-09-23T09:24:31.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-23T09:24:31.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:31.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:31 smithi123 ceph-mon[103577]: Deploying daemon osd.4 on smithi098 2023-09-23T09:24:33.512 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:33 smithi098 ceph-mon[102386]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-09-23T09:24:33.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:33 smithi123 ceph-mon[103577]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-09-23T09:24:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:33 smithi045 ceph-mon[98627]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-09-23T09:24:35.467 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:35 smithi098 ceph-mon[102386]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T09:24:35.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:35 smithi123 ceph-mon[103577]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T09:24:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:35 smithi045 ceph-mon[98627]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T09:24:36.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:36.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:36.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:36.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:36.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:36.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:37.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:37 smithi098 ceph-mon[102386]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:24:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:37 smithi045 ceph-mon[98627]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:24:37.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:37 smithi123 ceph-mon[103577]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:24:38.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.406+0000 7f9d3888a700 1 -- 172.21.15.98:0/2232523970 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9d3402d350 con 0x7f9d1c0a15a0 2023-09-23T09:24:38.408 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 4 on host 'smithi098' 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9d1c0a15a0 msgr2=0x7f9d1c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9d1c0a15a0 0x7f9d1c0a3a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d341980f0 tx=0x7f9d2800a680 comp rx=0 tx=0).stop 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34104280 msgr2=0x7f9d34197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34104280 0x7f9d34197150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400d980 tx=0x7f9d2400dd40 comp rx=0 tx=0).stop 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 shutdown_connections 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9d1c0a15a0 0x7f9d1c0a3a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d34106bb0 0x7f9d3419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d34101950 0x7f9d34196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:38.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 --2- 172.21.15.98:0/2232523970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d34104280 0x7f9d34197150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:38.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 >> 172.21.15.98:0/2232523970 conn(0x7f9d340fb360 msgr2=0x7f9d341025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:24:38.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.409+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 shutdown_connections 2023-09-23T09:24:38.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:24:38.410+0000 7f9d3988c700 1 -- 172.21.15.98:0/2232523970 wait complete. 2023-09-23T09:24:38.900 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:38.900 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:38.900 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:38.900 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:38.903 DEBUG:teuthology.orchestra.run.smithi098:osd.4> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.4.service 2023-09-23T09:24:38.905 INFO:tasks.cephadm:Deploying osd.5 on smithi123 with /dev/vg_nvme/lv_3... 2023-09-23T09:24:38.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-23T09:24:38.934 INFO:journalctl@ceph.osd.4.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:24:39.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:24:39.742 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:24:39 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4[115193]: 2023-09-23T09:24:39.420+0000 7f2b9b037640 -1 osd.4 0 log_to_monitors true 2023-09-23T09:24:39.982 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:39 smithi123 ceph-mon[103577]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:39.982 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:39 smithi123 ceph-mon[103577]: from='osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T09:24:39.982 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:39 smithi123 ceph-mon[103577]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T09:24:40.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:39 smithi098 ceph-mon[102386]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:40.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:39 smithi098 ceph-mon[102386]: from='osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T09:24:40.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:39 smithi098 ceph-mon[102386]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T09:24:40.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:39 smithi045 ceph-mon[98627]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:40.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:39 smithi045 ceph-mon[98627]: from='osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T09:24:40.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:39 smithi045 ceph-mon[98627]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T09:24:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-23T09:24:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-23T09:24:40.937 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-23T09:24:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-23T09:24:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109985 s, 953 MB/s 2023-09-23T09:24:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-23T09:24:41.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:40 smithi045 ceph-mon[98627]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-23T09:24:41.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:40 smithi045 ceph-mon[98627]: osdmap e31: 5 total, 4 up, 5 in 2023-09-23T09:24:41.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:41.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:40 smithi045 ceph-mon[98627]: from='osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:24:41.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:40 smithi045 ceph-mon[98627]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:24:41.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:40 smithi123 ceph-mon[103577]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-23T09:24:41.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:40 smithi123 ceph-mon[103577]: osdmap e31: 5 total, 4 up, 5 in 2023-09-23T09:24:41.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:41.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:40 smithi123 ceph-mon[103577]: from='osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:24:41.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:40 smithi123 ceph-mon[103577]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:24:41.088 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:24:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4[115193]: 2023-09-23T09:24:40.798+0000 7f2b906af700 -1 osd.4 0 waiting for initial osdmap 2023-09-23T09:24:41.088 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:24:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4[115193]: 2023-09-23T09:24:40.804+0000 7f2b8949e700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:24:41.089 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:40 smithi098 ceph-mon[102386]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-23T09:24:41.089 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:40 smithi098 ceph-mon[102386]: osdmap e31: 5 total, 4 up, 5 in 2023-09-23T09:24:41.089 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:41.089 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:40 smithi098 ceph-mon[102386]: from='osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:24:41.089 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:40 smithi098 ceph-mon[102386]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:24:41.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2023-09-23T09:24:41.882 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:24:42.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:42.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: osdmap e32: 5 total, 4 up, 5 in 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:42.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:42.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:42.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.087 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: osdmap e32: 5 total, 4 up, 5 in 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:42.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:42.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:42.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:42.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:24:42.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: osdmap e32: 5 total, 4 up, 5 in 2023-09-23T09:24:42.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:42.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:42.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:42.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:42.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:42.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:43.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:42 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:24:43.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:42 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:24:43.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:42 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi098 2023-09-23T09:24:43.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:42 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi098 to 8584M 2023-09-23T09:24:43.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:42 smithi045 ceph-mon[98627]: osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] boot 2023-09-23T09:24:43.459 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:42 smithi045 ceph-mon[98627]: osdmap e33: 5 total, 5 up, 5 in 2023-09-23T09:24:43.459 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:43.459 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:42 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:42 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:42 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi098 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:42 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi098 to 8584M 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:42 smithi098 ceph-mon[102386]: osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] boot 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:42 smithi098 ceph-mon[102386]: osdmap e33: 5 total, 5 up, 5 in 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:42 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:42 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:42 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi098 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:42 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi098 to 8584M 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:42 smithi123 ceph-mon[103577]: osd.4 [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] boot 2023-09-23T09:24:43.460 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:42 smithi123 ceph-mon[103577]: osdmap e33: 5 total, 5 up, 5 in 2023-09-23T09:24:43.461 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:24:43.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.087+0000 7f692f467700 1 -- 172.21.15.123:0/1425305145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928102a40 msgr2=0x7f6928109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:43.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.087+0000 7f692f467700 1 --2- 172.21.15.123:0/1425305145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928102a40 0x7f6928109360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f692000b0a0 tx=0x7f692000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:24:43.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.087+0000 7f692f467700 1 -- 172.21.15.123:0/1425305145 shutdown_connections 2023-09-23T09:24:43.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.087+0000 7f692f467700 1 --2- 172.21.15.123:0/1425305145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928102a40 0x7f6928109360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:43.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.087+0000 7f692f467700 1 --2- 172.21.15.123:0/1425305145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6928100e90 0x7f69281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:43.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.087+0000 7f692f467700 1 --2- 172.21.15.123:0/1425305145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6928102080 0x7f6928102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.087+0000 7f692f467700 1 -- 172.21.15.123:0/1425305145 >> 172.21.15.123:0/1425305145 conn(0x7f69280fc4a0 msgr2=0x7f69280fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 -- 172.21.15.123:0/1425305145 shutdown_connections 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 -- 172.21.15.123:0/1425305145 wait complete. 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 Processor -- start 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 -- start start 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928100e90 0x7f6928196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6928102080 0x7f6928197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6928102a40 0x7f692819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819e650 con 0x7f6928102080 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819e790 con 0x7f6928102a40 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692f467700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819e920 con 0x7f6928100e90 2023-09-23T09:24:43.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692d203700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928100e90 0x7f6928196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692d203700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928100e90 0x7f6928196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48192/0 (socket says 172.21.15.123:48192) 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.088+0000 7f692d203700 1 -- 172.21.15.123:0/1723898753 learned_addr learned my addr 172.21.15.123:0/1723898753 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692da04700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6928102a40 0x7f692819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692ca02700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6928102080 0x7f6928197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692d203700 1 -- 172.21.15.123:0/1723898753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6928102a40 msgr2=0x7f692819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692d203700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6928102a40 0x7f692819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692d203700 1 -- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6928102080 msgr2=0x7f6928197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692d203700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6928102080 0x7f6928197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692d203700 1 -- 172.21.15.123:0/1723898753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920009d00 con 0x7f6928100e90 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692ca02700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6928102080 0x7f6928197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692d203700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928100e90 0x7f6928196ac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f691800d9b0 tx=0x7f691800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:24:43.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f691e7fc700 1 -- 172.21.15.123:0/1723898753 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800b950 con 0x7f6928100e90 2023-09-23T09:24:43.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692819ec00 con 0x7f6928100e90 2023-09-23T09:24:43.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f691e7fc700 1 -- 172.21.15.123:0/1723898753 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6918007620 con 0x7f6928100e90 2023-09-23T09:24:43.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f691e7fc700 1 -- 172.21.15.123:0/1723898753 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800f3c0 con 0x7f6928100e90 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.089+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692819f100 con 0x7f6928100e90 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.090+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6928108460 con 0x7f6928100e90 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.091+0000 7f691e7fc700 1 -- 172.21.15.123:0/1723898753 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f6918010640 con 0x7f6928100e90 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.091+0000 7f691e7fc700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69140a15a0 0x7f69140a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.092+0000 7f691e7fc700 1 -- 172.21.15.123:0/1723898753 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f69180cc020 con 0x7f6928100e90 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.092+0000 7f692ca02700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69140a15a0 0x7f69140a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.092+0000 7f692ca02700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69140a15a0 0x7f69140a3a60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6924005e10 tx=0x7f6924005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.096+0000 7f691e7fc700 1 -- 172.21.15.123:0/1723898753 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6918096680 con 0x7f6928100e90 2023-09-23T09:24:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:43.290+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f692802d180 con 0x7f69140a15a0 2023-09-23T09:24:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:43 smithi045 ceph-mon[98627]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:43 smithi045 ceph-mon[98627]: osdmap e34: 5 total, 5 up, 5 in 2023-09-23T09:24:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:43 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:43 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:43 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:44.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:43 smithi123 ceph-mon[103577]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:44.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:43 smithi123 ceph-mon[103577]: osdmap e34: 5 total, 5 up, 5 in 2023-09-23T09:24:44.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:43 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:44.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:43 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:44.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:43 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:44.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:43 smithi098 ceph-mon[102386]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T09:24:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:43 smithi098 ceph-mon[102386]: osdmap e34: 5 total, 5 up, 5 in 2023-09-23T09:24:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:43 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:24:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:43 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:24:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:43 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:44.912 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:44 smithi123 ceph-mon[103577]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:45.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:44 smithi045 ceph-mon[98627]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:45.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:44 smithi098 ceph-mon[102386]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:24:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:45 smithi045 ceph-mon[98627]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:45 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/902982175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]: dispatch 2023-09-23T09:24:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:45 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]: dispatch 2023-09-23T09:24:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:45 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]': finished 2023-09-23T09:24:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:45 smithi045 ceph-mon[98627]: osdmap e35: 6 total, 5 up, 6 in 2023-09-23T09:24:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:24:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:45 smithi123 ceph-mon[103577]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:45 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/902982175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]: dispatch 2023-09-23T09:24:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:45 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]: dispatch 2023-09-23T09:24:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:45 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]': finished 2023-09-23T09:24:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:45 smithi123 ceph-mon[103577]: osdmap e35: 6 total, 5 up, 6 in 2023-09-23T09:24:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:24:46.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:45 smithi098 ceph-mon[102386]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:45 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/902982175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]: dispatch 2023-09-23T09:24:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:45 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]: dispatch 2023-09-23T09:24:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:45 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d"}]': finished 2023-09-23T09:24:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:45 smithi098 ceph-mon[102386]: osdmap e35: 6 total, 5 up, 6 in 2023-09-23T09:24:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:24:47.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:46 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/3854798722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:47.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:46 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/3854798722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:47.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:46 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/3854798722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:24:48.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:47 smithi045 ceph-mon[98627]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:48.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:47 smithi123 ceph-mon[103577]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:48.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:47 smithi098 ceph-mon[102386]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:49 smithi045 ceph-mon[98627]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:49 smithi123 ceph-mon[103577]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:50.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:49 smithi098 ceph-mon[102386]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:51.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:51 smithi098 ceph-mon[102386]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:51.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:51 smithi045 ceph-mon[98627]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:51.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:51 smithi123 ceph-mon[103577]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:52.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-23T09:24:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:52 smithi098 ceph-mon[102386]: Deploying daemon osd.5 on smithi123 2023-09-23T09:24:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-23T09:24:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:52 smithi045 ceph-mon[98627]: Deploying daemon osd.5 on smithi123 2023-09-23T09:24:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-23T09:24:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:52 smithi123 ceph-mon[103577]: Deploying daemon osd.5 on smithi123 2023-09-23T09:24:53.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:53 smithi123 ceph-mon[103577]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:53.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:53 smithi098 ceph-mon[102386]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:53.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:53 smithi045 ceph-mon[98627]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:55.613 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:55 smithi123 ceph-mon[103577]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:55.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:55 smithi098 ceph-mon[102386]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:55 smithi045 ceph-mon[98627]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:57.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:57 smithi123 ceph-mon[103577]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:57.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:57.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:57.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:57 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:57.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:57 smithi098 ceph-mon[102386]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:57.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:57.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:57.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:57 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:57 smithi045 ceph-mon[98627]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:57 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:59.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.051+0000 7f691e7fc700 1 -- 172.21.15.123:0/1723898753 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f692802d180 con 0x7f69140a15a0 2023-09-23T09:24:59.053 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 5 on host 'smithi123' 2023-09-23T09:24:59.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69140a15a0 msgr2=0x7f69140a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:59.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69140a15a0 0x7f69140a3a60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6924005e10 tx=0x7f6924005d20 comp rx=0 tx=0).stop 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928100e90 msgr2=0x7f6928196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928100e90 0x7f6928196ac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f691800d9b0 tx=0x7f691800dcc0 comp rx=0 tx=0).stop 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 shutdown_connections 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6928102a40 0x7f692819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69140a15a0 0x7f69140a3a60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6928100e90 0x7f6928196ac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 --2- 172.21.15.123:0/1723898753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6928102080 0x7f6928197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 >> 172.21.15.123:0/1723898753 conn(0x7f69280fc4a0 msgr2=0x7f69281055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 shutdown_connections 2023-09-23T09:24:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:24:59.054+0000 7f692f467700 1 -- 172.21.15.123:0/1723898753 wait complete. 2023-09-23T09:24:59.472 DEBUG:teuthology.orchestra.run.smithi123:osd.5> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.5.service 2023-09-23T09:24:59.474 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2023-09-23T09:24:59.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-23T09:24:59.506 INFO:journalctl@ceph.osd.5.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:24:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:24:59 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:24:59.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:24:59.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:24:59.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:24:59.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:24:59.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:24:59 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:24:59 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:00.060 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:25:00.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:25:00 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:25:00.380+0000 7fcd19cf2640 -1 osd.5 0 log_to_monitors true 2023-09-23T09:25:01.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:01 smithi045 ceph-mon[98627]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:25:01.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:01 smithi045 ceph-mon[98627]: from='osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T09:25:01.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:01 smithi045 ceph-mon[98627]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T09:25:01.518 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-23T09:25:01.518 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-23T09:25:01.518 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-09-23T09:25:01.519 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-09-23T09:25:01.519 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00833756 s, 1.3 GB/s 2023-09-23T09:25:01.519 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-09-23T09:25:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:01 smithi098 ceph-mon[102386]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:25:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:01 smithi098 ceph-mon[102386]: from='osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T09:25:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:01 smithi098 ceph-mon[102386]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T09:25:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:01 smithi123 ceph-mon[103577]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:25:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:01 smithi123 ceph-mon[103577]: from='osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T09:25:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:01 smithi123 ceph-mon[103577]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T09:25:01.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2023-09-23T09:25:02.229 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-23T09:25:02.229 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: osdmap e36: 6 total, 5 up, 6 in 2023-09-23T09:25:02.229 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:02.229 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi123 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi123 to 8584M 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:02.230 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:25:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: osdmap e36: 6 total, 5 up, 6 in 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi123 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi123 to 8584M 2023-09-23T09:25:02.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:02.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:02.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-23T09:25:02.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: osdmap e36: 6 total, 5 up, 6 in 2023-09-23T09:25:02.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:02.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:25:02.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi123 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi123 to 8584M 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:02.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:02.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:25:02.230+0000 7fcd0f36a700 -1 osd.5 0 waiting for initial osdmap 2023-09-23T09:25:02.589 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:25:02 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:25:02.236+0000 7fcd0b15f700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:25:03.495 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:03 smithi045 ceph-mon[98627]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:25:03.495 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:03 smithi045 ceph-mon[98627]: osdmap e37: 6 total, 5 up, 6 in 2023-09-23T09:25:03.495 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:03 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:03.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:03 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:03.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:03 smithi045 ceph-mon[98627]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:25:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:03 smithi098 ceph-mon[102386]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:25:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:03 smithi098 ceph-mon[102386]: osdmap e37: 6 total, 5 up, 6 in 2023-09-23T09:25:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:03 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:03 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:03 smithi098 ceph-mon[102386]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:25:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:03 smithi123 ceph-mon[103577]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:25:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:03 smithi123 ceph-mon[103577]: osdmap e37: 6 total, 5 up, 6 in 2023-09-23T09:25:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:03 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:03.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:03 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:03.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:03 smithi123 ceph-mon[103577]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T09:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 -- 172.21.15.45:0/3596777447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc102050 msgr2=0x7f79cc1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3596777447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc102050 0x7f79cc1024d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009990 tx=0x7f79b8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 -- 172.21.15.45:0/3596777447 shutdown_connections 2023-09-23T09:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3596777447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79cc102a10 0x7f79cc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3596777447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79cc100e60 0x7f79cc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3596777447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc102050 0x7f79cc1024d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 -- 172.21.15.45:0/3596777447 >> 172.21.15.45:0/3596777447 conn(0x7f79cc0fc410 msgr2=0x7f79cc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 -- 172.21.15.45:0/3596777447 shutdown_connections 2023-09-23T09:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.691+0000 7f79d31f1700 1 -- 172.21.15.45:0/3596777447 wait complete. 2023-09-23T09:25:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 Processor -- start 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 -- start start 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc100e60 0x7f79cc074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79cc102050 0x7f79cc073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79cc102a10 0x7f79cc073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc073ed0 con 0x7f79cc100e60 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc074040 con 0x7f79cc102050 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d31f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc1a0ba0 con 0x7f79cc102a10 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc100e60 0x7f79cc074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc100e60 0x7f79cc074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58106/0 (socket says 172.21.15.45:58106) 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 -- 172.21.15.45:0/3515431121 learned_addr learned my addr 172.21.15.45:0/3515431121 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79cbfff700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79cc102050 0x7f79cc073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 -- 172.21.15.45:0/3515431121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79cc102a10 msgr2=0x7f79cc073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79cc102a10 0x7f79cc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 -- 172.21.15.45:0/3515431121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79cc102050 msgr2=0x7f79cc073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79cc102050 0x7f79cc073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 -- 172.21.15.45:0/3515431121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80095f0 con 0x7f79cc100e60 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.692+0000 7f79d0f8d700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc100e60 0x7f79cc074bc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d980 tx=0x7f79c000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.693+0000 7f79c9ffb700 1 -- 172.21.15.45:0/3515431121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000de60 con 0x7f79cc100e60 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.693+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79cc1a0e80 con 0x7f79cc100e60 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.693+0000 7f79c9ffb700 1 -- 172.21.15.45:0/3515431121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c00044e0 con 0x7f79cc100e60 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.693+0000 7f79c9ffb700 1 -- 172.21.15.45:0/3515431121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0003be0 con 0x7f79cc100e60 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.693+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc1a12a0 con 0x7f79cc100e60 2023-09-23T09:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.694+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc1084d0 con 0x7f79cc100e60 2023-09-23T09:25:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.695+0000 7f79c9ffb700 1 -- 172.21.15.45:0/3515431121 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f79c0009bb0 con 0x7f79cc100e60 2023-09-23T09:25:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.695+0000 7f79c9ffb700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f79bc0a1620 0x7f79bc0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.696+0000 7f79c9ffb700 1 -- 172.21.15.45:0/3515431121 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f79c00cbe00 con 0x7f79cc100e60 2023-09-23T09:25:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.696+0000 7f79cbfff700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f79bc0a1620 0x7f79bc0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.696+0000 7f79cbfff700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f79bc0a1620 0x7f79bc0a3ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79b8006010 tx=0x7f79b800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:25:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.700+0000 7f79c9ffb700 1 -- 172.21.15.45:0/3515431121 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f79c00961f0 con 0x7f79cc100e60 2023-09-23T09:25:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:03.909+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f79cc02d210 con 0x7f79bc0a1620 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] boot 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: osdmap e38: 6 total, 6 up, 6 in 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:04.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:04.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:04 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:25:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:25:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] boot 2023-09-23T09:25:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: osdmap e38: 6 total, 6 up, 6 in 2023-09-23T09:25:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:04.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:04.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:04.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:04.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:04 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:04.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:25:04.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: osd.5 [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] boot 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: osdmap e38: 6 total, 6 up, 6 in 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:04 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:05.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:05 smithi045 ceph-mon[98627]: osdmap e39: 6 total, 6 up, 6 in 2023-09-23T09:25:05.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:05 smithi045 ceph-mon[98627]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:05 smithi098 ceph-mon[102386]: osdmap e39: 6 total, 6 up, 6 in 2023-09-23T09:25:05.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:05 smithi098 ceph-mon[102386]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:05 smithi123 ceph-mon[103577]: osdmap e39: 6 total, 6 up, 6 in 2023-09-23T09:25:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:05 smithi123 ceph-mon[103577]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:06.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:06 smithi098 ceph-mon[102386]: osdmap e40: 6 total, 6 up, 6 in 2023-09-23T09:25:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:06 smithi045 ceph-mon[98627]: osdmap e40: 6 total, 6 up, 6 in 2023-09-23T09:25:06.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:06 smithi123 ceph-mon[103577]: osdmap e40: 6 total, 6 up, 6 in 2023-09-23T09:25:07.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:07 smithi098 ceph-mon[102386]: osdmap e41: 6 total, 6 up, 6 in 2023-09-23T09:25:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:07 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3056659396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5cc090-fa1f-4813-8950-970817a59b7d"}]: dispatch 2023-09-23T09:25:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:07 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3056659396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce5cc090-fa1f-4813-8950-970817a59b7d"}]': finished 2023-09-23T09:25:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:07 smithi098 ceph-mon[102386]: osdmap e42: 7 total, 6 up, 7 in 2023-09-23T09:25:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:07 smithi098 ceph-mon[102386]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:07 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3103130186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:07 smithi045 ceph-mon[98627]: osdmap e41: 6 total, 6 up, 6 in 2023-09-23T09:25:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:07 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3056659396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5cc090-fa1f-4813-8950-970817a59b7d"}]: dispatch 2023-09-23T09:25:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:07 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3056659396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce5cc090-fa1f-4813-8950-970817a59b7d"}]': finished 2023-09-23T09:25:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:07 smithi045 ceph-mon[98627]: osdmap e42: 7 total, 6 up, 7 in 2023-09-23T09:25:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:07 smithi045 ceph-mon[98627]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:07.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:07 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3103130186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:07 smithi123 ceph-mon[103577]: osdmap e41: 6 total, 6 up, 6 in 2023-09-23T09:25:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:07 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3056659396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce5cc090-fa1f-4813-8950-970817a59b7d"}]: dispatch 2023-09-23T09:25:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:07 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3056659396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce5cc090-fa1f-4813-8950-970817a59b7d"}]': finished 2023-09-23T09:25:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:07 smithi123 ceph-mon[103577]: osdmap e42: 7 total, 6 up, 7 in 2023-09-23T09:25:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:07 smithi123 ceph-mon[103577]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:07 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3103130186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:09 smithi098 ceph-mon[102386]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:09.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:09 smithi045 ceph-mon[98627]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:09 smithi123 ceph-mon[103577]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:11.470 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:11 smithi045 ceph-mon[98627]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-23T09:25:11.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:11 smithi098 ceph-mon[102386]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-23T09:25:11.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:11 smithi123 ceph-mon[103577]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-23T09:25:13.604 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:13 smithi045 ceph-mon[98627]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-09-23T09:25:13.604 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:13 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-23T09:25:13.604 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:13 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:13.604 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:13 smithi045 ceph-mon[98627]: Deploying daemon osd.6 on smithi045 2023-09-23T09:25:13.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:13 smithi098 ceph-mon[102386]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-09-23T09:25:13.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:13 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-23T09:25:13.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:13 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:13.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:13 smithi098 ceph-mon[102386]: Deploying daemon osd.6 on smithi045 2023-09-23T09:25:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:13 smithi123 ceph-mon[103577]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-09-23T09:25:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:13 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-23T09:25:13.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:13 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:13.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:13 smithi123 ceph-mon[103577]: Deploying daemon osd.6 on smithi045 2023-09-23T09:25:15.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:15 smithi045 ceph-mon[98627]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:25:15.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:15 smithi098 ceph-mon[102386]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:25:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:15 smithi123 ceph-mon[103577]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:25:18.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:18 smithi045 ceph-mon[98627]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T09:25:18.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:18.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:18.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:18 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:18.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:18 smithi123 ceph-mon[103577]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T09:25:18.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:18.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:18.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:18 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:18 smithi098 ceph-mon[102386]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T09:25:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:18.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:18 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:19.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.861+0000 7f79c9ffb700 1 -- 172.21.15.45:0/3515431121 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f79cc02d210 con 0x7f79bc0a1620 2023-09-23T09:25:19.863 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2023-09-23T09:25:19.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f79bc0a1620 msgr2=0x7f79bc0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:19.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f79bc0a1620 0x7f79bc0a3ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79b8006010 tx=0x7f79b800b540 comp rx=0 tx=0).stop 2023-09-23T09:25:19.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc100e60 msgr2=0x7f79cc074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:19.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc100e60 0x7f79cc074bc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d980 tx=0x7f79c000dd40 comp rx=0 tx=0).stop 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 shutdown_connections 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f79bc0a1620 0x7f79bc0a3ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79cc102a10 0x7f79cc073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79cc100e60 0x7f79cc074bc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 --2- 172.21.15.45:0/3515431121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79cc102050 0x7f79cc073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.863+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 >> 172.21.15.45:0/3515431121 conn(0x7f79cc0fc410 msgr2=0x7f79cc105610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.864+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 shutdown_connections 2023-09-23T09:25:19.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:25:19.864+0000 7f79d31f1700 1 -- 172.21.15.45:0/3515431121 wait complete. 2023-09-23T09:25:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:20 smithi045 ceph-mon[98627]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:25:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.299 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.6.service 2023-09-23T09:25:20.301 INFO:tasks.cephadm:Deploying osd.7 on smithi098 with /dev/vg_nvme/lv_2... 2023-09-23T09:25:20.302 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-23T09:25:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:20 smithi123 ceph-mon[103577]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:25:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.346 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:20 smithi098 ceph-mon[102386]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:25:20.346 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.346 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.347 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.347 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:20.357 INFO:journalctl@ceph.osd.6.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:25:20.881 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:25:21.119 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:25:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:25:21.059+0000 7f6bc2ca9640 -1 osd.6 0 log_to_monitors true 2023-09-23T09:25:21.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:21 smithi098 ceph-mon[102386]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:25:21.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:21 smithi098 ceph-mon[102386]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-23T09:25:21.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:21 smithi045 ceph-mon[98627]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:25:21.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:21 smithi045 ceph-mon[98627]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-23T09:25:21.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:21 smithi123 ceph-mon[103577]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:25:21.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:21 smithi123 ceph-mon[103577]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-23T09:25:22.404 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-23T09:25:22.404 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-23T09:25:22.404 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-23T09:25:22.404 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-23T09:25:22.404 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110477 s, 949 MB/s 2023-09-23T09:25:22.404 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-23T09:25:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:22 smithi098 ceph-mon[102386]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-23T09:25:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:22 smithi098 ceph-mon[102386]: osdmap e43: 7 total, 6 up, 7 in 2023-09-23T09:25:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:22 smithi098 ceph-mon[102386]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:25:22.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:22 smithi045 ceph-mon[98627]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-23T09:25:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:22 smithi045 ceph-mon[98627]: osdmap e43: 7 total, 6 up, 7 in 2023-09-23T09:25:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:22 smithi045 ceph-mon[98627]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:25:22.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:22.580 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:25:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:25:22.229+0000 7f6bb8321700 -1 osd.6 0 waiting for initial osdmap 2023-09-23T09:25:22.581 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:25:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:25:22.240+0000 7f6bb010e700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:25:22.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:22 smithi123 ceph-mon[103577]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-23T09:25:22.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:22 smithi123 ceph-mon[103577]: osdmap e43: 7 total, 6 up, 7 in 2023-09-23T09:25:22.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:22 smithi123 ceph-mon[103577]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:25:22.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:22.941 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: osdmap e44: 7 total, 6 up, 7 in 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi045 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi045 to 5724M 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:23.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:25:23.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: osdmap e44: 7 total, 6 up, 7 in 2023-09-23T09:25:23.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:23.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:23.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:23.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi045 2023-09-23T09:25:23.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi045 to 5724M 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:23.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:23.239 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.502 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:25:23.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:25:23.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: osdmap e44: 7 total, 6 up, 7 in 2023-09-23T09:25:23.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi045 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi045 to 5724M 2023-09-23T09:25:23.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:23.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:23.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:23.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:23.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:23.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:23.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:24.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:24 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:25:24.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:24 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:25:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:24 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:24 smithi098 ceph-mon[102386]: osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] boot 2023-09-23T09:25:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:24 smithi098 ceph-mon[102386]: osdmap e45: 7 total, 7 up, 7 in 2023-09-23T09:25:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:24 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:24 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:25:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:24 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:25:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:24 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:24 smithi045 ceph-mon[98627]: osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] boot 2023-09-23T09:25:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:24 smithi045 ceph-mon[98627]: osdmap e45: 7 total, 7 up, 7 in 2023-09-23T09:25:24.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:24 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:24.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:24 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:25:24.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:24 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:25:24.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:24 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:24.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:24 smithi123 ceph-mon[103577]: osd.6 [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] boot 2023-09-23T09:25:24.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:24 smithi123 ceph-mon[103577]: osdmap e45: 7 total, 7 up, 7 in 2023-09-23T09:25:24.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:24 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:25:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 -- 172.21.15.98:0/1587490499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c100fb0 msgr2=0x7eff1c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 --2- 172.21.15.98:0/1587490499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c100fb0 0x7eff1c101410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7eff180099c0 tx=0x7eff18009cd0 comp rx=0 tx=0).stop 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 -- 172.21.15.98:0/1587490499 shutdown_connections 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 --2- 172.21.15.98:0/1587490499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff1c101950 0x7eff1c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 --2- 172.21.15.98:0/1587490499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c0ffdb0 0x7eff1c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 --2- 172.21.15.98:0/1587490499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c100fb0 0x7eff1c101410 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 -- 172.21.15.98:0/1587490499 >> 172.21.15.98:0/1587490499 conn(0x7eff1c0fb330 msgr2=0x7eff1c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 -- 172.21.15.98:0/1587490499 shutdown_connections 2023-09-23T09:25:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.682+0000 7eff23531700 1 -- 172.21.15.98:0/1587490499 wait complete. 2023-09-23T09:25:24.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 Processor -- start 2023-09-23T09:25:24.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 -- start start 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff1c0ffdb0 0x7eff1c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c100fb0 0x7eff1c197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c101950 0x7eff1c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c19e6d0 con 0x7eff1c0ffdb0 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c19e810 con 0x7eff1c101950 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff23531700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c19e9a0 con 0x7eff1c100fb0 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c101950 0x7eff1c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c101950 0x7eff1c19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46696/0 (socket says 172.21.15.98:46696) 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 -- 172.21.15.98:0/1383772420 learned_addr learned my addr 172.21.15.98:0/1383772420 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff20acc700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c100fb0 0x7eff1c197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff212cd700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff1c0ffdb0 0x7eff1c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 -- 172.21.15.98:0/1383772420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c100fb0 msgr2=0x7eff1c197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c100fb0 0x7eff1c197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 -- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff1c0ffdb0 msgr2=0x7eff1c196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff1c0ffdb0 0x7eff1c196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 -- 172.21.15.98:0/1383772420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff180095f0 con 0x7eff1c101950 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff20acc700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c100fb0 0x7eff1c197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff21ace700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c101950 0x7eff1c19e100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7eff1400ada0 tx=0x7eff1400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.683+0000 7eff127fc700 1 -- 172.21.15.98:0/1383772420 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400f070 con 0x7eff1c101950 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.684+0000 7eff127fc700 1 -- 172.21.15.98:0/1383772420 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff14007b30 con 0x7eff1c101950 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.684+0000 7eff127fc700 1 -- 172.21.15.98:0/1383772420 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff140054e0 con 0x7eff1c101950 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.684+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c19ec80 con 0x7eff1c101950 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.684+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c19f130 con 0x7eff1c101950 2023-09-23T09:25:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.685+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c10b5b0 con 0x7eff1c101950 2023-09-23T09:25:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.686+0000 7eff127fc700 1 -- 172.21.15.98:0/1383772420 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7eff14004ad0 con 0x7eff1c101950 2023-09-23T09:25:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.686+0000 7eff127fc700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7eff080a1620 0x7eff080a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.687+0000 7eff127fc700 1 -- 172.21.15.98:0/1383772420 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7eff140cbdb0 con 0x7eff1c101950 2023-09-23T09:25:24.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.687+0000 7eff212cd700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7eff080a1620 0x7eff080a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:24.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.687+0000 7eff212cd700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7eff080a1620 0x7eff080a3ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff0c005950 tx=0x7eff0c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:25:24.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.690+0000 7eff127fc700 1 -- 172.21.15.98:0/1383772420 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7eff14096140 con 0x7eff1c101950 2023-09-23T09:25:24.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:24.885+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7eff1c0698e0 con 0x7eff080a1620 2023-09-23T09:25:25.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:25 smithi098 ceph-mon[102386]: osdmap e46: 7 total, 7 up, 7 in 2023-09-23T09:25:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:25 smithi098 ceph-mon[102386]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:25 smithi098 ceph-mon[102386]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:25 smithi045 ceph-mon[98627]: osdmap e46: 7 total, 7 up, 7 in 2023-09-23T09:25:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:25 smithi045 ceph-mon[98627]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:25 smithi045 ceph-mon[98627]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:25.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:25.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:25 smithi123 ceph-mon[103577]: osdmap e46: 7 total, 7 up, 7 in 2023-09-23T09:25:25.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:25 smithi123 ceph-mon[103577]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:25 smithi123 ceph-mon[103577]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:27 smithi098 ceph-mon[102386]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:27 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/3652041410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]: dispatch 2023-09-23T09:25:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:27 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]: dispatch 2023-09-23T09:25:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:27 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]': finished 2023-09-23T09:25:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:27 smithi098 ceph-mon[102386]: osdmap e47: 8 total, 7 up, 8 in 2023-09-23T09:25:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:27 smithi045 ceph-mon[98627]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:27 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/3652041410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]: dispatch 2023-09-23T09:25:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:27 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]: dispatch 2023-09-23T09:25:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:27 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]': finished 2023-09-23T09:25:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:27 smithi045 ceph-mon[98627]: osdmap e47: 8 total, 7 up, 8 in 2023-09-23T09:25:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:27.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:27 smithi123 ceph-mon[103577]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:27.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:27 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/3652041410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]: dispatch 2023-09-23T09:25:27.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:27 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]: dispatch 2023-09-23T09:25:27.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:27 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f045102a-f2d2-4e94-a7ac-ff19e15c4126"}]': finished 2023-09-23T09:25:27.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:27 smithi123 ceph-mon[103577]: osdmap e47: 8 total, 7 up, 8 in 2023-09-23T09:25:27.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:28.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:28 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/37726899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:28.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:28 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/37726899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:28.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:28 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/37726899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:29.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:29 smithi098 ceph-mon[102386]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:29.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:29 smithi045 ceph-mon[98627]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:29.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:29 smithi123 ceph-mon[103577]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:31 smithi098 ceph-mon[102386]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:31.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:31 smithi045 ceph-mon[98627]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:31.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:31 smithi123 ceph-mon[103577]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:33.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:33 smithi098 ceph-mon[102386]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:33 smithi045 ceph-mon[98627]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:33.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:33 smithi123 ceph-mon[103577]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:34.699 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:34 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-23T09:25:34.700 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:34 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:34.700 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:34 smithi098 ceph-mon[102386]: Deploying daemon osd.7 on smithi098 2023-09-23T09:25:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:34 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-23T09:25:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:34 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:34 smithi045 ceph-mon[98627]: Deploying daemon osd.7 on smithi098 2023-09-23T09:25:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:34 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-23T09:25:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:34 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:34.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:34 smithi123 ceph-mon[103577]: Deploying daemon osd.7 on smithi098 2023-09-23T09:25:35.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:35 smithi098 ceph-mon[102386]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:35 smithi045 ceph-mon[98627]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:35 smithi123 ceph-mon[103577]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:37.701 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:37 smithi098 ceph-mon[102386]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:37 smithi045 ceph-mon[98627]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:37.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:37 smithi123 ceph-mon[103577]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:39.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:39 smithi098 ceph-mon[102386]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:39.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:39.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:39.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:39 smithi045 ceph-mon[98627]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:39 smithi123 ceph-mon[103577]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:41.051 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 7 on host 'smithi098' 2023-09-23T09:25:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.048+0000 7eff127fc700 1 -- 172.21.15.98:0/1383772420 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff1c0698e0 con 0x7eff080a1620 2023-09-23T09:25:41.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7eff080a1620 msgr2=0x7eff080a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7eff080a1620 0x7eff080a3ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff0c005950 tx=0x7eff0c0058e0 comp rx=0 tx=0).stop 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c101950 msgr2=0x7eff1c19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c101950 0x7eff1c19e100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7eff1400ada0 tx=0x7eff1400c330 comp rx=0 tx=0).stop 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 shutdown_connections 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff1c101950 0x7eff1c19e100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff1c0ffdb0 0x7eff1c196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7eff080a1620 0x7eff080a3ae0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 --2- 172.21.15.98:0/1383772420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c100fb0 0x7eff1c197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 >> 172.21.15.98:0/1383772420 conn(0x7eff1c0fb330 msgr2=0x7eff1c0691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 shutdown_connections 2023-09-23T09:25:41.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:25:41.052+0000 7eff23531700 1 -- 172.21.15.98:0/1383772420 wait complete. 2023-09-23T09:25:41.592 DEBUG:teuthology.orchestra.run.smithi098:osd.7> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.7.service 2023-09-23T09:25:41.594 INFO:tasks.cephadm:Deploying osd.8 on smithi123 with /dev/vg_nvme/lv_2... 2023-09-23T09:25:41.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-23T09:25:41.626 INFO:journalctl@ceph.osd.7.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:25:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:41 smithi123 ceph-mon[103577]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.172 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:25:42.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:41 smithi098 ceph-mon[102386]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:42.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:41 smithi045 ceph-mon[98627]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:42.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:42.450 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:25:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:25:42.204+0000 7feceab36640 -1 osd.7 0 log_to_monitors true 2023-09-23T09:25:43.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:42 smithi123 ceph-mon[103577]: from='osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T09:25:43.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:42 smithi123 ceph-mon[103577]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T09:25:43.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:42 smithi098 ceph-mon[102386]: from='osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T09:25:43.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:42 smithi098 ceph-mon[102386]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T09:25:43.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:42 smithi045 ceph-mon[98627]: from='osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T09:25:43.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:42 smithi045 ceph-mon[98627]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T09:25:43.685 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-23T09:25:43.686 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-23T09:25:43.686 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-23T09:25:43.686 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-23T09:25:43.686 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145961 s, 718 MB/s 2023-09-23T09:25:43.686 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-23T09:25:44.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:43 smithi123 ceph-mon[103577]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:44.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:43 smithi123 ceph-mon[103577]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-23T09:25:44.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:43 smithi123 ceph-mon[103577]: osdmap e48: 8 total, 7 up, 8 in 2023-09-23T09:25:44.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:43 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:44.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:43 smithi123 ceph-mon[103577]: from='osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:25:44.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:43 smithi123 ceph-mon[103577]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:25:44.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2023-09-23T09:25:44.044 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:25:43.879+0000 7fece01ae700 -1 osd.7 0 waiting for initial osdmap 2023-09-23T09:25:44.044 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:25:43.891+0000 7fecd7f9b700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:25:44.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-mon[102386]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:44.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-mon[102386]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-23T09:25:44.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-mon[102386]: osdmap e48: 8 total, 7 up, 8 in 2023-09-23T09:25:44.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:44.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-mon[102386]: from='osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:25:44.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:43 smithi098 ceph-mon[102386]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:25:44.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:43 smithi045 ceph-mon[98627]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:44.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:43 smithi045 ceph-mon[98627]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-23T09:25:44.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:43 smithi045 ceph-mon[98627]: osdmap e48: 8 total, 7 up, 8 in 2023-09-23T09:25:44.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:43 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:44.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:43 smithi045 ceph-mon[98627]: from='osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:25:44.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:43 smithi045 ceph-mon[98627]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:25:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:25:45.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:25:45.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: osdmap e49: 8 total, 7 up, 8 in 2023-09-23T09:25:45.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:45.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:45.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi098 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi098 to 5722M 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:45.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:45.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:45.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:45.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: osdmap e49: 8 total, 7 up, 8 in 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi098 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi098 to 5722M 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:45.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:25:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: osdmap e49: 8 total, 7 up, 8 in 2023-09-23T09:25:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi098 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi098 to 5722M 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:25:45.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:45.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:25:45.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 -- 172.21.15.123:0/1743156142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560102af0 msgr2=0x7f05601093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 --2- 172.21.15.123:0/1743156142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560102af0 0x7f05601093a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f055800b0a0 tx=0x7f055800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 -- 172.21.15.123:0/1743156142 shutdown_connections 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 --2- 172.21.15.123:0/1743156142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560102af0 0x7f05601093a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 --2- 172.21.15.123:0/1743156142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600ff170 0x7f05600ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 --2- 172.21.15.123:0/1743156142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560102150 0x7f05601025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 -- 172.21.15.123:0/1743156142 >> 172.21.15.123:0/1743156142 conn(0x7f05600faad0 msgr2=0x7f05600fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:25:45.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 -- 172.21.15.123:0/1743156142 shutdown_connections 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.869+0000 7f05672f4700 1 -- 172.21.15.123:0/1743156142 wait complete. 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 Processor -- start 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 -- start start 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05600ff170 0x7f0560101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560102150 0x7f0560100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560102af0 0x7f0560100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560100dc0 con 0x7f0560102af0 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560100f30 con 0x7f0560102150 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f05672f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601a0cb0 con 0x7f05600ff170 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05600ff170 0x7f0560101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05600ff170 0x7f0560101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48312/0 (socket says 172.21.15.123:48312) 2023-09-23T09:25:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 -- 172.21.15.123:0/3771212948 learned_addr learned my addr 172.21.15.123:0/3771212948 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:25:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f056488f700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560102150 0x7f0560100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565891700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560102af0 0x7f0560100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 -- 172.21.15.123:0/3771212948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560102150 msgr2=0x7f0560100100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560102150 0x7f0560100100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 -- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560102af0 msgr2=0x7f0560100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:25:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560102af0 0x7f0560100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:25:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 -- 172.21.15.123:0/3771212948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0558009d00 con 0x7f05600ff170 2023-09-23T09:25:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.870+0000 7f0565090700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05600ff170 0x7f0560101ab0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f055000b500 tx=0x7f055000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:25:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.871+0000 7f05567fc700 1 -- 172.21.15.123:0/3771212948 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550011830 con 0x7f05600ff170 2023-09-23T09:25:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.871+0000 7f05567fc700 1 -- 172.21.15.123:0/3771212948 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0550011e50 con 0x7f05600ff170 2023-09-23T09:25:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.871+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05601a0f90 con 0x7f05600ff170 2023-09-23T09:25:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.871+0000 7f05567fc700 1 -- 172.21.15.123:0/3771212948 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000f2e0 con 0x7f05600ff170 2023-09-23T09:25:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.871+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05601a1430 con 0x7f05600ff170 2023-09-23T09:25:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.872+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0560108400 con 0x7f05600ff170 2023-09-23T09:25:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.873+0000 7f05567fc700 1 -- 172.21.15.123:0/3771212948 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f0550010580 con 0x7f05600ff170 2023-09-23T09:25:45.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.873+0000 7f05567fc700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f054c0a1620 0x7f054c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:25:45.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.873+0000 7f05567fc700 1 -- 172.21.15.123:0/3771212948 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f05500cc380 con 0x7f05600ff170 2023-09-23T09:25:45.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.874+0000 7f056488f700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f054c0a1620 0x7f054c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:25:45.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.874+0000 7f056488f700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f054c0a1620 0x7f054c0a3ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f055c005ea0 tx=0x7f055c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:25:45.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:45.877+0000 7f05567fc700 1 -- 172.21.15.123:0/3771212948 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f05500964b0 con 0x7f05600ff170 2023-09-23T09:25:46.076 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:25:46.074+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f056002d270 con 0x7f054c0a1620 2023-09-23T09:25:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:45 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:25:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:45 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:25:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:45 smithi123 ceph-mon[103577]: pgmap v154: 1 pgs: 1 unknown; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:45 smithi123 ceph-mon[103577]: osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] boot 2023-09-23T09:25:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:45 smithi123 ceph-mon[103577]: osdmap e50: 8 total, 8 up, 8 in 2023-09-23T09:25:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:45 smithi123 ceph-mon[103577]: osdmap e51: 8 total, 8 up, 8 in 2023-09-23T09:25:46.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:45 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:25:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:45 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:25:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:45 smithi098 ceph-mon[102386]: pgmap v154: 1 pgs: 1 unknown; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:45 smithi098 ceph-mon[102386]: osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] boot 2023-09-23T09:25:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:45 smithi098 ceph-mon[102386]: osdmap e50: 8 total, 8 up, 8 in 2023-09-23T09:25:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:46.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:45 smithi098 ceph-mon[102386]: osdmap e51: 8 total, 8 up, 8 in 2023-09-23T09:25:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:45 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:25:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:45 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:25:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:45 smithi045 ceph-mon[98627]: pgmap v154: 1 pgs: 1 unknown; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T09:25:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:45 smithi045 ceph-mon[98627]: osd.7 [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] boot 2023-09-23T09:25:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:45 smithi045 ceph-mon[98627]: osdmap e50: 8 total, 8 up, 8 in 2023-09-23T09:25:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:25:46.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:45 smithi045 ceph-mon[98627]: osdmap e51: 8 total, 8 up, 8 in 2023-09-23T09:25:47.204 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:46 smithi123 ceph-mon[103577]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:47.204 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:47.204 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:47.204 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:46 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:47.205 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:46 smithi123 ceph-mon[103577]: osdmap e52: 8 total, 8 up, 8 in 2023-09-23T09:25:47.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:46 smithi098 ceph-mon[102386]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:47.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:46 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:47.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:46 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:47.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:46 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:47.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:46 smithi098 ceph-mon[102386]: osdmap e52: 8 total, 8 up, 8 in 2023-09-23T09:25:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:46 smithi045 ceph-mon[98627]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:25:47.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:25:47.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:25:47.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:46 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:47.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:46 smithi045 ceph-mon[98627]: osdmap e52: 8 total, 8 up, 8 in 2023-09-23T09:25:48.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:47 smithi123 ceph-mon[103577]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:48.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:47 smithi123 ceph-mon[103577]: osdmap e53: 8 total, 8 up, 8 in 2023-09-23T09:25:48.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:47 smithi098 ceph-mon[102386]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:47 smithi098 ceph-mon[102386]: osdmap e53: 8 total, 8 up, 8 in 2023-09-23T09:25:48.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:47 smithi045 ceph-mon[98627]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:48.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:47 smithi045 ceph-mon[98627]: osdmap e53: 8 total, 8 up, 8 in 2023-09-23T09:25:49.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: osdmap e54: 8 total, 8 up, 8 in 2023-09-23T09:25:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/3038208742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]: dispatch 2023-09-23T09:25:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]: dispatch 2023-09-23T09:25:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]': finished 2023-09-23T09:25:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: osdmap e55: 9 total, 8 up, 9 in 2023-09-23T09:25:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:25:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:49 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/2467179517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:49.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: osdmap e54: 8 total, 8 up, 8 in 2023-09-23T09:25:49.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:49.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/3038208742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]: dispatch 2023-09-23T09:25:49.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]: dispatch 2023-09-23T09:25:49.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]': finished 2023-09-23T09:25:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: osdmap e55: 9 total, 8 up, 9 in 2023-09-23T09:25:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:25:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:49 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/2467179517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:49.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: osdmap e54: 8 total, 8 up, 8 in 2023-09-23T09:25:49.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:49.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/3038208742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]: dispatch 2023-09-23T09:25:49.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]: dispatch 2023-09-23T09:25:49.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc7b9f6-15f6-4025-8728-353e53602172"}]': finished 2023-09-23T09:25:49.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: osdmap e55: 9 total, 8 up, 9 in 2023-09-23T09:25:49.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:25:49.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:49 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/2467179517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:25:51.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:51 smithi098 ceph-mon[102386]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:51.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:51 smithi045 ceph-mon[98627]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:51.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:51 smithi123 ceph-mon[103577]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:53.662 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:53 smithi123 ceph-mon[103577]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:53.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:53 smithi098 ceph-mon[102386]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:53.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:53 smithi045 ceph-mon[98627]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:55.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:55 smithi123 ceph-mon[103577]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:55.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-23T09:25:55.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:55.718 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:55 smithi123 ceph-mon[103577]: Deploying daemon osd.8 on smithi123 2023-09-23T09:25:55.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:55 smithi098 ceph-mon[102386]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:55.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-23T09:25:55.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:55.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:55 smithi098 ceph-mon[102386]: Deploying daemon osd.8 on smithi123 2023-09-23T09:25:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:55 smithi045 ceph-mon[98627]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-23T09:25:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:25:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:55 smithi045 ceph-mon[98627]: Deploying daemon osd.8 on smithi123 2023-09-23T09:25:57.573 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:57 smithi123 ceph-mon[103577]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:57.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:57 smithi098 ceph-mon[102386]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:57 smithi045 ceph-mon[98627]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:59.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:25:59 smithi123 ceph-mon[103577]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:59.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:25:59 smithi098 ceph-mon[102386]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:25:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:25:59 smithi045 ceph-mon[98627]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:00.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:00.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:00.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:00 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:00.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:00.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:00.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:00 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:01.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:01.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:01.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:00 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:01.582 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:01 smithi123 ceph-mon[103577]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:02.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:01 smithi098 ceph-mon[102386]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:02.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:01 smithi045 ceph-mon[98627]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:02.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.136+0000 7f05567fc700 1 -- 172.21.15.123:0/3771212948 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f056002d270 con 0x7f054c0a1620 2023-09-23T09:26:02.139 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 8 on host 'smithi123' 2023-09-23T09:26:02.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f054c0a1620 msgr2=0x7f054c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:02.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f054c0a1620 0x7f054c0a3ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f055c005ea0 tx=0x7f055c005e30 comp rx=0 tx=0).stop 2023-09-23T09:26:02.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05600ff170 msgr2=0x7f0560101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:02.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05600ff170 0x7f0560101ab0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f055000b500 tx=0x7f055000b8c0 comp rx=0 tx=0).stop 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 shutdown_connections 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560102af0 0x7f0560100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f054c0a1620 0x7f054c0a3ae0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05600ff170 0x7f0560101ab0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 --2- 172.21.15.123:0/3771212948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560102150 0x7f0560100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 >> 172.21.15.123:0/3771212948 conn(0x7f05600faad0 msgr2=0x7f0560105540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 shutdown_connections 2023-09-23T09:26:02.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:02.139+0000 7f05672f4700 1 -- 172.21.15.123:0/3771212948 wait complete. 2023-09-23T09:26:02.575 DEBUG:teuthology.orchestra.run.smithi123:osd.8> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.8.service 2023-09-23T09:26:02.577 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2023-09-23T09:26:02.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-23T09:26:02.608 INFO:journalctl@ceph.osd.8.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:26:02.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:02.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:02.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:02.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:02 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:03.159 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:26:03.216 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.217 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.217 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:03.217 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:03.217 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.217 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.217 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.217 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:02 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:03.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:03.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:03.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:03.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:02 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:03.839 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:26:03 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:26:03.486+0000 7fbe46445640 -1 osd.8 0 log_to_monitors true 2023-09-23T09:26:04.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:04 smithi045 ceph-mon[98627]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:04.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:04 smithi045 ceph-mon[98627]: from='osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T09:26:04.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:04 smithi045 ceph-mon[98627]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T09:26:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-23T09:26:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-23T09:26:04.539 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-09-23T09:26:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-09-23T09:26:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00795107 s, 1.3 GB/s 2023-09-23T09:26:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-09-23T09:26:04.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:04 smithi098 ceph-mon[102386]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:04.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:04 smithi098 ceph-mon[102386]: from='osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T09:26:04.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:04 smithi098 ceph-mon[102386]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T09:26:04.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:04 smithi123 ceph-mon[103577]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:04.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:04 smithi123 ceph-mon[103577]: from='osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T09:26:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:04 smithi123 ceph-mon[103577]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T09:26:05.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2023-09-23T09:26:05.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:05 smithi045 ceph-mon[98627]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-23T09:26:05.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:05 smithi045 ceph-mon[98627]: osdmap e56: 9 total, 8 up, 9 in 2023-09-23T09:26:05.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:05 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:05.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:05 smithi045 ceph-mon[98627]: from='osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:26:05.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:05 smithi045 ceph-mon[98627]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:26:05.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:26:05 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:26:05.109+0000 7fbe3babd700 -1 osd.8 0 waiting for initial osdmap 2023-09-23T09:26:05.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:26:05 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:26:05.119+0000 7fbe368b0700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:26:05.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:05 smithi123 ceph-mon[103577]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-23T09:26:05.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:05 smithi123 ceph-mon[103577]: osdmap e56: 9 total, 8 up, 9 in 2023-09-23T09:26:05.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:05 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:05.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:05 smithi123 ceph-mon[103577]: from='osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:26:05.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:05 smithi123 ceph-mon[103577]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:26:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:05 smithi098 ceph-mon[102386]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-23T09:26:05.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:05 smithi098 ceph-mon[102386]: osdmap e56: 9 total, 8 up, 9 in 2023-09-23T09:26:05.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:05 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:05.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:05 smithi098 ceph-mon[102386]: from='osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:26:05.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:05 smithi098 ceph-mon[102386]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:26:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:26:06.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:06.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi123 2023-09-23T09:26:06.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: osdmap e57: 9 total, 8 up, 9 in 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi123 to 5722M 2023-09-23T09:26:06.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:06.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:06.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] boot 2023-09-23T09:26:06.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: osdmap e58: 9 total, 9 up, 9 in 2023-09-23T09:26:06.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:06 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi123 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: osdmap e57: 9 total, 8 up, 9 in 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi123 to 5722M 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] boot 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: osdmap e58: 9 total, 9 up, 9 in 2023-09-23T09:26:06.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:06 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi123 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: osdmap e57: 9 total, 8 up, 9 in 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi123 to 5722M 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: osd.8 [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] boot 2023-09-23T09:26:06.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: osdmap e58: 9 total, 9 up, 9 in 2023-09-23T09:26:06.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:06 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:26:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 -- 172.21.15.45:0/3076216209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100a20 msgr2=0x7fd88c1082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 --2- 172.21.15.45:0/3076216209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100a20 0x7fd88c1082c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b0a0 tx=0x7fd88400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 -- 172.21.15.45:0/3076216209 shutdown_connections 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 --2- 172.21.15.45:0/3076216209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100a20 0x7fd88c1082c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 --2- 172.21.15.45:0/3076216209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd88c101400 0x7fd88c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 --2- 172.21.15.45:0/3076216209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd88c100060 0x7fd88c1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 -- 172.21.15.45:0/3076216209 >> 172.21.15.45:0/3076216209 conn(0x7fd88c0fab10 msgr2=0x7fd88c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 -- 172.21.15.45:0/3076216209 shutdown_connections 2023-09-23T09:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.788+0000 7fd892d85700 1 -- 172.21.15.45:0/3076216209 wait complete. 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.789+0000 7fd892d85700 1 Processor -- start 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.789+0000 7fd892d85700 1 -- start start 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.789+0000 7fd892d85700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100060 0x7fd88c0ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.789+0000 7fd892d85700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd88c100a20 0x7fd88c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd892d85700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd88c101400 0x7fd88c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd892d85700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c19e740 con 0x7fd88c100060 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd892d85700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c19e8b0 con 0x7fd88c100a20 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd892d85700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c19ea70 con 0x7fd88c101400 2023-09-23T09:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100060 0x7fd88c0ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100060 0x7fd88c0ffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58262/0 (socket says 172.21.15.45:58262) 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 -- 172.21.15.45:0/3025694595 learned_addr learned my addr 172.21.15.45:0/3025694595 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd88bfff700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd88c100a20 0x7fd88c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd891322700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd88c101400 0x7fd88c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 -- 172.21.15.45:0/3025694595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd88c101400 msgr2=0x7fd88c19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd88c101400 0x7fd88c19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 -- 172.21.15.45:0/3025694595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd88c100a20 msgr2=0x7fd88c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd88c100a20 0x7fd88c19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 -- 172.21.15.45:0/3025694595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884009d00 con 0x7fd88c100060 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd891322700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd88c101400 0x7fd88c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd890b21700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100060 0x7fd88c0ffab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd88000d980 tx=0x7fd88000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd889ffb700 1 -- 172.21.15.45:0/3025694595 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000de60 con 0x7fd88c100060 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd889ffb700 1 -- 172.21.15.45:0/3025694595 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8800044e0 con 0x7fd88c100060 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c19ed50 con 0x7fd88c100060 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.790+0000 7fd889ffb700 1 -- 172.21.15.45:0/3025694595 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880003be0 con 0x7fd88c100060 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.791+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c19f1f0 con 0x7fd88c100060 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.792+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c106d60 con 0x7fd88c100060 2023-09-23T09:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.792+0000 7fd889ffb700 1 -- 172.21.15.45:0/3025694595 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fd880009bb0 con 0x7fd88c100060 2023-09-23T09:26:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.793+0000 7fd889ffb700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd87c0a15a0 0x7fd87c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.793+0000 7fd889ffb700 1 -- 172.21.15.45:0/3025694595 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fd8800cc250 con 0x7fd88c100060 2023-09-23T09:26:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.793+0000 7fd88bfff700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd87c0a15a0 0x7fd87c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.793+0000 7fd88bfff700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd87c0a15a0 0x7fd87c0a3a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd88c100030 tx=0x7fd87800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:26:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:06.798+0000 7fd889ffb700 1 -- 172.21.15.45:0/3025694595 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd8800961d0 con 0x7fd88c100060 2023-09-23T09:26:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:07.007+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd88c02d1e0 con 0x7fd87c0a15a0 2023-09-23T09:26:07.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:26:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:26:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: osdmap e59: 9 total, 9 up, 9 in 2023-09-23T09:26:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:07.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: osdmap e59: 9 total, 9 up, 9 in 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:26:07.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:26:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: osdmap e59: 9 total, 9 up, 9 in 2023-09-23T09:26:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:07.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:08.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:08 smithi045 ceph-mon[98627]: osdmap e60: 9 total, 9 up, 9 in 2023-09-23T09:26:08.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:08 smithi098 ceph-mon[102386]: osdmap e60: 9 total, 9 up, 9 in 2023-09-23T09:26:08.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:08 smithi123 ceph-mon[103577]: osdmap e60: 9 total, 9 up, 9 in 2023-09-23T09:26:09.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:09 smithi098 ceph-mon[102386]: osdmap e61: 9 total, 9 up, 9 in 2023-09-23T09:26:09.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:09 smithi098 ceph-mon[102386]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:09.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:09 smithi045 ceph-mon[98627]: osdmap e61: 9 total, 9 up, 9 in 2023-09-23T09:26:09.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:09 smithi045 ceph-mon[98627]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:09.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:09 smithi123 ceph-mon[103577]: osdmap e61: 9 total, 9 up, 9 in 2023-09-23T09:26:09.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:09 smithi123 ceph-mon[103577]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:10.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:10 smithi098 ceph-mon[102386]: osdmap e62: 9 total, 9 up, 9 in 2023-09-23T09:26:10.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:10 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1835058331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3029adaf-0e5e-4ab8-8b6d-d5376029092c"}]: dispatch 2023-09-23T09:26:10.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:10 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1835058331' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3029adaf-0e5e-4ab8-8b6d-d5376029092c"}]': finished 2023-09-23T09:26:10.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:10 smithi098 ceph-mon[102386]: osdmap e63: 10 total, 9 up, 10 in 2023-09-23T09:26:10.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:10 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:10.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:10 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/404996703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:10.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:10 smithi045 ceph-mon[98627]: osdmap e62: 9 total, 9 up, 9 in 2023-09-23T09:26:10.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:10 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1835058331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3029adaf-0e5e-4ab8-8b6d-d5376029092c"}]: dispatch 2023-09-23T09:26:10.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:10 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1835058331' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3029adaf-0e5e-4ab8-8b6d-d5376029092c"}]': finished 2023-09-23T09:26:10.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:10 smithi045 ceph-mon[98627]: osdmap e63: 10 total, 9 up, 10 in 2023-09-23T09:26:10.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:10 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:10.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:10 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/404996703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:10.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:10 smithi123 ceph-mon[103577]: osdmap e62: 9 total, 9 up, 9 in 2023-09-23T09:26:10.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:10 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1835058331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3029adaf-0e5e-4ab8-8b6d-d5376029092c"}]: dispatch 2023-09-23T09:26:10.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:10 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1835058331' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3029adaf-0e5e-4ab8-8b6d-d5376029092c"}]': finished 2023-09-23T09:26:10.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:10 smithi123 ceph-mon[103577]: osdmap e63: 10 total, 9 up, 10 in 2023-09-23T09:26:10.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:10 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:10.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:10 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/404996703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:11.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:11 smithi098 ceph-mon[102386]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T09:26:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:11 smithi045 ceph-mon[98627]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T09:26:11.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:11 smithi123 ceph-mon[103577]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T09:26:13.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:13 smithi098 ceph-mon[102386]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-09-23T09:26:13.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:13 smithi045 ceph-mon[98627]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-09-23T09:26:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:13 smithi123 ceph-mon[103577]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-09-23T09:26:15.767 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:15 smithi045 ceph-mon[98627]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-23T09:26:15.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:15 smithi098 ceph-mon[102386]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-23T09:26:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:15 smithi123 ceph-mon[103577]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-23T09:26:16.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:16 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-23T09:26:16.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:16 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:16.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:16 smithi045 ceph-mon[98627]: Deploying daemon osd.9 on smithi045 2023-09-23T09:26:16.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:16 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-23T09:26:16.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:16 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:16.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:16 smithi098 ceph-mon[102386]: Deploying daemon osd.9 on smithi045 2023-09-23T09:26:16.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:16 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-23T09:26:16.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:16 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:16.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:16 smithi123 ceph-mon[103577]: Deploying daemon osd.9 on smithi045 2023-09-23T09:26:17.473 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:17 smithi045 ceph-mon[98627]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:26:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:17 smithi098 ceph-mon[102386]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:26:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:17 smithi123 ceph-mon[103577]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:26:19.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:19 smithi045 ceph-mon[98627]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-23T09:26:19.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:19 smithi098 ceph-mon[102386]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-23T09:26:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:19 smithi123 ceph-mon[103577]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-23T09:26:21.493 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:21.493 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:21.493 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:21 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:21.493 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:21 smithi045 ceph-mon[98627]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-09-23T09:26:21.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:21 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:21.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:21 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:21.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:21 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:21.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:21 smithi098 ceph-mon[102386]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-09-23T09:26:21.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:21 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:21.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:21 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:21.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:21 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:21.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:21 smithi123 ceph-mon[103577]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-09-23T09:26:22.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.906+0000 7fd889ffb700 1 -- 172.21.15.45:0/3025694595 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd88c02d1e0 con 0x7fd87c0a15a0 2023-09-23T09:26:22.908 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2023-09-23T09:26:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd87c0a15a0 msgr2=0x7fd87c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd87c0a15a0 0x7fd87c0a3a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd88c100030 tx=0x7fd87800b5b0 comp rx=0 tx=0).stop 2023-09-23T09:26:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100060 msgr2=0x7fd88c0ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100060 0x7fd88c0ffab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd88000d980 tx=0x7fd88000dd40 comp rx=0 tx=0).stop 2023-09-23T09:26:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 shutdown_connections 2023-09-23T09:26:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd87c0a15a0 0x7fd87c0a3a60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:22.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd88c101400 0x7fd88c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:22.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c100060 0x7fd88c0ffab0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:22.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 --2- 172.21.15.45:0/3025694595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd88c100a20 0x7fd88c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:22.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 >> 172.21.15.45:0/3025694595 conn(0x7fd88c0fab10 msgr2=0x7fd88c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:26:22.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 shutdown_connections 2023-09-23T09:26:22.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:26:22.908+0000 7fd892d85700 1 -- 172.21.15.45:0/3025694595 wait complete. 2023-09-23T09:26:23.287 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.9.service 2023-09-23T09:26:23.289 INFO:tasks.cephadm:Deploying osd.10 on smithi098 with /dev/vg_nvme/lv_1... 2023-09-23T09:26:23.289 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-23T09:26:23.323 INFO:journalctl@ceph.osd.9.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:26:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:26:24.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:23 smithi045 ceph-mon[98627]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:24.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.280 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:23 smithi098 ceph-mon[102386]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:24.280 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.280 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.280 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.280 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:23 smithi123 ceph-mon[103577]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:24.462 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:26:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:26:24.153+0000 7fe6b34e8640 -1 osd.9 0 log_to_monitors true 2023-09-23T09:26:25.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:24 smithi045 ceph-mon[98627]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-23T09:26:25.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:24 smithi098 ceph-mon[102386]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-23T09:26:25.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:24 smithi123 ceph-mon[103577]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-23T09:26:25.407 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-23T09:26:25.407 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-23T09:26:25.407 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-23T09:26:25.407 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-23T09:26:25.407 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0156744 s, 669 MB/s 2023-09-23T09:26:25.407 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-23T09:26:25.812 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2023-09-23T09:26:26.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:25 smithi098 ceph-mon[102386]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:26.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:25 smithi098 ceph-mon[102386]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-23T09:26:26.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:25 smithi098 ceph-mon[102386]: osdmap e64: 10 total, 9 up, 10 in 2023-09-23T09:26:26.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:25 smithi098 ceph-mon[102386]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:26:26.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:26.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:25 smithi045 ceph-mon[98627]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:26.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:25 smithi045 ceph-mon[98627]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-23T09:26:26.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:25 smithi045 ceph-mon[98627]: osdmap e64: 10 total, 9 up, 10 in 2023-09-23T09:26:26.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:25 smithi045 ceph-mon[98627]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:26:26.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:26.202 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:26:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:26:25.905+0000 7fe6a8b60700 -1 osd.9 0 waiting for initial osdmap 2023-09-23T09:26:26.202 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:26:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:26:25.920+0000 7fe6a4955700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:26:26.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:25 smithi123 ceph-mon[103577]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:26.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:25 smithi123 ceph-mon[103577]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-23T09:26:26.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:25 smithi123 ceph-mon[103577]: osdmap e64: 10 total, 9 up, 10 in 2023-09-23T09:26:26.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:25 smithi123 ceph-mon[103577]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-09-23T09:26:26.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:26.366 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: osdmap e65: 10 total, 9 up, 10 in 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:27.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:26 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:26:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: osdmap e65: 10 total, 9 up, 10 in 2023-09-23T09:26:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:27.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:26 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-09-23T09:26:27.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: osdmap e65: 10 total, 9 up, 10 in 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:27.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:27.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:27.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:27.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:26 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:27.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 -- 172.21.15.98:0/1464772595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 msgr2=0x7fd620106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:27.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 --2- 172.21.15.98:0/1464772595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd620106660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd610009990 tx=0x7fd610009ca0 comp rx=0 tx=0).stop 2023-09-23T09:26:27.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 -- 172.21.15.98:0/1464772595 shutdown_connections 2023-09-23T09:26:27.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 --2- 172.21.15.98:0/1464772595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd620106ba0 0x7fd620109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:27.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 --2- 172.21.15.98:0/1464772595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd620101940 0x7fd620103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:27.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 --2- 172.21.15.98:0/1464772595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd620106660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:27.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 -- 172.21.15.98:0/1464772595 >> 172.21.15.98:0/1464772595 conn(0x7fd6200fb330 msgr2=0x7fd6200fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:26:27.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 -- 172.21.15.98:0/1464772595 shutdown_connections 2023-09-23T09:26:27.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.605+0000 7fd627588700 1 -- 172.21.15.98:0/1464772595 wait complete. 2023-09-23T09:26:27.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 Processor -- start 2023-09-23T09:26:27.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 -- start start 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd620101940 0x7fd620196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd6201970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd620106ba0 0x7fd62019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019e700 con 0x7fd620106ba0 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019e840 con 0x7fd620104270 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd627588700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019e9d0 con 0x7fd620101940 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd624b23700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd6201970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd624b23700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd6201970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46820/0 (socket says 172.21.15.98:46820) 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd624b23700 1 -- 172.21.15.98:0/2166715249 learned_addr learned my addr 172.21.15.98:0/2166715249 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-23T09:26:27.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd625324700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd620101940 0x7fd620196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd625b25700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd620106ba0 0x7fd62019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd624b23700 1 -- 172.21.15.98:0/2166715249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd620101940 msgr2=0x7fd620196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd624b23700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd620101940 0x7fd620196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd624b23700 1 -- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd620106ba0 msgr2=0x7fd62019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.606+0000 7fd624b23700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd620106ba0 0x7fd62019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.607+0000 7fd624b23700 1 -- 172.21.15.98:0/2166715249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100095f0 con 0x7fd620104270 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.607+0000 7fd624b23700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd6201970b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd610000c00 tx=0x7fd61000f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.607+0000 7fd6167fc700 1 -- 172.21.15.98:0/2166715249 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61001d070 con 0x7fd620104270 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.607+0000 7fd6167fc700 1 -- 172.21.15.98:0/2166715249 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd61000fb50 con 0x7fd620104270 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.607+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62019ec50 con 0x7fd620104270 2023-09-23T09:26:27.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.607+0000 7fd6167fc700 1 -- 172.21.15.98:0/2166715249 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610017650 con 0x7fd620104270 2023-09-23T09:26:27.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.607+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62019f140 con 0x7fd620104270 2023-09-23T09:26:27.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.608+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6200fceb0 con 0x7fd620104270 2023-09-23T09:26:27.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.609+0000 7fd6167fc700 1 -- 172.21.15.98:0/2166715249 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fd61000fcc0 con 0x7fd620104270 2023-09-23T09:26:27.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.609+0000 7fd6167fc700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd60c0a1620 0x7fd60c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:27.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.610+0000 7fd6167fc700 1 -- 172.21.15.98:0/2166715249 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd6100cdc80 con 0x7fd620104270 2023-09-23T09:26:27.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.610+0000 7fd625324700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd60c0a1620 0x7fd60c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:27.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.610+0000 7fd625324700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd60c0a1620 0x7fd60c0a3ae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd61c005950 tx=0x7fd61c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:26:27.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.613+0000 7fd6167fc700 1 -- 172.21.15.98:0/2166715249 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd610097b00 con 0x7fd620104270 2023-09-23T09:26:27.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:27.812+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd62002d160 con 0x7fd60c0a1620 2023-09-23T09:26:28.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi045 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi045 to 4293M 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] boot 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: osdmap e66: 10 total, 10 up, 10 in 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:28.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:28.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:28.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:26:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:26:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi045 2023-09-23T09:26:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi045 to 4293M 2023-09-23T09:26:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] boot 2023-09-23T09:26:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: osdmap e66: 10 total, 10 up, 10 in 2023-09-23T09:26:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:28.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:28.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi045 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi045 to 4293M 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: osd.9 [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] boot 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: osdmap e66: 10 total, 10 up, 10 in 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:29.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:28 smithi098 ceph-mon[102386]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:29.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:28 smithi098 ceph-mon[102386]: osdmap e67: 10 total, 10 up, 10 in 2023-09-23T09:26:29.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:28 smithi045 ceph-mon[98627]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:29.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:28 smithi045 ceph-mon[98627]: osdmap e67: 10 total, 10 up, 10 in 2023-09-23T09:26:29.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:28 smithi123 ceph-mon[103577]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:29.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:28 smithi123 ceph-mon[103577]: osdmap e67: 10 total, 10 up, 10 in 2023-09-23T09:26:30.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:29 smithi098 ceph-mon[102386]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:30.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:29 smithi045 ceph-mon[98627]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:30.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:29 smithi123 ceph-mon[103577]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:30 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/1928305370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]: dispatch 2023-09-23T09:26:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:30 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]: dispatch 2023-09-23T09:26:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:30 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]': finished 2023-09-23T09:26:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:30 smithi098 ceph-mon[102386]: osdmap e68: 11 total, 10 up, 11 in 2023-09-23T09:26:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:30 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:30 smithi098 ceph-mon[102386]: from='client.? 172.21.15.98:0/2514676045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:31.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:30 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/1928305370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]: dispatch 2023-09-23T09:26:31.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:30 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]: dispatch 2023-09-23T09:26:31.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:30 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]': finished 2023-09-23T09:26:31.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:30 smithi045 ceph-mon[98627]: osdmap e68: 11 total, 10 up, 11 in 2023-09-23T09:26:31.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:30 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:31.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:30 smithi045 ceph-mon[98627]: from='client.? 172.21.15.98:0/2514676045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:31.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:30 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/1928305370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]: dispatch 2023-09-23T09:26:31.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:30 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]: dispatch 2023-09-23T09:26:31.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:30 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30007304-e7de-46d9-8b01-c03dfc7c22a7"}]': finished 2023-09-23T09:26:31.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:30 smithi123 ceph-mon[103577]: osdmap e68: 11 total, 10 up, 11 in 2023-09-23T09:26:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:30 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:30 smithi123 ceph-mon[103577]: from='client.? 172.21.15.98:0/2514676045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:32.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:31 smithi098 ceph-mon[102386]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:32.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:31 smithi045 ceph-mon[98627]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:32.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:31 smithi123 ceph-mon[103577]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:34.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:33 smithi098 ceph-mon[102386]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:34.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:33 smithi045 ceph-mon[98627]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:34.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:33 smithi123 ceph-mon[103577]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:36.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:35 smithi098 ceph-mon[102386]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:36.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:35 smithi045 ceph-mon[98627]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:36.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:35 smithi123 ceph-mon[103577]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:37.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-23T09:26:37.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:37.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-23T09:26:37.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:37.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-23T09:26:37.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:38.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:37 smithi098 ceph-mon[102386]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:38.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:37 smithi098 ceph-mon[102386]: Deploying daemon osd.10 on smithi098 2023-09-23T09:26:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:37 smithi045 ceph-mon[98627]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:37 smithi045 ceph-mon[98627]: Deploying daemon osd.10 on smithi098 2023-09-23T09:26:38.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:37 smithi123 ceph-mon[103577]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:38.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:37 smithi123 ceph-mon[103577]: Deploying daemon osd.10 on smithi098 2023-09-23T09:26:40.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:39 smithi098 ceph-mon[102386]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:40.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:39 smithi045 ceph-mon[98627]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:40.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:39 smithi123 ceph-mon[103577]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:41.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:41 smithi098 ceph-mon[102386]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:41.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:41 smithi045 ceph-mon[98627]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:41.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:41 smithi123 ceph-mon[103577]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:42.426 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:42.426 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:42.426 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:42.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:42.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:42.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:43 smithi098 ceph-mon[102386]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:43.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:43 smithi045 ceph-mon[98627]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:43.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:43 smithi123 ceph-mon[103577]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:44.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.128+0000 7fd6167fc700 1 -- 172.21.15.98:0/2166715249 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd62002d160 con 0x7fd60c0a1620 2023-09-23T09:26:44.130 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 10 on host 'smithi098' 2023-09-23T09:26:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd60c0a1620 msgr2=0x7fd60c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd60c0a1620 0x7fd60c0a3ae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd61c005950 tx=0x7fd61c00b410 comp rx=0 tx=0).stop 2023-09-23T09:26:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 msgr2=0x7fd6201970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd6201970b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd610000c00 tx=0x7fd61000f710 comp rx=0 tx=0).stop 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 shutdown_connections 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd620106ba0 0x7fd62019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd60c0a1620 0x7fd60c0a3ae0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd620101940 0x7fd620196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 --2- 172.21.15.98:0/2166715249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd620104270 0x7fd6201970b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 >> 172.21.15.98:0/2166715249 conn(0x7fd6200fb330 msgr2=0x7fd6201025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 shutdown_connections 2023-09-23T09:26:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-23T09:26:44.130+0000 7fd627588700 1 -- 172.21.15.98:0/2166715249 wait complete. 2023-09-23T09:26:44.552 DEBUG:teuthology.orchestra.run.smithi098:osd.10> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.10.service 2023-09-23T09:26:44.554 INFO:tasks.cephadm:Deploying osd.11 on smithi123 with /dev/vg_nvme/lv_1... 2023-09-23T09:26:44.555 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-23T09:26:44.587 INFO:journalctl@ceph.osd.10.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:26:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:26:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.498 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:26:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:26:45.282+0000 7f5fec8b7640 -1 osd.10 0 log_to_monitors true 2023-09-23T09:26:45.499 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.499 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.499 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:45.499 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:46.428 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:46 smithi098 ceph-mon[102386]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:46.428 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:46 smithi098 ceph-mon[102386]: from='osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T09:26:46.428 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:46 smithi098 ceph-mon[102386]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T09:26:46.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:46 smithi045 ceph-mon[98627]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:46.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:46 smithi045 ceph-mon[98627]: from='osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T09:26:46.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:46 smithi045 ceph-mon[98627]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T09:26:46.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:46 smithi123 ceph-mon[103577]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:46.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:46 smithi123 ceph-mon[103577]: from='osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T09:26:46.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:46 smithi123 ceph-mon[103577]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T09:26:46.612 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-23T09:26:46.612 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-23T09:26:46.612 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-23T09:26:46.612 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-23T09:26:46.612 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117391 s, 893 MB/s 2023-09-23T09:26:46.612 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-23T09:26:46.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2023-09-23T09:26:47.521 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:26:47.544 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:26:47.251+0000 7f5fe1f2f700 -1 osd.10 0 waiting for initial osdmap 2023-09-23T09:26:47.544 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:26:47.267+0000 7f5fdb51f700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:26:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-mon[102386]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-23T09:26:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-mon[102386]: osdmap e69: 11 total, 10 up, 11 in 2023-09-23T09:26:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-mon[102386]: from='osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:26:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-mon[102386]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:26:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:47 smithi098 ceph-mon[102386]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:47.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:47 smithi045 ceph-mon[98627]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-23T09:26:47.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:47 smithi045 ceph-mon[98627]: osdmap e69: 11 total, 10 up, 11 in 2023-09-23T09:26:47.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:47.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:47 smithi045 ceph-mon[98627]: from='osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:26:47.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:47 smithi045 ceph-mon[98627]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:26:47.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:47 smithi045 ceph-mon[98627]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:47 smithi123 ceph-mon[103577]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-23T09:26:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:47 smithi123 ceph-mon[103577]: osdmap e69: 11 total, 10 up, 11 in 2023-09-23T09:26:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:47 smithi123 ceph-mon[103577]: from='osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:26:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:47 smithi123 ceph-mon[103577]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-23T09:26:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:47 smithi123 ceph-mon[103577]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:48.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: osdmap e70: 11 total, 10 up, 11 in 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi098 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi098 to 4292M 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:48.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:48.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:26:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: osdmap e70: 11 total, 10 up, 11 in 2023-09-23T09:26:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi098 2023-09-23T09:26:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi098 to 4292M 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:48.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:48.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:48.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-23T09:26:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: osdmap e70: 11 total, 10 up, 11 in 2023-09-23T09:26:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi098 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi098 to 4292M 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:48.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:48.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:26:48.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:48.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:26:48.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:26:48.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.751+0000 7ff35e06f700 1 -- 172.21.15.123:0/2143574582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 msgr2=0x7ff358102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.751+0000 7ff35e06f700 1 --2- 172.21.15.123:0/2143574582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358102500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009990 tx=0x7ff34c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.751+0000 7ff35e06f700 1 -- 172.21.15.123:0/2143574582 shutdown_connections 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.751+0000 7ff35e06f700 1 --2- 172.21.15.123:0/2143574582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358102a40 0x7ff358109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.751+0000 7ff35e06f700 1 --2- 172.21.15.123:0/2143574582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff358100e90 0x7ff3581012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.752+0000 7ff35e06f700 1 --2- 172.21.15.123:0/2143574582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358102500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.752+0000 7ff35e06f700 1 -- 172.21.15.123:0/2143574582 >> 172.21.15.123:0/2143574582 conn(0x7ff3580fc4a0 msgr2=0x7ff3580fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.752+0000 7ff35e06f700 1 -- 172.21.15.123:0/2143574582 shutdown_connections 2023-09-23T09:26:48.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.752+0000 7ff35e06f700 1 -- 172.21.15.123:0/2143574582 wait complete. 2023-09-23T09:26:48.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.752+0000 7ff35e06f700 1 Processor -- start 2023-09-23T09:26:48.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.752+0000 7ff35e06f700 1 -- start start 2023-09-23T09:26:48.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358100e90 0x7ff358196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:48.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff358102a40 0x7ff35819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35819e640 con 0x7ff358100e90 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35819e780 con 0x7ff358102a40 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35819e910 con 0x7ff358102080 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48460/0 (socket says 172.21.15.123:48460) 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 -- 172.21.15.123:0/3037353009 learned_addr learned my addr 172.21.15.123:0/3037353009 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff3577fe700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358100e90 0x7ff358196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff357fff700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff358102a40 0x7ff35819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 -- 172.21.15.123:0/3037353009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff358102a40 msgr2=0x7ff35819e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff358102a40 0x7ff35819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:48.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 -- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358100e90 msgr2=0x7ff358196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358100e90 0x7ff358196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 -- 172.21.15.123:0/3037353009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c0095f0 con 0x7ff358102080 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff3577fe700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358100e90 0x7ff358196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff356ffd700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358196ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c010 tx=0x7ff34c00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff354ff9700 1 -- 172.21.15.123:0/3037353009 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c01d070 con 0x7ff358102080 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35819eb90 con 0x7ff358102080 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff354ff9700 1 -- 172.21.15.123:0/3037353009 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34c00fc20 con 0x7ff358102080 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff354ff9700 1 -- 172.21.15.123:0/3037353009 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c017650 con 0x7ff358102080 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.753+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35819f050 con 0x7ff358102080 2023-09-23T09:26:48.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.754+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff358108490 con 0x7ff358102080 2023-09-23T09:26:48.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.756+0000 7ff354ff9700 1 -- 172.21.15.123:0/3037353009 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7ff34c0050d0 con 0x7ff358102080 2023-09-23T09:26:48.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.757+0000 7ff354ff9700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3400a15a0 0x7ff3400a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:26:48.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.757+0000 7ff354ff9700 1 -- 172.21.15.123:0/3037353009 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7ff34c0cdfe0 con 0x7ff358102080 2023-09-23T09:26:48.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.757+0000 7ff3577fe700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3400a15a0 0x7ff3400a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:26:48.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.757+0000 7ff3577fe700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3400a15a0 0x7ff3400a3a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff358101bf0 tx=0x7ff34800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:26:48.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.760+0000 7ff354ff9700 1 -- 172.21.15.123:0/3037353009 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff34c097c30 con 0x7ff358102080 2023-09-23T09:26:48.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:26:48.960+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff35802d210 con 0x7ff3400a15a0 2023-09-23T09:26:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:26:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:26:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] boot 2023-09-23T09:26:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: osdmap e71: 11 total, 11 up, 11 in 2023-09-23T09:26:49.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:49.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:49.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:49.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:49.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:49.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:49 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:49.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] boot 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: osdmap e71: 11 total, 11 up, 11 in 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:49.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:49 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:49.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:26:49.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: osd.10 [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] boot 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: osdmap e71: 11 total, 11 up, 11 in 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T09:26:49.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:49 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:50.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:50 smithi098 ceph-mon[102386]: osdmap e72: 11 total, 11 up, 11 in 2023-09-23T09:26:50.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:50 smithi123 ceph-mon[103577]: osdmap e72: 11 total, 11 up, 11 in 2023-09-23T09:26:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:50 smithi045 ceph-mon[98627]: osdmap e72: 11 total, 11 up, 11 in 2023-09-23T09:26:51.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:51 smithi098 ceph-mon[102386]: osdmap e73: 11 total, 11 up, 11 in 2023-09-23T09:26:51.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:51 smithi098 ceph-mon[102386]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:51.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:51 smithi045 ceph-mon[98627]: osdmap e73: 11 total, 11 up, 11 in 2023-09-23T09:26:51.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:51 smithi045 ceph-mon[98627]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:51.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:51 smithi123 ceph-mon[103577]: osdmap e73: 11 total, 11 up, 11 in 2023-09-23T09:26:51.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:51 smithi123 ceph-mon[103577]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:52.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:52 smithi098 ceph-mon[102386]: osdmap e74: 11 total, 11 up, 11 in 2023-09-23T09:26:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:52 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/3433388940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]: dispatch 2023-09-23T09:26:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:52 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]: dispatch 2023-09-23T09:26:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:52 smithi098 ceph-mon[102386]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]': finished 2023-09-23T09:26:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:52 smithi098 ceph-mon[102386]: osdmap e75: 12 total, 11 up, 12 in 2023-09-23T09:26:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:52 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:26:52.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:52 smithi098 ceph-mon[102386]: from='client.? 172.21.15.123:0/2997456096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:52 smithi045 ceph-mon[98627]: osdmap e74: 11 total, 11 up, 11 in 2023-09-23T09:26:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:52 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/3433388940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]: dispatch 2023-09-23T09:26:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:52 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]: dispatch 2023-09-23T09:26:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:52 smithi045 ceph-mon[98627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]': finished 2023-09-23T09:26:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:52 smithi045 ceph-mon[98627]: osdmap e75: 12 total, 11 up, 12 in 2023-09-23T09:26:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:52 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:26:52.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:52 smithi045 ceph-mon[98627]: from='client.? 172.21.15.123:0/2997456096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:52 smithi123 ceph-mon[103577]: osdmap e74: 11 total, 11 up, 11 in 2023-09-23T09:26:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:52 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/3433388940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]: dispatch 2023-09-23T09:26:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:52 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]: dispatch 2023-09-23T09:26:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:52 smithi123 ceph-mon[103577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbebaf92-0e82-4758-b581-d15dc738abec"}]': finished 2023-09-23T09:26:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:52 smithi123 ceph-mon[103577]: osdmap e75: 12 total, 11 up, 12 in 2023-09-23T09:26:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:52 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:26:52.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:52 smithi123 ceph-mon[103577]: from='client.? 172.21.15.123:0/2997456096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T09:26:53.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:53 smithi098 ceph-mon[102386]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:53.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:53 smithi045 ceph-mon[98627]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:53.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:53 smithi123 ceph-mon[103577]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:55.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:55 smithi098 ceph-mon[102386]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:55 smithi045 ceph-mon[98627]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:55.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:55 smithi123 ceph-mon[103577]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:26:57.580 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:57 smithi123 ceph-mon[103577]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-23T09:26:57.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:57 smithi098 ceph-mon[102386]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-23T09:26:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:57 smithi045 ceph-mon[98627]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-23T09:26:58.626 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:58 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-23T09:26:58.627 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:58 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:58.627 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:58 smithi123 ceph-mon[103577]: Deploying daemon osd.11 on smithi123 2023-09-23T09:26:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:58 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-23T09:26:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:58 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:58 smithi098 ceph-mon[102386]: Deploying daemon osd.11 on smithi123 2023-09-23T09:26:58.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:58 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-23T09:26:58.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:58 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:26:58.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:58 smithi045 ceph-mon[98627]: Deploying daemon osd.11 on smithi123 2023-09-23T09:26:59.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:26:59 smithi123 ceph-mon[103577]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:26:59.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:26:59 smithi098 ceph-mon[102386]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:26:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:26:59 smithi045 ceph-mon[98627]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T09:27:01.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:01 smithi098 ceph-mon[102386]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-23T09:27:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:01 smithi045 ceph-mon[98627]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-23T09:27:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:01 smithi123 ceph-mon[103577]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-23T09:27:03.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:03 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:03.394 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:03 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:03.394 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:03 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:03.394 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:03 smithi123 ceph-mon[103577]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-09-23T09:27:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:03 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:03 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:03 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:03 smithi098 ceph-mon[102386]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-09-23T09:27:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:03 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:03 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:03 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:03 smithi045 ceph-mon[98627]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-09-23T09:27:04.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.774+0000 7ff354ff9700 1 -- 172.21.15.123:0/3037353009 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff35802d210 con 0x7ff3400a15a0 2023-09-23T09:27:04.776 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 11 on host 'smithi123' 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3400a15a0 msgr2=0x7ff3400a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3400a15a0 0x7ff3400a3a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff358101bf0 tx=0x7ff34800a680 comp rx=0 tx=0).stop 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 msgr2=0x7ff358196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358196ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c010 tx=0x7ff34c00f740 comp rx=0 tx=0).stop 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 shutdown_connections 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff358102a40 0x7ff35819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358100e90 0x7ff358196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:04.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3400a15a0 0x7ff3400a3a60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:04.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 --2- 172.21.15.123:0/3037353009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff358102080 0x7ff358196ff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:04.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 >> 172.21.15.123:0/3037353009 conn(0x7ff3580fc4a0 msgr2=0x7ff3581055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:04.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 shutdown_connections 2023-09-23T09:27:04.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:04.776+0000 7ff35e06f700 1 -- 172.21.15.123:0/3037353009 wait complete. 2023-09-23T09:27:05.215 DEBUG:teuthology.orchestra.run.smithi123:osd.11> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.11.service 2023-09-23T09:27:05.217 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-23T09:27:05.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd stat -f json 2023-09-23T09:27:05.250 INFO:journalctl@ceph.osd.11.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:27:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:27:06.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:05 smithi045 ceph-mon[98627]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:27:06.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:05 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:05 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:05 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:05 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:05 smithi098 ceph-mon[102386]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:27:06.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:05 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:05 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:05 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:05 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:05 smithi123 ceph-mon[103577]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:27:06.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:05 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:05 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:05 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:05 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:06.525 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:27:06 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:27:06.205+0000 7f790f613640 -1 osd.11 0 log_to_monitors true 2023-09-23T09:27:06.800 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:06 smithi123 ceph-mon[103577]: from='osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T09:27:06.800 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:06 smithi123 ceph-mon[103577]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T09:27:06.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 -- 172.21.15.45:0/3783957524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade8100fb0 msgr2=0x7fade8101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 --2- 172.21.15.45:0/3783957524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade8100fb0 0x7fade8101410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fadd80099c0 tx=0x7fadd8009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 -- 172.21.15.45:0/3783957524 shutdown_connections 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 --2- 172.21.15.45:0/3783957524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade8101950 0x7fade8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 --2- 172.21.15.45:0/3783957524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade80ffdb0 0x7fade81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 --2- 172.21.15.45:0/3783957524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade8100fb0 0x7fade8101410 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 -- 172.21.15.45:0/3783957524 >> 172.21.15.45:0/3783957524 conn(0x7fade80fb330 msgr2=0x7fade80fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 -- 172.21.15.45:0/3783957524 shutdown_connections 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.956+0000 7fadf04b9700 1 -- 172.21.15.45:0/3783957524 wait complete. 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 Processor -- start 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 -- start start 2023-09-23T09:27:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade80ffdb0 0x7fade8195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade8100fb0 0x7fade8195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade8101950 0x7fade819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade819d600 con 0x7fade80ffdb0 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade819d740 con 0x7fade8100fb0 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadf04b9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade819d8d0 con 0x7fade8101950 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadee255700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade80ffdb0 0x7fade8195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadeea56700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade8101950 0x7fade819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadee255700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade80ffdb0 0x7fade8195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58412/0 (socket says 172.21.15.45:58412) 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadee255700 1 -- 172.21.15.45:0/1632646662 learned_addr learned my addr 172.21.15.45:0/1632646662 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.957+0000 7fadeda54700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade8100fb0 0x7fade8195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadeea56700 1 -- 172.21.15.45:0/1632646662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade8100fb0 msgr2=0x7fade8195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadeea56700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade8100fb0 0x7fade8195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadeea56700 1 -- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade80ffdb0 msgr2=0x7fade8195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadeea56700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade80ffdb0 0x7fade8195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadeea56700 1 -- 172.21.15.45:0/1632646662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80095f0 con 0x7fade8101950 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadee255700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade80ffdb0 0x7fade8195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadeea56700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade8101950 0x7fade819d030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fade0009f60 tx=0x7fade0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7faddf7fe700 1 -- 172.21.15.45:0/1632646662 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0018070 con 0x7fade8101950 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7faddf7fe700 1 -- 172.21.15.45:0/1632646662 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fade000f040 con 0x7fade8101950 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade819db50 con 0x7fade8101950 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7faddf7fe700 1 -- 172.21.15.45:0/1632646662 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade000ccc0 con 0x7fade8101950 2023-09-23T09:27:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.958+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade819e050 con 0x7fade8101950 2023-09-23T09:27:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.959+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade8107400 con 0x7fade8101950 2023-09-23T09:27:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.960+0000 7faddf7fe700 1 -- 172.21.15.45:0/1632646662 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fade0003a00 con 0x7fade8101950 2023-09-23T09:27:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.961+0000 7faddf7fe700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fadd40a1620 0x7fadd40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.961+0000 7fadee255700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fadd40a1620 0x7fadd40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.961+0000 7faddf7fe700 1 -- 172.21.15.45:0/1632646662 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fade00cc4f0 con 0x7fade8101950 2023-09-23T09:27:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.961+0000 7fadee255700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fadd40a1620 0x7fadd40a3ae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fade4007fd0 tx=0x7fade4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:06.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:06.964+0000 7faddf7fe700 1 -- 172.21.15.45:0/1632646662 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fade00960a0 con 0x7fade8101950 2023-09-23T09:27:07.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:06 smithi098 ceph-mon[102386]: from='osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T09:27:07.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:06 smithi098 ceph-mon[102386]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T09:27:07.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:06 smithi045 ceph-mon[98627]: from='osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T09:27:07.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:06 smithi045 ceph-mon[98627]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T09:27:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.163+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fade810b7c0 con 0x7fade8101950 2023-09-23T09:27:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.163+0000 7faddf7fe700 1 -- 172.21.15.45:0/1632646662 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7fade000cea0 con 0x7fade8101950 2023-09-23T09:27:07.165 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:07.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fadd40a1620 msgr2=0x7fadd40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fadd40a1620 0x7fadd40a3ae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fade4007fd0 tx=0x7fade4007480 comp rx=0 tx=0).stop 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade8101950 msgr2=0x7fade819d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade8101950 0x7fade819d030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fade0009f60 tx=0x7fade0009f90 comp rx=0 tx=0).stop 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 shutdown_connections 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fadd40a1620 0x7fadd40a3ae0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade8101950 0x7fade819d030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade80ffdb0 0x7fade8195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 --2- 172.21.15.45:0/1632646662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade8100fb0 0x7fade8195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.165+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 >> 172.21.15.45:0/1632646662 conn(0x7fade80fb330 msgr2=0x7fade8104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.166+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 shutdown_connections 2023-09-23T09:27:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:07.166+0000 7fadf04b9700 1 -- 172.21.15.45:0/1632646662 wait complete. 2023-09-23T09:27:07.559 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1695461208,"num_in_osds":12,"osd_in_since":1695461211,"num_remapped_pgs":0} 2023-09-23T09:27:08.041 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:27:07.765+0000 7f790548c700 -1 osd.11 0 waiting for initial osdmap 2023-09-23T09:27:08.042 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:27:07.781+0000 7f78fea7c700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T09:27:08.042 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-mon[103577]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:27:08.042 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-mon[103577]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-23T09:27:08.042 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-mon[103577]: osdmap e76: 12 total, 11 up, 12 in 2023-09-23T09:27:08.042 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:08.042 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-mon[103577]: from='osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:27:08.042 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-mon[103577]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:27:08.042 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:07 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1632646662' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T09:27:08.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:07 smithi098 ceph-mon[102386]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:27:08.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:07 smithi098 ceph-mon[102386]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-23T09:27:08.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:07 smithi098 ceph-mon[102386]: osdmap e76: 12 total, 11 up, 12 in 2023-09-23T09:27:08.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:07 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:08.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:07 smithi098 ceph-mon[102386]: from='osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:27:08.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:07 smithi098 ceph-mon[102386]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:27:08.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:07 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1632646662' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T09:27:08.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:07 smithi045 ceph-mon[98627]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-23T09:27:08.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:07 smithi045 ceph-mon[98627]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-23T09:27:08.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:07 smithi045 ceph-mon[98627]: osdmap e76: 12 total, 11 up, 12 in 2023-09-23T09:27:08.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:07 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:08.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:07 smithi045 ceph-mon[98627]: from='osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:27:08.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:07 smithi045 ceph-mon[98627]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-23T09:27:08.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:07 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1632646662' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T09:27:08.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd stat -f json 2023-09-23T09:27:09.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: osdmap e77: 12 total, 11 up, 12 in 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:09.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:09.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:09.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:09.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:09.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:08 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: osdmap e77: 12 total, 11 up, 12 in 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:09.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:08 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-23T09:27:09.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: osdmap e77: 12 total, 11 up, 12 in 2023-09-23T09:27:09.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:09.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:09.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:09.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:08 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:09.132 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:27:10.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: purged_snaps scrub starts 2023-09-23T09:27:10.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: purged_snaps scrub ok 2023-09-23T09:27:10.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:27:10.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: Detected new or changed devices on smithi123 2023-09-23T09:27:10.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi123 to 4292M 2023-09-23T09:27:10.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] boot 2023-09-23T09:27:10.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: osdmap e78: 12 total, 12 up, 12 in 2023-09-23T09:27:10.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:09 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: purged_snaps scrub starts 2023-09-23T09:27:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: purged_snaps scrub ok 2023-09-23T09:27:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:27:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: Detected new or changed devices on smithi123 2023-09-23T09:27:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi123 to 4292M 2023-09-23T09:27:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] boot 2023-09-23T09:27:10.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: osdmap e78: 12 total, 12 up, 12 in 2023-09-23T09:27:10.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:09 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: purged_snaps scrub starts 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: purged_snaps scrub ok 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: Detected new or changed devices on smithi123 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi123 to 4292M 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: osd.11 [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] boot 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: osdmap e78: 12 total, 12 up, 12 in 2023-09-23T09:27:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:09 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:27:10.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 -- 172.21.15.45:0/4163171284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c0ffdb0 msgr2=0x7f123c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 --2- 172.21.15.45:0/4163171284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c0ffdb0 0x7f123c1001d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f122c0099c0 tx=0x7f122c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 -- 172.21.15.45:0/4163171284 shutdown_connections 2023-09-23T09:27:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 --2- 172.21.15.45:0/4163171284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c101950 0x7f123c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 --2- 172.21.15.45:0/4163171284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c0ffdb0 0x7f123c1001d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 --2- 172.21.15.45:0/4163171284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c100fb0 0x7f123c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 -- 172.21.15.45:0/4163171284 >> 172.21.15.45:0/4163171284 conn(0x7f123c0fb330 msgr2=0x7f123c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 -- 172.21.15.45:0/4163171284 shutdown_connections 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.316+0000 7f124417e700 1 -- 172.21.15.45:0/4163171284 wait complete. 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 Processor -- start 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 -- start start 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0ffdb0 0x7f123c103d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c100fb0 0x7f123c102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c101950 0x7f123c1028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c103050 con 0x7f123c100fb0 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1031c0 con 0x7f123c0ffdb0 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124417e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1a0c70 con 0x7f123c101950 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c100fb0 0x7f123c102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c100fb0 0x7f123c102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58424/0 (socket says 172.21.15.45:58424) 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 -- 172.21.15.45:0/2337045551 learned_addr learned my addr 172.21.15.45:0/2337045551 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241f1a700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0ffdb0 0x7f123c103d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f124271b700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c101950 0x7f123c1028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 -- 172.21.15.45:0/2337045551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c101950 msgr2=0x7f123c1028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c101950 0x7f123c1028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 -- 172.21.15.45:0/2337045551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0ffdb0 msgr2=0x7f123c103d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0ffdb0 0x7f123c103d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.317+0000 7f1241719700 1 -- 172.21.15.45:0/2337045551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0095f0 con 0x7f123c100fb0 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.318+0000 7f1241719700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c100fb0 0x7f123c102390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f123800b530 tx=0x7f123800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.318+0000 7f1232ffd700 1 -- 172.21.15.45:0/2337045551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238011830 con 0x7f123c100fb0 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.318+0000 7f1232ffd700 1 -- 172.21.15.45:0/2337045551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1238011e50 con 0x7f123c100fb0 2023-09-23T09:27:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.318+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c1a0f50 con 0x7f123c100fb0 2023-09-23T09:27:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.318+0000 7f1232ffd700 1 -- 172.21.15.45:0/2337045551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123800f2e0 con 0x7f123c100fb0 2023-09-23T09:27:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.318+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c1a13c0 con 0x7f123c100fb0 2023-09-23T09:27:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.319+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c10b630 con 0x7f123c100fb0 2023-09-23T09:27:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.320+0000 7f1232ffd700 1 -- 172.21.15.45:0/2337045551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f1238010380 con 0x7f123c100fb0 2023-09-23T09:27:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.320+0000 7f1232ffd700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f12280a15a0 0x7f12280a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.321+0000 7f1232ffd700 1 -- 172.21.15.45:0/2337045551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12380ccad0 con 0x7f123c100fb0 2023-09-23T09:27:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.321+0000 7f1241f1a700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f12280a15a0 0x7f12280a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:10.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.321+0000 7f1241f1a700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f12280a15a0 0x7f12280a3a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f122c009f50 tx=0x7f122c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:10.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.324+0000 7f1232ffd700 1 -- 172.21.15.45:0/2337045551 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1238096560 con 0x7f123c100fb0 2023-09-23T09:27:10.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.525+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f123c1a1610 con 0x7f123c100fb0 2023-09-23T09:27:10.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.525+0000 7f1232ffd700 1 -- 172.21.15.45:0/2337045551 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1238099b80 con 0x7f123c100fb0 2023-09-23T09:27:10.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:10.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f12280a15a0 msgr2=0x7f12280a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:10.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f12280a15a0 0x7f12280a3a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f122c009f50 tx=0x7f122c00b540 comp rx=0 tx=0).stop 2023-09-23T09:27:10.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c100fb0 msgr2=0x7f123c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c100fb0 0x7f123c102390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f123800b530 tx=0x7f123800b840 comp rx=0 tx=0).stop 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 shutdown_connections 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f12280a15a0 0x7f12280a3a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c101950 0x7f123c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0ffdb0 0x7f123c103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 --2- 172.21.15.45:0/2337045551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f123c100fb0 0x7f123c102390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 >> 172.21.15.45:0/2337045551 conn(0x7f123c0fb330 msgr2=0x7f123c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:10.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 shutdown_connections 2023-09-23T09:27:10.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:10.527+0000 7f124417e700 1 -- 172.21.15.45:0/2337045551 wait complete. 2023-09-23T09:27:10.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:10 smithi045 ceph-mon[98627]: osdmap e79: 12 total, 12 up, 12 in 2023-09-23T09:27:10.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:10 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2337045551' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T09:27:10.995 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1695461228,"num_in_osds":12,"osd_in_since":1695461211,"num_remapped_pgs":0} 2023-09-23T09:27:10.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd dump --format=json 2023-09-23T09:27:11.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:10 smithi098 ceph-mon[102386]: osdmap e79: 12 total, 12 up, 12 in 2023-09-23T09:27:11.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:10 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2337045551' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T09:27:11.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:10 smithi123 ceph-mon[103577]: osdmap e79: 12 total, 12 up, 12 in 2023-09-23T09:27:11.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:10 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2337045551' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T09:27:11.557 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:27:12.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:11 smithi045 ceph-mon[98627]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:12.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:11 smithi098 ceph-mon[102386]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:12.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:11 smithi123 ceph-mon[103577]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:12.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 -- 172.21.15.45:0/2432797500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4100720 msgr2=0x7f05b410cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:12.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 --2- 172.21.15.45:0/2432797500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4100720 0x7f05b410cbc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00b0a0 tx=0x7f05ac00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 -- 172.21.15.45:0/2432797500 shutdown_connections 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 --2- 172.21.15.45:0/2432797500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4100720 0x7f05b410cbc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 --2- 172.21.15.45:0/2432797500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b4105e10 0x7f05b41061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 --2- 172.21.15.45:0/2432797500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b40ffd60 0x7f05b41001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 -- 172.21.15.45:0/2432797500 >> 172.21.15.45:0/2432797500 conn(0x7f05b4076e20 msgr2=0x7f05b4077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 -- 172.21.15.45:0/2432797500 shutdown_connections 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 -- 172.21.15.45:0/2432797500 wait complete. 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 Processor -- start 2023-09-23T09:27:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.734+0000 7f05bbad4700 1 -- start start 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05bbad4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b40ffd60 0x7f05b4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05bbad4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b4100720 0x7f05b41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05bbad4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4105e10 0x7f05b419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05bbad4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b419e0e0 con 0x7f05b4105e10 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05bbad4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b419e250 con 0x7f05b4100720 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05bbad4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b419e410 con 0x7f05b40ffd60 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4105e10 0x7f05b419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4105e10 0x7f05b419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58450/0 (socket says 172.21.15.45:58450) 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 -- 172.21.15.45:0/524623471 learned_addr learned my addr 172.21.15.45:0/524623471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05b9870700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b40ffd60 0x7f05b4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05b906f700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b4100720 0x7f05b41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:12.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 -- 172.21.15.45:0/524623471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b40ffd60 msgr2=0x7f05b4199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b40ffd60 0x7f05b4199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 -- 172.21.15.45:0/524623471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b4100720 msgr2=0x7f05b41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b4100720 0x7f05b41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05ba071700 1 -- 172.21.15.45:0/524623471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009d00 con 0x7f05b4105e10 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.735+0000 7f05b9870700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b40ffd60 0x7f05b4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.736+0000 7f05ba071700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4105e10 0x7f05b419dad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0062a0 tx=0x7f05ac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.736+0000 7f05aaffd700 1 -- 172.21.15.45:0/524623471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00e040 con 0x7f05b4105e10 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.736+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b419e690 con 0x7f05b4105e10 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.736+0000 7f05aaffd700 1 -- 172.21.15.45:0/524623471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05ac004e20 con 0x7f05b4105e10 2023-09-23T09:27:12.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.736+0000 7f05aaffd700 1 -- 172.21.15.45:0/524623471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac01b450 con 0x7f05b4105e10 2023-09-23T09:27:12.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.736+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b41a5f50 con 0x7f05b4105e10 2023-09-23T09:27:12.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.737+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b404eb20 con 0x7f05b4105e10 2023-09-23T09:27:12.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.738+0000 7f05aaffd700 1 -- 172.21.15.45:0/524623471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f05ac003a00 con 0x7f05b4105e10 2023-09-23T09:27:12.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.738+0000 7f05aaffd700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f05a00a15a0 0x7f05a00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:12.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.739+0000 7f05b9870700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f05a00a15a0 0x7f05a00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:12.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.739+0000 7f05aaffd700 1 -- 172.21.15.45:0/524623471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05ac0ce000 con 0x7f05b4105e10 2023-09-23T09:27:12.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.739+0000 7f05b9870700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f05a00a15a0 0x7f05a00a3a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05b000afd0 tx=0x7f05b000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:12.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.743+0000 7f05aaffd700 1 -- 172.21.15.45:0/524623471 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f05ac097b20 con 0x7f05b4105e10 2023-09-23T09:27:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.943+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05b410fd00 con 0x7f05b4105e10 2023-09-23T09:27:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.944+0000 7f05aaffd700 1 -- 172.21.15.45:0/524623471 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f05ac09b140 con 0x7f05b4105e10 2023-09-23T09:27:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:12.946 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:09.768315+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f05a00a15a0 msgr2=0x7f05a00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f05a00a15a0 0x7f05a00a3a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05b000afd0 tx=0x7f05b000a380 comp rx=0 tx=0).stop 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4105e10 msgr2=0x7f05b419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4105e10 0x7f05b419dad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0062a0 tx=0x7f05ac00cf00 comp rx=0 tx=0).stop 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 shutdown_connections 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b4105e10 0x7f05b419dad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f05a00a15a0 0x7f05a00a3a60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b40ffd60 0x7f05b4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 --2- 172.21.15.45:0/524623471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b4100720 0x7f05b41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 >> 172.21.15.45:0/524623471 conn(0x7f05b4076e20 msgr2=0x7f05b40688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 shutdown_connections 2023-09-23T09:27:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:12.946+0000 7f05bbad4700 1 -- 172.21.15.45:0/524623471 wait complete. 2023-09-23T09:27:13.304 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:27:13.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-09-23T09:27:13.871 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:27:14.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:13 smithi098 ceph-mon[102386]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:14.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:13 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/524623471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:14.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:13 smithi045 ceph-mon[98627]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:14.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:13 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/524623471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:14.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:13 smithi123 ceph-mon[103577]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:14.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:13 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/524623471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 -- 172.21.15.45:0/1187819557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53000ffd60 msgr2=0x7f53001001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 --2- 172.21.15.45:0/1187819557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53000ffd60 0x7f53001001e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009990 tx=0x7f52f0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 -- 172.21.15.45:0/1187819557 shutdown_connections 2023-09-23T09:27:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 --2- 172.21.15.45:0/1187819557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5300100720 0x7f530010cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 --2- 172.21.15.45:0/1187819557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5300105e10 0x7f53001061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 --2- 172.21.15.45:0/1187819557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53000ffd60 0x7f53001001e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 -- 172.21.15.45:0/1187819557 >> 172.21.15.45:0/1187819557 conn(0x7f5300076e20 msgr2=0x7f5300077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 -- 172.21.15.45:0/1187819557 shutdown_connections 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.016+0000 7f5305c18700 1 -- 172.21.15.45:0/1187819557 wait complete. 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 Processor -- start 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 -- start start 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53000ffd60 0x7f5300195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5300100720 0x7f53001963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5300105e10 0x7f530019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530019ae20 con 0x7f5300105e10 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530019af90 con 0x7f5300100720 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f5305c18700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530019b150 con 0x7f53000ffd60 2023-09-23T09:27:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f52fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5300105e10 0x7f530019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f52fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5300105e10 0x7f530019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58460/0 (socket says 172.21.15.45:58460) 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f52fffff700 1 -- 172.21.15.45:0/3217085398 learned_addr learned my addr 172.21.15.45:0/3217085398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.017+0000 7f52feffd700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5300100720 0x7f53001963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52ff7fe700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53000ffd60 0x7f5300195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fffff700 1 -- 172.21.15.45:0/3217085398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53000ffd60 msgr2=0x7f5300195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fffff700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53000ffd60 0x7f5300195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fffff700 1 -- 172.21.15.45:0/3217085398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5300100720 msgr2=0x7f53001963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fffff700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5300100720 0x7f53001963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fffff700 1 -- 172.21.15.45:0/3217085398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00095f0 con 0x7f5300105e10 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fffff700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5300105e10 0x7f530019a810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f52f8007f10 tx=0x7f52f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fcff9700 1 -- 172.21.15.45:0/3217085398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8017070 con 0x7f5300105e10 2023-09-23T09:27:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fcff9700 1 -- 172.21.15.45:0/3217085398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52f800f040 con 0x7f5300105e10 2023-09-23T09:27:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f52fcff9700 1 -- 172.21.15.45:0/3217085398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8005740 con 0x7f5300105e10 2023-09-23T09:27:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530019b430 con 0x7f5300105e10 2023-09-23T09:27:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.018+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53001a2c90 con 0x7f5300105e10 2023-09-23T09:27:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.019+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530004eb20 con 0x7f5300105e10 2023-09-23T09:27:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.020+0000 7f52fcff9700 1 -- 172.21.15.45:0/3217085398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f52f800dc60 con 0x7f5300105e10 2023-09-23T09:27:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.021+0000 7f52fcff9700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f52ec0a1620 0x7f52ec0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.021+0000 7f52ff7fe700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f52ec0a1620 0x7f52ec0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.021+0000 7f52fcff9700 1 -- 172.21.15.45:0/3217085398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f52f80cc1f0 con 0x7f5300105e10 2023-09-23T09:27:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.021+0000 7f52ff7fe700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f52ec0a1620 0x7f52ec0a3ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009bd0 tx=0x7f52e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.025+0000 7f52fcff9700 1 -- 172.21.15.45:0/3217085398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f52f8095be0 con 0x7f5300105e10 2023-09-23T09:27:15.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.223+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f530010fec0 con 0x7f5300105e10 2023-09-23T09:27:15.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.223+0000 7f52fcff9700 1 -- 172.21.15.45:0/3217085398 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52f8099200 con 0x7f5300105e10 2023-09-23T09:27:15.225 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:27:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f52ec0a1620 msgr2=0x7f52ec0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f52ec0a1620 0x7f52ec0a3ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009bd0 tx=0x7f52e8009260 comp rx=0 tx=0).stop 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5300105e10 msgr2=0x7f530019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5300105e10 0x7f530019a810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f52f8007f10 tx=0x7f52f800d3b0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 shutdown_connections 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5300105e10 0x7f530019a810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f52ec0a1620 0x7f52ec0a3ae0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53000ffd60 0x7f5300195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.225+0000 7f5305c18700 1 --2- 172.21.15.45:0/3217085398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5300100720 0x7f53001963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.226+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 >> 172.21.15.45:0/3217085398 conn(0x7f5300076e20 msgr2=0x7f53000fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.226+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 shutdown_connections 2023-09-23T09:27:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:15.226+0000 7f5305c18700 1 -- 172.21.15.45:0/3217085398 wait complete. 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.a on smithi045 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.d on smithi045 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.g on smithi045 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.j on smithi045 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.b on smithi098 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.e on smithi098 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.h on smithi098 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.k on smithi098 2023-09-23T09:27:15.629 INFO:tasks.cephadm:Adding mds.c on smithi123 2023-09-23T09:27:15.630 INFO:tasks.cephadm:Adding mds.f on smithi123 2023-09-23T09:27:15.630 INFO:tasks.cephadm:Adding mds.i on smithi123 2023-09-23T09:27:15.630 INFO:tasks.cephadm:Adding mds.l on smithi123 2023-09-23T09:27:15.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l' 2023-09-23T09:27:16.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:15 smithi098 ceph-mon[102386]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:16.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3217085398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:16.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:15 smithi045 ceph-mon[98627]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:16.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3217085398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:16.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:15 smithi123 ceph-mon[103577]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:16.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3217085398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:16.206 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:27:17.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 -- 172.21.15.123:0/3029667085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ffdb0 msgr2=0x7f90781001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:17.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 --2- 172.21.15.123:0/3029667085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ffdb0 0x7f90781001d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9068009990 tx=0x7f9068009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 -- 172.21.15.123:0/3029667085 shutdown_connections 2023-09-23T09:27:17.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 --2- 172.21.15.123:0/3029667085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9078101950 0x7f9078108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 --2- 172.21.15.123:0/3029667085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ffdb0 0x7f90781001d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 --2- 172.21.15.123:0/3029667085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9078100fb0 0x7f9078101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 -- 172.21.15.123:0/3029667085 >> 172.21.15.123:0/3029667085 conn(0x7f90780fb330 msgr2=0x7f90780fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:17.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 -- 172.21.15.123:0/3029667085 shutdown_connections 2023-09-23T09:27:17.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.487+0000 7f90803c1700 1 -- 172.21.15.123:0/3029667085 wait complete. 2023-09-23T09:27:17.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 Processor -- start 2023-09-23T09:27:17.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 -- start start 2023-09-23T09:27:17.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90780ffdb0 0x7f9078195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:17.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9078100fb0 0x7f9078195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:17.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9078101950 0x7f907819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:17.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907819d630 con 0x7f90780ffdb0 2023-09-23T09:27:17.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907819d770 con 0x7f9078100fb0 2023-09-23T09:27:17.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f90803c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907819d900 con 0x7f9078101950 2023-09-23T09:27:17.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f907e95e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9078101950 0x7f907819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:17.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f907e95e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9078101950 0x7f907819d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48582/0 (socket says 172.21.15.123:48582) 2023-09-23T09:27:17.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.488+0000 7f907e95e700 1 -- 172.21.15.123:0/1895564129 learned_addr learned my addr 172.21.15.123:0/1895564129 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-23T09:27:17.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907d95c700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9078100fb0 0x7f9078195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:17.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907e15d700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90780ffdb0 0x7f9078195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:17.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907e95e700 1 -- 172.21.15.123:0/1895564129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9078100fb0 msgr2=0x7f9078195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:17.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907e95e700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9078100fb0 0x7f9078195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907e95e700 1 -- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90780ffdb0 msgr2=0x7f9078195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:17.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907e95e700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90780ffdb0 0x7f9078195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907e95e700 1 -- 172.21.15.123:0/1895564129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90680095f0 con 0x7f9078101950 2023-09-23T09:27:17.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f907e95e700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9078101950 0x7f907819d060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9070007f10 tx=0x7f907000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:17.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f906f7fe700 1 -- 172.21.15.123:0/1895564129 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907000ddd0 con 0x7f9078101950 2023-09-23T09:27:17.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f906f7fe700 1 -- 172.21.15.123:0/1895564129 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f907000f040 con 0x7f9078101950 2023-09-23T09:27:17.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907819dbe0 con 0x7f9078101950 2023-09-23T09:27:17.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f906f7fe700 1 -- 172.21.15.123:0/1895564129 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070015400 con 0x7f9078101950 2023-09-23T09:27:17.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.489+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907819e0e0 con 0x7f9078101950 2023-09-23T09:27:17.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.490+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907810b5b0 con 0x7f9078101950 2023-09-23T09:27:17.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.491+0000 7f906f7fe700 1 -- 172.21.15.123:0/1895564129 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f9070004000 con 0x7f9078101950 2023-09-23T09:27:17.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.492+0000 7f906f7fe700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f90640a1620 0x7f90640a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:17.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.492+0000 7f906f7fe700 1 -- 172.21.15.123:0/1895564129 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90700ccb40 con 0x7f9078101950 2023-09-23T09:27:17.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.492+0000 7f907e15d700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f90640a1620 0x7f90640a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:17.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.492+0000 7f907e15d700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f90640a1620 0x7f90640a3ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9068005b40 tx=0x7f9068005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:17.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.495+0000 7f906f7fe700 1 -- 172.21.15.123:0/1895564129 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f90700965d0 con 0x7f9078101950 2023-09-23T09:27:17.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.691+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f9078061360 con 0x7f90640a1620 2023-09-23T09:27:17.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.701+0000 7f906f7fe700 1 -- 172.21.15.123:0/1895564129 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f9078061360 con 0x7f90640a1620 2023-09-23T09:27:17.703 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mds.all update... 2023-09-23T09:27:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f90640a1620 msgr2=0x7f90640a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:17.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f90640a1620 0x7f90640a3ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9068005b40 tx=0x7f9068005ad0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9078101950 msgr2=0x7f907819d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:17.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9078101950 0x7f907819d060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9070007f10 tx=0x7f907000d3b0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 shutdown_connections 2023-09-23T09:27:17.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f90640a1620 0x7f90640a3ae0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9078101950 0x7f907819d060 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90780ffdb0 0x7f9078195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 --2- 172.21.15.123:0/1895564129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9078100fb0 0x7f9078195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:17.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 >> 172.21.15.123:0/1895564129 conn(0x7f90780fb330 msgr2=0x7f90780fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:17.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 shutdown_connections 2023-09-23T09:27:17.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-23T09:27:17.703+0000 7f90803c1700 1 -- 172.21.15.123:0/1895564129 wait complete. 2023-09-23T09:27:18.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:18.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:18.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:18.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:18.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:18.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:18.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:18.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:18.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:17 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:18.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:18.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:18.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:18.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:17 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:18.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:18.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:18.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:18.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:18.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:18.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:18.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:18.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:18.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:17 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:18.163 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.a.service 2023-09-23T09:27:18.166 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.d.service 2023-09-23T09:27:18.204 INFO:journalctl@ceph.mds.a.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:27:18.206 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.g.service 2023-09-23T09:27:18.235 INFO:journalctl@ceph.mds.d.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:27:18.236 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.j.service 2023-09-23T09:27:18.265 INFO:journalctl@ceph.mds.g.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:27:18.266 DEBUG:teuthology.orchestra.run.smithi098:mds.b> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.b.service 2023-09-23T09:27:18.269 DEBUG:teuthology.orchestra.run.smithi098:mds.e> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.e.service 2023-09-23T09:27:18.293 INFO:journalctl@ceph.mds.j.smithi045.stdout:-- Logs begin at Sat 2023-09-23 09:07:19 UTC. -- 2023-09-23T09:27:18.306 INFO:journalctl@ceph.mds.b.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:27:18.307 DEBUG:teuthology.orchestra.run.smithi098:mds.h> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.h.service 2023-09-23T09:27:18.337 INFO:journalctl@ceph.mds.e.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:27:18.338 DEBUG:teuthology.orchestra.run.smithi098:mds.k> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.k.service 2023-09-23T09:27:18.365 INFO:journalctl@ceph.mds.h.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:27:18.366 DEBUG:teuthology.orchestra.run.smithi123:mds.c> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.c.service 2023-09-23T09:27:18.369 DEBUG:teuthology.orchestra.run.smithi123:mds.f> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.f.service 2023-09-23T09:27:18.392 INFO:journalctl@ceph.mds.k.smithi098.stdout:-- Logs begin at Sat 2023-09-23 09:07:24 UTC. -- 2023-09-23T09:27:18.397 INFO:journalctl@ceph.mds.c.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:27:18.398 DEBUG:teuthology.orchestra.run.smithi123:mds.i> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.i.service 2023-09-23T09:27:18.424 INFO:journalctl@ceph.mds.f.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:27:18.426 DEBUG:teuthology.orchestra.run.smithi123:mds.l> sudo journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.l.service 2023-09-23T09:27:18.465 INFO:journalctl@ceph.mds.i.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:27:18.467 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-23T09:27:18.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:27:18.492 INFO:journalctl@ceph.mds.l.smithi123.stdout:-- Logs begin at Sat 2023-09-23 09:07:29 UTC. -- 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 -- 172.21.15.45:0/2693230958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c101880 msgr2=0x7f5a4c103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/2693230958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c101880 0x7f5a4c103c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009990 tx=0x7f5a34009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 -- 172.21.15.45:0/2693230958 shutdown_connections 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/2693230958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c106ae0 0x7f5a4c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/2693230958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c101880 0x7f5a4c103c70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/2693230958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4c1041b0 0x7f5a4c1065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 -- 172.21.15.45:0/2693230958 >> 172.21.15.45:0/2693230958 conn(0x7f5a4c0fb250 msgr2=0x7f5a4c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 -- 172.21.15.45:0/2693230958 shutdown_connections 2023-09-23T09:27:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.612+0000 7f5a51b2b700 1 -- 172.21.15.45:0/2693230958 wait complete. 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 Processor -- start 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 -- start start 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c101880 0x7f5a4c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c1041b0 0x7f5a4c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4c106ae0 0x7f5a4c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c19e640 con 0x7f5a4c1041b0 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c19e780 con 0x7f5a4c101880 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a51b2b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c19e910 con 0x7f5a4c106ae0 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a4affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c1041b0 0x7f5a4c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a4affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c1041b0 0x7f5a4c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58472/0 (socket says 172.21.15.45:58472) 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a4affd700 1 -- 172.21.15.45:0/4034944785 learned_addr learned my addr 172.21.15.45:0/4034944785 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a4bfff700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4c106ae0 0x7f5a4c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.613+0000 7f5a4b7fe700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c101880 0x7f5a4c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a4affd700 1 -- 172.21.15.45:0/4034944785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4c106ae0 msgr2=0x7f5a4c19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a4affd700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4c106ae0 0x7f5a4c19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a4affd700 1 -- 172.21.15.45:0/4034944785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c101880 msgr2=0x7f5a4c196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a4affd700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c101880 0x7f5a4c196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a4affd700 1 -- 172.21.15.45:0/4034944785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a340095f0 con 0x7f5a4c1041b0 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a4affd700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c1041b0 0x7f5a4c197000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00e9a0 tx=0x7f5a3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a48ff9700 1 -- 172.21.15.45:0/4034944785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c009790 con 0x7f5a4c1041b0 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c19ebf0 con 0x7f5a4c1041b0 2023-09-23T09:27:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a48ff9700 1 -- 172.21.15.45:0/4034944785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a3c010430 con 0x7f5a4c1041b0 2023-09-23T09:27:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a48ff9700 1 -- 172.21.15.45:0/4034944785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c018680 con 0x7f5a4c1041b0 2023-09-23T09:27:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.614+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c19f150 con 0x7f5a4c1041b0 2023-09-23T09:27:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.615+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c10c630 con 0x7f5a4c1041b0 2023-09-23T09:27:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.616+0000 7f5a48ff9700 1 -- 172.21.15.45:0/4034944785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f5a3c0105a0 con 0x7f5a4c1041b0 2023-09-23T09:27:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.617+0000 7f5a48ff9700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5a380a1670 0x7f5a380a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.617+0000 7f5a48ff9700 1 -- 172.21.15.45:0/4034944785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a3c014070 con 0x7f5a4c1041b0 2023-09-23T09:27:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.617+0000 7f5a4b7fe700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5a380a1670 0x7f5a380a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.617+0000 7f5a4b7fe700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5a380a1670 0x7f5a380a3b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400b2e0 tx=0x7f5a34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.621+0000 7f5a48ff9700 1 -- 172.21.15.45:0/4034944785 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5a3c096870 con 0x7f5a4c1041b0 2023-09-23T09:27:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.830+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a4c19f2e0 con 0x7f5a4c1041b0 2023-09-23T09:27:18.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.831+0000 7f5a48ff9700 1 -- 172.21.15.45:0/4034944785 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f5a3c099e90 con 0x7f5a4c1041b0 2023-09-23T09:27:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:18.833 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:09.768315+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:27:18.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5a380a1670 msgr2=0x7f5a380a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5a380a1670 0x7f5a380a3b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400b2e0 tx=0x7f5a34005fb0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c1041b0 msgr2=0x7f5a4c197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c1041b0 0x7f5a4c197000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00e9a0 tx=0x7f5a3c00ecb0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 shutdown_connections 2023-09-23T09:27:18.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5a380a1670 0x7f5a380a3b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4c106ae0 0x7f5a4c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c101880 0x7f5a4c196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 --2- 172.21.15.45:0/4034944785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4c1041b0 0x7f5a4c197000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 >> 172.21.15.45:0/4034944785 conn(0x7f5a4c0fb250 msgr2=0x7f5a4c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:18.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 shutdown_connections 2023-09-23T09:27:18.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.833+0000 7f5a51b2b700 1 -- 172.21.15.45:0/4034944785 wait complete. 2023-09-23T09:27:18.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:27:18.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:27:18.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.975+0000 7f3e9f751700 1 -- 172.21.15.45:0/960931976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98101fa0 msgr2=0x7f3e98102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.975+0000 7f3e9f751700 1 --2- 172.21.15.45:0/960931976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98101fa0 0x7f3e98102420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009990 tx=0x7f3e94009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.975+0000 7f3e9f751700 1 -- 172.21.15.45:0/960931976 shutdown_connections 2023-09-23T09:27:18.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.975+0000 7f3e9f751700 1 --2- 172.21.15.45:0/960931976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e98102960 0x7f3e98109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.975+0000 7f3e9f751700 1 --2- 172.21.15.45:0/960931976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98100e50 0x7f3e98101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.975+0000 7f3e9f751700 1 --2- 172.21.15.45:0/960931976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98101fa0 0x7f3e98102420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.975+0000 7f3e9f751700 1 -- 172.21.15.45:0/960931976 >> 172.21.15.45:0/960931976 conn(0x7f3e980fc3a0 msgr2=0x7f3e980fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:18.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 -- 172.21.15.45:0/960931976 shutdown_connections 2023-09-23T09:27:18.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 -- 172.21.15.45:0/960931976 wait complete. 2023-09-23T09:27:18.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 Processor -- start 2023-09-23T09:27:18.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 -- start start 2023-09-23T09:27:18.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98100e50 0x7f3e98196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98101fa0 0x7f3e98197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e98102960 0x7f3e9819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9819e680 con 0x7f3e98100e50 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9819e7c0 con 0x7f3e98102960 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9f751700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9819e950 con 0x7f3e98101fa0 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9d4ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98100e50 0x7f3e98196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9ccec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98101fa0 0x7f3e98197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9d4ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98100e50 0x7f3e98196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58484/0 (socket says 172.21.15.45:58484) 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.976+0000 7f3e9d4ed700 1 -- 172.21.15.45:0/93958168 learned_addr learned my addr 172.21.15.45:0/93958168 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9dcee700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e98102960 0x7f3e9819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9d4ed700 1 -- 172.21.15.45:0/93958168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98101fa0 msgr2=0x7f3e98197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9d4ed700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98101fa0 0x7f3e98197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9d4ed700 1 -- 172.21.15.45:0/93958168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e98102960 msgr2=0x7f3e9819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9d4ed700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e98102960 0x7f3e9819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:18.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9d4ed700 1 -- 172.21.15.45:0/93958168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940095f0 con 0x7f3e98100e50 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9ccec700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98101fa0 0x7f3e98197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9dcee700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e98102960 0x7f3e9819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9d4ed700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98100e50 0x7f3e98196b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800e9a0 tx=0x7f3e8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e8e7fc700 1 -- 172.21.15.45:0/93958168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88009790 con 0x7f3e98100e50 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9819ec30 con 0x7f3e98100e50 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e8e7fc700 1 -- 172.21.15.45:0/93958168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e8800cbd0 con 0x7f3e98100e50 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e8e7fc700 1 -- 172.21.15.45:0/93958168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88018680 con 0x7f3e98100e50 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.977+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9819f140 con 0x7f3e98100e50 2023-09-23T09:27:18.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.978+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9810c650 con 0x7f3e98100e50 2023-09-23T09:27:18.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.979+0000 7f3e8e7fc700 1 -- 172.21.15.45:0/93958168 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f3e8800cd40 con 0x7f3e98100e50 2023-09-23T09:27:18.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.980+0000 7f3e8e7fc700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3e840a15a0 0x7f3e840a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:18.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.980+0000 7f3e9ccec700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3e840a15a0 0x7f3e840a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:18.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.980+0000 7f3e8e7fc700 1 -- 172.21.15.45:0/93958168 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3e88014070 con 0x7f3e98100e50 2023-09-23T09:27:18.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.980+0000 7f3e9ccec700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3e840a15a0 0x7f3e840a3a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400b2e0 tx=0x7f3e94005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:18.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:18.983+0000 7f3e8e7fc700 1 -- 172.21.15.45:0/93958168 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3e880969f0 con 0x7f3e98100e50 2023-09-23T09:27:19.038 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:18 smithi123 ceph-mon[103577]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:19.038 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:18 smithi123 ceph-mon[103577]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-23T09:27:19.038 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:18 smithi123 ceph-mon[103577]: Deploying daemon mds.l on smithi123 2023-09-23T09:27:19.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:18 smithi098 ceph-mon[102386]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:19.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:18 smithi098 ceph-mon[102386]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-23T09:27:19.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:18 smithi098 ceph-mon[102386]: Deploying daemon mds.l on smithi123 2023-09-23T09:27:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:18 smithi045 ceph-mon[98627]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:18 smithi045 ceph-mon[98627]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-23T09:27:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:18 smithi045 ceph-mon[98627]: Deploying daemon mds.l on smithi123 2023-09-23T09:27:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.186+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e98107d40 con 0x7f3e98100e50 2023-09-23T09:27:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.187+0000 7f3e8e7fc700 1 -- 172.21.15.45:0/93958168 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e8809a010 con 0x7f3e98100e50 2023-09-23T09:27:19.189 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:27:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3e840a15a0 msgr2=0x7f3e840a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3e840a15a0 0x7f3e840a3a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400b2e0 tx=0x7f3e94005960 comp rx=0 tx=0).stop 2023-09-23T09:27:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98100e50 msgr2=0x7f3e98196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98100e50 0x7f3e98196b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800e9a0 tx=0x7f3e8800ecb0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 shutdown_connections 2023-09-23T09:27:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e98102960 0x7f3e9819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e98100e50 0x7f3e98196b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3e840a15a0 0x7f3e840a3a60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 --2- 172.21.15.45:0/93958168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98101fa0 0x7f3e98197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 >> 172.21.15.45:0/93958168 conn(0x7f3e980fc3a0 msgr2=0x7f3e980fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 shutdown_connections 2023-09-23T09:27:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.189+0000 7f3e9f751700 1 -- 172.21.15.45:0/93958168 wait complete. 2023-09-23T09:27:19.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:19.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 -- 172.21.15.45:0/2248554590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28100910 msgr2=0x7fda281081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:19.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 --2- 172.21.15.45:0/2248554590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28100910 0x7fda281081b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b0a0 tx=0x7fda2000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 -- 172.21.15.45:0/2248554590 shutdown_connections 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 --2- 172.21.15.45:0/2248554590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28100910 0x7fda281081b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 --2- 172.21.15.45:0/2248554590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda28101390 0x7fda281017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 --2- 172.21.15.45:0/2248554590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280fff50 0x7fda281003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 -- 172.21.15.45:0/2248554590 >> 172.21.15.45:0/2248554590 conn(0x7fda280fa9e0 msgr2=0x7fda280fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 -- 172.21.15.45:0/2248554590 shutdown_connections 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.327+0000 7fda2e24e700 1 -- 172.21.15.45:0/2248554590 wait complete. 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 Processor -- start 2023-09-23T09:27:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 -- start start 2023-09-23T09:27:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda280fff50 0x7fda280ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda28100910 0x7fda2819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28101390 0x7fda2819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2819e710 con 0x7fda28101390 2023-09-23T09:27:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2819e880 con 0x7fda280fff50 2023-09-23T09:27:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda2e24e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2819ea40 con 0x7fda28100910 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda27fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28101390 0x7fda2819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda26ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda28100910 0x7fda2819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda26ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda28100910 0x7fda2819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:60606/0 (socket says 172.21.15.45:60606) 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda27fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28101390 0x7fda2819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58502/0 (socket says 172.21.15.45:58502) 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda26ffd700 1 -- 172.21.15.45:0/200240529 learned_addr learned my addr 172.21.15.45:0/200240529 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.328+0000 7fda277fe700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda280fff50 0x7fda280ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda27fff700 1 -- 172.21.15.45:0/200240529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda28100910 msgr2=0x7fda2819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda27fff700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda28100910 0x7fda2819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda27fff700 1 -- 172.21.15.45:0/200240529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda280fff50 msgr2=0x7fda280ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda27fff700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda280fff50 0x7fda280ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda27fff700 1 -- 172.21.15.45:0/200240529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20009d00 con 0x7fda28101390 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda26ffd700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda28100910 0x7fda2819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda27fff700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28101390 0x7fda2819df40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda20009fd0 tx=0x7fda20007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda24ff9700 1 -- 172.21.15.45:0/200240529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2000e040 con 0x7fda28101390 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2819ecc0 con 0x7fda28101390 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda24ff9700 1 -- 172.21.15.45:0/200240529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda20004c50 con 0x7fda28101390 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda24ff9700 1 -- 172.21.15.45:0/200240529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2001b6c0 con 0x7fda28101390 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.329+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2819f0e0 con 0x7fda28101390 2023-09-23T09:27:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.330+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda28106c00 con 0x7fda28101390 2023-09-23T09:27:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.331+0000 7fda24ff9700 1 -- 172.21.15.45:0/200240529 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fda2001d470 con 0x7fda28101390 2023-09-23T09:27:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.332+0000 7fda24ff9700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fda100a15f0 0x7fda100a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.332+0000 7fda277fe700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fda100a15f0 0x7fda100a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.332+0000 7fda24ff9700 1 -- 172.21.15.45:0/200240529 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fda200ce010 con 0x7fda28101390 2023-09-23T09:27:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.332+0000 7fda277fe700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fda100a15f0 0x7fda100a3ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda18009c60 tx=0x7fda18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.335+0000 7fda24ff9700 1 -- 172.21.15.45:0/200240529 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fda20097b30 con 0x7fda28101390 2023-09-23T09:27:19.338 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:27:19 smithi123 systemd[1]: Starting Ceph mds.l for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:19.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.584+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda2802d2a0 con 0x7fda28101390 2023-09-23T09:27:19.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.584+0000 7fda24ff9700 1 -- 172.21.15.45:0/200240529 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fda2009b150 con 0x7fda28101390 2023-09-23T09:27:19.586 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:19.587 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-23T09:27:19.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fda100a15f0 msgr2=0x7fda100a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fda100a15f0 0x7fda100a3ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda18009c60 tx=0x7fda18009260 comp rx=0 tx=0).stop 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28101390 msgr2=0x7fda2819df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28101390 0x7fda2819df40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda20009fd0 tx=0x7fda20007930 comp rx=0 tx=0).stop 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 shutdown_connections 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda28101390 0x7fda2819df40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda280fff50 0x7fda280ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fda100a15f0 0x7fda100a3ab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 --2- 172.21.15.45:0/200240529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda28100910 0x7fda2819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 >> 172.21.15.45:0/200240529 conn(0x7fda280fa9e0 msgr2=0x7fda280fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:19.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 shutdown_connections 2023-09-23T09:27:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:19.587+0000 7fda2e24e700 1 -- 172.21.15.45:0/200240529 wait complete. 2023-09-23T09:27:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2023-09-23T09:27:19.814 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:19 smithi123 ceph-mon[103577]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:19.814 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:19 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/4034944785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:19.814 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:19 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/93958168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:19.814 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:19 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/200240529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:19.814 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:27:19 smithi123 podman[138829]: 2023-09-23T09:27:19.815 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:27:19 smithi123 ceph-mds[138869]: -- 172.21.15.123:0/3970932028 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cf0da351e0 con 0x55cf0da0ac00 2023-09-23T09:27:19.815 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:27:19 smithi123 bash[138829]: 34e967a2318292b684937a5fbe349cc7ece55c99d6a4351132e7de55ea1c87c9 2023-09-23T09:27:20.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:19 smithi098 ceph-mon[102386]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:20.095 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:19 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/4034944785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:20.095 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:19 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/93958168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:20.095 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:19 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/200240529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:20.096 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:27:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-l[138845]: starting mds.l at 2023-09-23T09:27:20.096 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:27:19 smithi123 systemd[1]: Started Ceph mds.l for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:20.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:19 smithi045 ceph-mon[98627]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:20.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:19 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/4034944785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:20.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:19 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/93958168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:20.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:19 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/200240529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:21.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6832/782227022,v1:172.21.15.123:6833/782227022] up:boot 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: fsmap 1 up:standby 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:21.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:21.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:20 smithi123 ceph-mon[103577]: Deploying daemon mds.k on smithi098 2023-09-23T09:27:21.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6832/782227022,v1:172.21.15.123:6833/782227022] up:boot 2023-09-23T09:27:21.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: fsmap 1 up:standby 2023-09-23T09:27:21.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T09:27:21.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:21.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:21.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:21.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:20 smithi098 ceph-mon[102386]: Deploying daemon mds.k on smithi098 2023-09-23T09:27:21.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6832/782227022,v1:172.21.15.123:6833/782227022] up:boot 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: fsmap 1 up:standby 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:21.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:20 smithi045 ceph-mon[98627]: Deploying daemon mds.k on smithi098 2023-09-23T09:27:21.601 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:21.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 -- 172.21.15.45:0/1542430551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98101f70 msgr2=0x7efd981023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:21.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 --2- 172.21.15.45:0/1542430551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98101f70 0x7efd981023f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009990 tx=0x7efd8c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:21.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 -- 172.21.15.45:0/1542430551 shutdown_connections 2023-09-23T09:27:21.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 --2- 172.21.15.45:0/1542430551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd98102930 0x7efd98109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:21.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 --2- 172.21.15.45:0/1542430551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd98100e20 0x7efd98101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:21.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 --2- 172.21.15.45:0/1542430551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98101f70 0x7efd981023f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 -- 172.21.15.45:0/1542430551 >> 172.21.15.45:0/1542430551 conn(0x7efd980fc370 msgr2=0x7efd980fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 -- 172.21.15.45:0/1542430551 shutdown_connections 2023-09-23T09:27:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.740+0000 7efd9f356700 1 -- 172.21.15.45:0/1542430551 wait complete. 2023-09-23T09:27:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.741+0000 7efd9f356700 1 Processor -- start 2023-09-23T09:27:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.741+0000 7efd9f356700 1 -- start start 2023-09-23T09:27:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98100e20 0x7efd98196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd98101f70 0x7efd98196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd98102930 0x7efd9819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9819e5f0 con 0x7efd98100e20 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9819e730 con 0x7efd98102930 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9819e8c0 con 0x7efd98101f70 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98100e20 0x7efd98196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98100e20 0x7efd98196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58512/0 (socket says 172.21.15.45:58512) 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 -- 172.21.15.45:0/4119272432 learned_addr learned my addr 172.21.15.45:0/4119272432 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d8f3700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd98102930 0x7efd9819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9c8f1700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd98101f70 0x7efd98196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 -- 172.21.15.45:0/4119272432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd98101f70 msgr2=0x7efd98196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd98101f70 0x7efd98196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 -- 172.21.15.45:0/4119272432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd98102930 msgr2=0x7efd9819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd98102930 0x7efd9819e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 -- 172.21.15.45:0/4119272432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0095f0 con 0x7efd98100e20 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9d0f2700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98100e20 0x7efd98196a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9a0 tx=0x7efd9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd8a7fc700 1 -- 172.21.15.45:0/4119272432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94009790 con 0x7efd98100e20 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd8a7fc700 1 -- 172.21.15.45:0/4119272432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd94010430 con 0x7efd98100e20 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9819eba0 con 0x7efd98100e20 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd8a7fc700 1 -- 172.21.15.45:0/4119272432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94018680 con 0x7efd98100e20 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.742+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9819f100 con 0x7efd98100e20 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.743+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd981083b0 con 0x7efd98100e20 2023-09-23T09:27:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.744+0000 7efd8a7fc700 1 -- 172.21.15.45:0/4119272432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7efd940105a0 con 0x7efd98100e20 2023-09-23T09:27:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.745+0000 7efd8a7fc700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efd840a1670 0x7efd840a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.745+0000 7efd9c8f1700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efd840a1670 0x7efd840a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.745+0000 7efd8a7fc700 1 -- 172.21.15.45:0/4119272432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efd94014070 con 0x7efd98100e20 2023-09-23T09:27:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.746+0000 7efd9c8f1700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efd840a1670 0x7efd840a3b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b2e0 tx=0x7efd8c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.749+0000 7efd8a7fc700 1 -- 172.21.15.45:0/4119272432 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7efd94096f70 con 0x7efd98100e20 2023-09-23T09:27:21.794 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:27:21 smithi098 systemd[1]: Starting Ceph mds.k for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.999+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd9810c8d0 con 0x7efd98100e20 2023-09-23T09:27:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:21.999+0000 7efd8a7fc700 1 -- 172.21.15.45:0/4119272432 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7efd9409a590 con 0x7efd98100e20 2023-09-23T09:27:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:22.001 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-23T09:27:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.001+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efd840a1670 msgr2=0x7efd840a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.001+0000 7efd9f356700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efd840a1670 0x7efd840a3b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b2e0 tx=0x7efd8c005960 comp rx=0 tx=0).stop 2023-09-23T09:27:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.001+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98100e20 msgr2=0x7efd98196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.001+0000 7efd9f356700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98100e20 0x7efd98196a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9a0 tx=0x7efd9400ecb0 comp rx=0 tx=0).stop 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 shutdown_connections 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd98102930 0x7efd9819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd98100e20 0x7efd98196a70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efd840a1670 0x7efd840a3b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 --2- 172.21.15.45:0/4119272432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd98101f70 0x7efd98196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 >> 172.21.15.45:0/4119272432 conn(0x7efd980fc370 msgr2=0x7efd981054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 shutdown_connections 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:22.002+0000 7efd9f356700 1 -- 172.21.15.45:0/4119272432 wait complete. 2023-09-23T09:27:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2023-09-23T09:27:22.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:21 smithi045 ceph-mon[98627]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:22.084 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:21 smithi098 ceph-mon[102386]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:22.084 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:27:21 smithi098 podman[136463]: 2023-09-23T09:27:22.084 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mds[136503]: -- 172.21.15.98:0/3475061861 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5641df4d71e0 con 0x5641df4ac800 2023-09-23T09:27:22.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:21 smithi123 ceph-mon[103577]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:22.544 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-k[136479]: starting mds.k at 2023-09-23T09:27:22.544 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:27:22 smithi098 bash[136463]: 0e5bf4c0319df7aea5616886db0f83407979a698bbb528eab1c4b227259953b5 2023-09-23T09:27:22.544 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:27:22 smithi098 systemd[1]: Started Ceph mds.k for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:23.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:22 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/4119272432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:23.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:23.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:23.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:22 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:23.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:22 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/4119272432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:23.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:23.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:23.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:22 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:23.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/4119272432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:23.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:23.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:23.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:23.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:22 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:23.830 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:27:23 smithi045 systemd[1]: Starting Ceph mds.j for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:24.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:24.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:23 smithi045 ceph-mon[98627]: Deploying daemon mds.j on smithi045 2023-09-23T09:27:24.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:23 smithi045 ceph-mon[98627]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:24.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:23 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6832/3376968618,v1:172.21.15.98:6833/3376968618] up:boot 2023-09-23T09:27:24.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:23 smithi045 ceph-mon[98627]: fsmap 2 up:standby 2023-09-23T09:27:24.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:23 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T09:27:24.109 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:27:23 smithi045 podman[150346]: 2023-09-23T09:27:24.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 -- 172.21.15.45:0/134580451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 msgr2=0x7fdca8101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 --2- 172.21.15.45:0/134580451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8101240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009990 tx=0x7fdc98009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 -- 172.21.15.45:0/134580451 shutdown_connections 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 --2- 172.21.15.45:0/134580451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca8102930 0x7fdca8109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 --2- 172.21.15.45:0/134580451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8101240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 --2- 172.21.15.45:0/134580451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdca8101f70 0x7fdca81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 -- 172.21.15.45:0/134580451 >> 172.21.15.45:0/134580451 conn(0x7fdca80fc370 msgr2=0x7fdca80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 -- 172.21.15.45:0/134580451 shutdown_connections 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 -- 172.21.15.45:0/134580451 wait complete. 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 Processor -- start 2023-09-23T09:27:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.150+0000 7fdcae174700 1 -- start start 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca8101f70 0x7fdca8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdca8102930 0x7fdca819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819e640 con 0x7fdca8100e20 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819e780 con 0x7fdca8101f70 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819e910 con 0x7fdca8102930 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58536/0 (socket says 172.21.15.45:58536) 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 -- 172.21.15.45:0/1194401576 learned_addr learned my addr 172.21.15.45:0/1194401576 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca7fff700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdca8102930 0x7fdca819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca6ffd700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca8101f70 0x7fdca8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 -- 172.21.15.45:0/1194401576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdca8102930 msgr2=0x7fdca819e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdca8102930 0x7fdca819e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 -- 172.21.15.45:0/1194401576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca8101f70 msgr2=0x7fdca8197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca8101f70 0x7fdca8197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 -- 172.21.15.45:0/1194401576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980095f0 con 0x7fdca8100e20 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca77fe700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8196ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc98000c00 tx=0x7fdc9800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca4ff9700 1 -- 172.21.15.45:0/1194401576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9801d070 con 0x7fdca8100e20 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca4ff9700 1 -- 172.21.15.45:0/1194401576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc9800fc60 con 0x7fdca8100e20 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca819eb90 con 0x7fdca8100e20 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdca4ff9700 1 -- 172.21.15.45:0/1194401576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98017650 con 0x7fdca8100e20 2023-09-23T09:27:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.151+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca819efb0 con 0x7fdca8100e20 2023-09-23T09:27:24.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.152+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8108400 con 0x7fdca8100e20 2023-09-23T09:27:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.153+0000 7fdca4ff9700 1 -- 172.21.15.45:0/1194401576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fdc980177b0 con 0x7fdca8100e20 2023-09-23T09:27:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.154+0000 7fdca4ff9700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdc900a15f0 0x7fdc900a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.154+0000 7fdca6ffd700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdc900a15f0 0x7fdc900a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.154+0000 7fdca4ff9700 1 -- 172.21.15.45:0/1194401576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdc980ce0f0 con 0x7fdca8100e20 2023-09-23T09:27:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.155+0000 7fdca6ffd700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdc900a15f0 0x7fdc900a3ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c005950 tx=0x7fdc9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.158+0000 7fdca4ff9700 1 -- 172.21.15.45:0/1194401576 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fdc98097d30 con 0x7fdca8100e20 2023-09-23T09:27:24.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:23 smithi098 ceph-mon[102386]: Deploying daemon mds.j on smithi045 2023-09-23T09:27:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:23 smithi098 ceph-mon[102386]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:23 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6832/3376968618,v1:172.21.15.98:6833/3376968618] up:boot 2023-09-23T09:27:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:23 smithi098 ceph-mon[102386]: fsmap 2 up:standby 2023-09-23T09:27:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:23 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T09:27:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:23 smithi123 ceph-mon[103577]: Deploying daemon mds.j on smithi045 2023-09-23T09:27:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:23 smithi123 ceph-mon[103577]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:23 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6832/3376968618,v1:172.21.15.98:6833/3376968618] up:boot 2023-09-23T09:27:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:23 smithi123 ceph-mon[103577]: fsmap 2 up:standby 2023-09-23T09:27:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:23 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T09:27:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.435+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdca810c870 con 0x7fdca8100e20 2023-09-23T09:27:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.436+0000 7fdca4ff9700 1 -- 172.21.15.45:0/1194401576 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7fdc98027860 con 0x7fdca8100e20 2023-09-23T09:27:24.439 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:24.439 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-23T09:27:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdc900a15f0 msgr2=0x7fdc900a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdc900a15f0 0x7fdc900a3ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c005950 tx=0x7fdc9c0058e0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 msgr2=0x7fdca8196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8196ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc98000c00 tx=0x7fdc9800f510 comp rx=0 tx=0).stop 2023-09-23T09:27:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 shutdown_connections 2023-09-23T09:27:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fdc900a15f0 0x7fdc900a3ab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdca8102930 0x7fdca819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca8100e20 0x7fdca8196ac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 --2- 172.21.15.45:0/1194401576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdca8101f70 0x7fdca8197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 >> 172.21.15.45:0/1194401576 conn(0x7fdca80fc370 msgr2=0x7fdca8105540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 shutdown_connections 2023-09-23T09:27:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:24.439+0000 7fdcae174700 1 -- 172.21.15.45:0/1194401576 wait complete. 2023-09-23T09:27:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2023-09-23T09:27:24.453 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:27:24 smithi045 bash[150346]: f67b3af3e858c6ecbb61304841ae12fa7a9b8d7d09e3c6d93f3743260e7be47e 2023-09-23T09:27:24.453 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:27:24 smithi045 ceph-mds[150409]: -- 172.21.15.45:0/3926050805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560cfd9831e0 con 0x560cfd958800 2023-09-23T09:27:24.453 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:27:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-j[150362]: starting mds.j at 2023-09-23T09:27:24.453 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:27:24 smithi045 systemd[1]: Started Ceph mds.j for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:25.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: Deploying daemon mds.d on smithi045 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1194401576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.45:6834/427947541,v1:172.21.15.45:6835/427947541] up:boot 2023-09-23T09:27:25.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: fsmap 3 up:standby 2023-09-23T09:27:25.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:25 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T09:27:25.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:25.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: Deploying daemon mds.d on smithi045 2023-09-23T09:27:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1194401576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.45:6834/427947541,v1:172.21.15.45:6835/427947541] up:boot 2023-09-23T09:27:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: fsmap 3 up:standby 2023-09-23T09:27:25.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:25 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T09:27:25.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: Deploying daemon mds.d on smithi045 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1194401576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.45:6834/427947541,v1:172.21.15.45:6835/427947541] up:boot 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: fsmap 3 up:standby 2023-09-23T09:27:25.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:25 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T09:27:25.663 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:27:25 smithi045 systemd[1]: Starting Ceph mds.d for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:26.252 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:27:25 smithi045 podman[150739]: 2023-09-23T09:27:26.252 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:27:26 smithi045 bash[150739]: 892fae0cafc419ac22cd5451ad378fbb25cf0f9624ee5c6146d0a49a44bb617c 2023-09-23T09:27:26.252 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:27:26 smithi045 ceph-mds[150779]: -- 172.21.15.45:0/151519314 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563da7ae11e0 con 0x563da7ab6800 2023-09-23T09:27:26.252 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:27:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-d[150755]: starting mds.d at 2023-09-23T09:27:26.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:26.580 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:27:26 smithi045 systemd[1]: Started Ceph mds.d for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 -- 172.21.15.45:0/736568684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f78100f10 msgr2=0x7f3f78101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 --2- 172.21.15.45:0/736568684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f78100f10 0x7f3f78101370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f680099c0 tx=0x7f3f68009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 -- 172.21.15.45:0/736568684 shutdown_connections 2023-09-23T09:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 --2- 172.21.15.45:0/736568684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f781018b0 0x7f3f78108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 --2- 172.21.15.45:0/736568684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f780ffd20 0x7f3f78100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 --2- 172.21.15.45:0/736568684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f78100f10 0x7f3f78101370 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 -- 172.21.15.45:0/736568684 >> 172.21.15.45:0/736568684 conn(0x7f3f780fb250 msgr2=0x7f3f780fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:26.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 -- 172.21.15.45:0/736568684 shutdown_connections 2023-09-23T09:27:26.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.585+0000 7f3f7f586700 1 -- 172.21.15.45:0/736568684 wait complete. 2023-09-23T09:27:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 Processor -- start 2023-09-23T09:27:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 -- start start 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f780ffd20 0x7f3f78196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78100f10 0x7f3f78196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f781018b0 0x7f3f7819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7819e5a0 con 0x7f3f780ffd20 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7819e6e0 con 0x7f3f781018b0 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7f586700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7819e870 con 0x7f3f78100f10 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7d322700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f780ffd20 0x7f3f78196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7d322700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f780ffd20 0x7f3f78196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58564/0 (socket says 172.21.15.45:58564) 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7d322700 1 -- 172.21.15.45:0/459812991 learned_addr learned my addr 172.21.15.45:0/459812991 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.586+0000 7f3f7db23700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f781018b0 0x7f3f7819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7cb21700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78100f10 0x7f3f78196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7d322700 1 -- 172.21.15.45:0/459812991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78100f10 msgr2=0x7f3f78196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7d322700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78100f10 0x7f3f78196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7d322700 1 -- 172.21.15.45:0/459812991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f781018b0 msgr2=0x7f3f7819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7d322700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f781018b0 0x7f3f7819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7d322700 1 -- 172.21.15.45:0/459812991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680095f0 con 0x7f3f780ffd20 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7db23700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f781018b0 0x7f3f7819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7d322700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f780ffd20 0x7f3f78196a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400e970 tx=0x7f3f7400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f6e7fc700 1 -- 172.21.15.45:0/459812991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74009790 con 0x7f3f780ffd20 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f6e7fc700 1 -- 172.21.15.45:0/459812991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f740044e0 con 0x7f3f780ffd20 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7819eb50 con 0x7f3f780ffd20 2023-09-23T09:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f6e7fc700 1 -- 172.21.15.45:0/459812991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74003e30 con 0x7f3f780ffd20 2023-09-23T09:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.587+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7819f060 con 0x7f3f780ffd20 2023-09-23T09:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.588+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7810b530 con 0x7f3f780ffd20 2023-09-23T09:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.589+0000 7f3f6e7fc700 1 -- 172.21.15.45:0/459812991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f3f74003f90 con 0x7f3f780ffd20 2023-09-23T09:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.590+0000 7f3f6e7fc700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3f640a15f0 0x7f3f640a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.590+0000 7f3f7cb21700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3f640a15f0 0x7f3f640a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.590+0000 7f3f6e7fc700 1 -- 172.21.15.45:0/459812991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f74014070 con 0x7f3f780ffd20 2023-09-23T09:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.590+0000 7f3f7cb21700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3f640a15f0 0x7f3f640a3ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f68006010 tx=0x7f3f68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:26.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.593+0000 7f3f6e7fc700 1 -- 172.21.15.45:0/459812991 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3f740969b0 con 0x7f3f780ffd20 2023-09-23T09:27:26.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.848+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f7819f420 con 0x7f3f780ffd20 2023-09-23T09:27:26.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.848+0000 7f3f6e7fc700 1 -- 172.21.15.45:0/459812991 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3695 (secure 0 0 0) 0x7f3f74099fd0 con 0x7f3f780ffd20 2023-09-23T09:27:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:26.851 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-23T09:27:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.850+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3f640a15f0 msgr2=0x7f3f640a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.850+0000 7f3f7f586700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3f640a15f0 0x7f3f640a3ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f68006010 tx=0x7f3f68000bc0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.850+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f780ffd20 msgr2=0x7f3f78196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.850+0000 7f3f7f586700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f780ffd20 0x7f3f78196a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400e970 tx=0x7f3f7400ed30 comp rx=0 tx=0).stop 2023-09-23T09:27:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 shutdown_connections 2023-09-23T09:27:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f781018b0 0x7f3f7819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f780ffd20 0x7f3f78196a20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3f640a15f0 0x7f3f640a3ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 --2- 172.21.15.45:0/459812991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78100f10 0x7f3f78196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:26.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 >> 172.21.15.45:0/459812991 conn(0x7f3f780fb250 msgr2=0x7f3f780fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:26.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 shutdown_connections 2023-09-23T09:27:26.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:26.851+0000 7f3f7f586700 1 -- 172.21.15.45:0/459812991 wait complete. 2023-09-23T09:27:26.854 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2023-09-23T09:27:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.45:6836/1158485570,v1:172.21.15.45:6837/1158485570] up:boot 2023-09-23T09:27:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: fsmap 4 up:standby 2023-09-23T09:27:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:27.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: Deploying daemon mds.e on smithi098 2023-09-23T09:27:27.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:27 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/459812991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.45:6836/1158485570,v1:172.21.15.45:6837/1158485570] up:boot 2023-09-23T09:27:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: fsmap 4 up:standby 2023-09-23T09:27:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:27.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: Deploying daemon mds.e on smithi098 2023-09-23T09:27:27.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:27 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/459812991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:27.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.45:6836/1158485570,v1:172.21.15.45:6837/1158485570] up:boot 2023-09-23T09:27:27.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: fsmap 4 up:standby 2023-09-23T09:27:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T09:27:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:27.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:27.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: Deploying daemon mds.e on smithi098 2023-09-23T09:27:27.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:27 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/459812991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:28.044 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:27:27 smithi098 systemd[1]: Starting Ceph mds.e for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:28.360 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:27:28 smithi098 podman[136804]: 2023-09-23T09:27:28.794 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:27:28 smithi098 ceph-mds[136844]: -- 172.21.15.98:0/268688919 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b8e16eb1e0 con 0x55b8e16c1000 2023-09-23T09:27:28.794 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:27:28 smithi098 bash[136804]: 0bd5d52b7302852ca03d04a5634af6a1c54dc839197784627628b3d35882bbe7 2023-09-23T09:27:28.794 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:27:28 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-e[136820]: starting mds.e at 2023-09-23T09:27:28.794 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:27:28 smithi098 systemd[1]: Started Ceph mds.e for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:28.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 -- 172.21.15.45:0/4122675028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc0ffd00 msgr2=0x7f88fc100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 --2- 172.21.15.45:0/4122675028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc0ffd00 0x7f88fc100120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f88f00099c0 tx=0x7f88f0009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 -- 172.21.15.45:0/4122675028 shutdown_connections 2023-09-23T09:27:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 --2- 172.21.15.45:0/4122675028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88fc101890 0x7f88fc108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 --2- 172.21.15.45:0/4122675028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc0ffd00 0x7f88fc100120 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 --2- 172.21.15.45:0/4122675028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88fc100ef0 0x7f88fc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 -- 172.21.15.45:0/4122675028 >> 172.21.15.45:0/4122675028 conn(0x7f88fc0fb250 msgr2=0x7f88fc0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 -- 172.21.15.45:0/4122675028 shutdown_connections 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.005+0000 7f8902b4c700 1 -- 172.21.15.45:0/4122675028 wait complete. 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 Processor -- start 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 -- start start 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88fc0ffd00 0x7f88fc103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc100ef0 0x7f88fc1022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88fc101890 0x7f88fc102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc102fb0 con 0x7f88fc100ef0 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc103120 con 0x7f88fc0ffd00 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f8902b4c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc1a0b20 con 0x7f88fc101890 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc100ef0 0x7f88fc1022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f89008e8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88fc0ffd00 0x7f88fc103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc100ef0 0x7f88fc1022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58580/0 (socket says 172.21.15.45:58580) 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 -- 172.21.15.45:0/1653763778 learned_addr learned my addr 172.21.15.45:0/1653763778 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f89010e9700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88fc101890 0x7f88fc102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 -- 172.21.15.45:0/1653763778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88fc101890 msgr2=0x7f88fc102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88fc101890 0x7f88fc102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 -- 172.21.15.45:0/1653763778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88fc0ffd00 msgr2=0x7f88fc103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88fc0ffd00 0x7f88fc103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f88fbfff700 1 -- 172.21.15.45:0/1653763778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f00095f0 con 0x7f88fc100ef0 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.006+0000 7f89008e8700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88fc0ffd00 0x7f88fc103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.007+0000 7f88fbfff700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc100ef0 0x7f88fc1022d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f88e800d950 tx=0x7f88e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.007+0000 7f88f9ffb700 1 -- 172.21.15.45:0/1653763778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800f890 con 0x7f88fc100ef0 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.007+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc1a0e00 con 0x7f88fc100ef0 2023-09-23T09:27:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.007+0000 7f88f9ffb700 1 -- 172.21.15.45:0/1653763778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88e8010460 con 0x7f88fc100ef0 2023-09-23T09:27:29.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.007+0000 7f88f9ffb700 1 -- 172.21.15.45:0/1653763778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800e610 con 0x7f88fc100ef0 2023-09-23T09:27:29.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.007+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc1a1280 con 0x7f88fc100ef0 2023-09-23T09:27:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.008+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc10b510 con 0x7f88fc100ef0 2023-09-23T09:27:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.009+0000 7f88f9ffb700 1 -- 172.21.15.45:0/1653763778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f88e80108d0 con 0x7f88fc100ef0 2023-09-23T09:27:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.010+0000 7f88f9ffb700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f88ec0a15f0 0x7f88ec0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.010+0000 7f88f9ffb700 1 -- 172.21.15.45:0/1653763778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f88e80ccb70 con 0x7f88fc100ef0 2023-09-23T09:27:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.010+0000 7f89008e8700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f88ec0a15f0 0x7f88ec0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.010+0000 7f89008e8700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f88ec0a15f0 0x7f88ec0a3ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009f50 tx=0x7f88f000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.014+0000 7f88f9ffb700 1 -- 172.21.15.45:0/1653763778 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f88e8096600 con 0x7f88fc100ef0 2023-09-23T09:27:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.260+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88fc1a14a0 con 0x7f88fc100ef0 2023-09-23T09:27:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.260+0000 7f88f9ffb700 1 -- 172.21.15.45:0/1653763778 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4486 (secure 0 0 0) 0x7f88e8099c20 con 0x7f88fc100ef0 2023-09-23T09:27:29.262 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:29.263 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-23T09:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f88ec0a15f0 msgr2=0x7f88ec0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f88ec0a15f0 0x7f88ec0a3ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009f50 tx=0x7f88f000b540 comp rx=0 tx=0).stop 2023-09-23T09:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc100ef0 msgr2=0x7f88fc1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc100ef0 0x7f88fc1022d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f88e800d950 tx=0x7f88e800dc60 comp rx=0 tx=0).stop 2023-09-23T09:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 shutdown_connections 2023-09-23T09:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f88ec0a15f0 0x7f88ec0a3ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88fc101890 0x7f88fc102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88fc0ffd00 0x7f88fc103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 --2- 172.21.15.45:0/1653763778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc100ef0 0x7f88fc1022d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 >> 172.21.15.45:0/1653763778 conn(0x7f88fc0fb250 msgr2=0x7f88fc067f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 shutdown_connections 2023-09-23T09:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:29.263+0000 7f8902b4c700 1 -- 172.21.15.45:0/1653763778 wait complete. 2023-09-23T09:27:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2023-09-23T09:27:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6834/3506316120,v1:172.21.15.98:6835/3506316120] up:boot 2023-09-23T09:27:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: fsmap 5 up:standby 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: Deploying daemon mds.f on smithi123 2023-09-23T09:27:29.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:29 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1653763778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:29.719 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:29.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6834/3506316120,v1:172.21.15.98:6835/3506316120] up:boot 2023-09-23T09:27:29.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: fsmap 5 up:standby 2023-09-23T09:27:29.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:27:29.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:29.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:29.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:29.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: Deploying daemon mds.f on smithi123 2023-09-23T09:27:29.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:29 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1653763778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:29.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:29.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6834/3506316120,v1:172.21.15.98:6835/3506316120] up:boot 2023-09-23T09:27:29.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: fsmap 5 up:standby 2023-09-23T09:27:29.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:27:29.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: Deploying daemon mds.f on smithi123 2023-09-23T09:27:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:29 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1653763778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:29.999 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:27:29 smithi123 systemd[1]: Starting Ceph mds.f for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:30.549 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:27:30 smithi123 podman[139167]: 2023-09-23T09:27:30.838 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:27:30 smithi123 ceph-mds[139207]: -- 172.21.15.123:0/971933039 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55de138011e0 con 0x55de137d6800 2023-09-23T09:27:30.838 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:27:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-f[139183]: starting mds.f at 2023-09-23T09:27:30.839 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:27:30 smithi123 bash[139167]: 184e6a3907de6fbfde3232edd3fde40bc31db4c830b12f022360e0bbb0f98d46 2023-09-23T09:27:30.839 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:27:30 smithi123 systemd[1]: Started Ceph mds.f for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:31.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:31.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.416+0000 7f3833c43700 1 -- 172.21.15.45:0/3916115787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c0ffd00 msgr2=0x7f382c100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:31.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.416+0000 7f3833c43700 1 --2- 172.21.15.45:0/3916115787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c0ffd00 0x7f382c100120 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f381c0099c0 tx=0x7f381c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.416+0000 7f3833c43700 1 -- 172.21.15.45:0/3916115787 shutdown_connections 2023-09-23T09:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.416+0000 7f3833c43700 1 --2- 172.21.15.45:0/3916115787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f382c101890 0x7f382c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.416+0000 7f3833c43700 1 --2- 172.21.15.45:0/3916115787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c0ffd00 0x7f382c100120 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.416+0000 7f3833c43700 1 --2- 172.21.15.45:0/3916115787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f382c100ef0 0x7f382c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.416+0000 7f3833c43700 1 -- 172.21.15.45:0/3916115787 >> 172.21.15.45:0/3916115787 conn(0x7f382c0fb250 msgr2=0x7f382c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 -- 172.21.15.45:0/3916115787 shutdown_connections 2023-09-23T09:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 -- 172.21.15.45:0/3916115787 wait complete. 2023-09-23T09:27:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 Processor -- start 2023-09-23T09:27:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 -- start start 2023-09-23T09:27:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f382c0ffd00 0x7f382c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c100ef0 0x7f382c197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f382c101890 0x7f382c19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.417+0000 7f3833c43700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c19e6c0 con 0x7f382c100ef0 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f3833c43700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c19e800 con 0x7f382c0ffd00 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f3833c43700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c19e990 con 0x7f382c101890 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38311de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c100ef0 0x7f382c197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38321e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f382c101890 0x7f382c19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38311de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c100ef0 0x7f382c197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58592/0 (socket says 172.21.15.45:58592) 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38311de700 1 -- 172.21.15.45:0/1931674347 learned_addr learned my addr 172.21.15.45:0/1931674347 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38319df700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f382c0ffd00 0x7f382c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38321e0700 1 -- 172.21.15.45:0/1931674347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f382c0ffd00 msgr2=0x7f382c196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38321e0700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f382c0ffd00 0x7f382c196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38321e0700 1 -- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c100ef0 msgr2=0x7f382c197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38321e0700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c100ef0 0x7f382c197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38321e0700 1 -- 172.21.15.45:0/1931674347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0095f0 con 0x7f382c101890 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38311de700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c100ef0 0x7f382c197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f38321e0700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f382c101890 0x7f382c19e0f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f382400efc0 tx=0x7f382400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f3822ffd700 1 -- 172.21.15.45:0/1931674347 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824018070 con 0x7f382c101890 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f3822ffd700 1 -- 172.21.15.45:0/1931674347 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f382400f040 con 0x7f382c101890 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.418+0000 7f3822ffd700 1 -- 172.21.15.45:0/1931674347 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382400cd40 con 0x7f382c101890 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.419+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c19ec10 con 0x7f382c101890 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.419+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c19f0d0 con 0x7f382c101890 2023-09-23T09:27:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.419+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c1072e0 con 0x7f382c101890 2023-09-23T09:27:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.421+0000 7f3822ffd700 1 -- 172.21.15.45:0/1931674347 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f3824003a00 con 0x7f382c101890 2023-09-23T09:27:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.421+0000 7f3822ffd700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38180a1670 0x7f38180a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.421+0000 7f38319df700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38180a1670 0x7f38180a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.421+0000 7f3822ffd700 1 -- 172.21.15.45:0/1931674347 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f38240cc6f0 con 0x7f382c101890 2023-09-23T09:27:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.422+0000 7f38319df700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38180a1670 0x7f38180a3b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f381c009fd0 tx=0x7f381c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.424+0000 7f3822ffd700 1 -- 172.21.15.45:0/1931674347 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3824096170 con 0x7f382c101890 2023-09-23T09:27:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.672+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f382c02cf30 con 0x7f382c101890 2023-09-23T09:27:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.674+0000 7f3822ffd700 1 -- 172.21.15.45:0/1931674347 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5280 (secure 0 0 0) 0x7f3824099790 con 0x7f382c101890 2023-09-23T09:27:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:31.675 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-09-23T09:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38180a1670 msgr2=0x7f38180a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38180a1670 0x7f38180a3b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f381c009fd0 tx=0x7f381c005fb0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f382c101890 msgr2=0x7f382c19e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f382c101890 0x7f382c19e0f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f382400efc0 tx=0x7f382400c3a0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 shutdown_connections 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f38180a1670 0x7f38180a3b30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f382c101890 0x7f382c19e0f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f382c0ffd00 0x7f382c196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 --2- 172.21.15.45:0/1931674347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f382c100ef0 0x7f382c197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 >> 172.21.15.45:0/1931674347 conn(0x7f382c0fb250 msgr2=0x7f382c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 shutdown_connections 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:31.675+0000 7f3833c43700 1 -- 172.21.15.45:0/1931674347 wait complete. 2023-09-23T09:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2023-09-23T09:27:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: Deploying daemon mds.c on smithi123 2023-09-23T09:27:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6834/3236511886,v1:172.21.15.123:6835/3236511886] up:boot 2023-09-23T09:27:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: fsmap 6 up:standby 2023-09-23T09:27:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:31 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: Deploying daemon mds.c on smithi123 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6834/3236511886,v1:172.21.15.123:6835/3236511886] up:boot 2023-09-23T09:27:32.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: fsmap 6 up:standby 2023-09-23T09:27:32.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:31 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:27:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:32.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:32.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:32.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: Deploying daemon mds.c on smithi123 2023-09-23T09:27:32.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6834/3236511886,v1:172.21.15.123:6835/3236511886] up:boot 2023-09-23T09:27:32.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: fsmap 6 up:standby 2023-09-23T09:27:32.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:31 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:27:32.180 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:27:32 smithi123 systemd[1]: Starting Ceph mds.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:32.703 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:32 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1931674347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:32.703 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:27:32 smithi123 podman[139509]: 2023-09-23T09:27:32.703 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:27:32 smithi123 ceph-mds[139549]: -- 172.21.15.123:0/325386931 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5587db12d1e0 con 0x5587db102c00 2023-09-23T09:27:32.703 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:27:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: starting mds.c at 2023-09-23T09:27:33.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:32 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1931674347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:33.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:32 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1931674347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:33.088 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:27:32 smithi123 bash[139509]: 60bcf51042bb49d46c72c56b2170bfd276e3c3b0e1c22702c4143b5b25ca1544 2023-09-23T09:27:33.088 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:27:32 smithi123 systemd[1]: Started Ceph mds.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:33.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:33.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 -- 172.21.15.45:0/235272474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8980ffd30 msgr2=0x7fa898100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:33.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/235272474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8980ffd30 0x7fa898100150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa89400b0d0 tx=0x7fa89400b3e0 comp rx=0 tx=0).stop 2023-09-23T09:27:33.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 -- 172.21.15.45:0/235272474 shutdown_connections 2023-09-23T09:27:33.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/235272474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8981018c0 0x7fa898108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:33.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/235272474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8980ffd30 0x7fa898100150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/235272474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898100f20 0x7fa898101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 -- 172.21.15.45:0/235272474 >> 172.21.15.45:0/235272474 conn(0x7fa8980fb280 msgr2=0x7fa8980fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 -- 172.21.15.45:0/235272474 shutdown_connections 2023-09-23T09:27:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.829+0000 7fa89fd7f700 1 -- 172.21.15.45:0/235272474 wait complete. 2023-09-23T09:27:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 Processor -- start 2023-09-23T09:27:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 -- start start 2023-09-23T09:27:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8980ffd30 0x7fa898195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898100f20 0x7fa898196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8981018c0 0x7fa89819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89819d6a0 con 0x7fa8981018c0 2023-09-23T09:27:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89819d7e0 con 0x7fa8980ffd30 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.830+0000 7fa89fd7f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89819d970 con 0x7fa898100f20 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8980ffd30 0x7fa898195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89e31c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8981018c0 0x7fa89819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8980ffd30 0x7fa898195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:51720/0 (socket says 172.21.15.45:51720) 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 -- 172.21.15.45:0/1594218788 learned_addr learned my addr 172.21.15.45:0/1594218788 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89d31a700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898100f20 0x7fa898196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 -- 172.21.15.45:0/1594218788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898100f20 msgr2=0x7fa898196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898100f20 0x7fa898196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 -- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8981018c0 msgr2=0x7fa89819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8981018c0 0x7fa89819d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 -- 172.21.15.45:0/1594218788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894009d00 con 0x7fa8980ffd30 2023-09-23T09:27:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89e31c700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8981018c0 0x7fa89819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89db1b700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8980ffd30 0x7fa898195b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa894000f80 tx=0x7fa894008f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa88effd700 1 -- 172.21.15.45:0/1594218788 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89400e040 con 0x7fa8980ffd30 2023-09-23T09:27:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa88effd700 1 -- 172.21.15.45:0/1594218788 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa89401e070 con 0x7fa8980ffd30 2023-09-23T09:27:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8980691e0 con 0x7fa8980ffd30 2023-09-23T09:27:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa88effd700 1 -- 172.21.15.45:0/1594218788 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894025940 con 0x7fa8980ffd30 2023-09-23T09:27:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.831+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8980696f0 con 0x7fa8980ffd30 2023-09-23T09:27:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.832+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89806a0e0 con 0x7fa8980ffd30 2023-09-23T09:27:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.833+0000 7fa88effd700 1 -- 172.21.15.45:0/1594218788 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fa894025aa0 con 0x7fa8980ffd30 2023-09-23T09:27:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.834+0000 7fa88effd700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8840a15f0 0x7fa8840a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.834+0000 7fa89d31a700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8840a15f0 0x7fa8840a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.834+0000 7fa88effd700 1 -- 172.21.15.45:0/1594218788 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa894020070 con 0x7fa8980ffd30 2023-09-23T09:27:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.834+0000 7fa89d31a700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8840a15f0 0x7fa8840a3ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa888005950 tx=0x7fa88800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:33.837+0000 7fa88effd700 1 -- 172.21.15.45:0/1594218788 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa894099b10 con 0x7fa8980ffd30 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: Deploying daemon mds.h on smithi098 2023-09-23T09:27:34.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6836/2272642085,v1:172.21.15.123:6837/2272642085] up:boot 2023-09-23T09:27:34.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: fsmap 7 up:standby 2023-09-23T09:27:34.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:33 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:27:34.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.081+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa89810b790 con 0x7fa8980ffd30 2023-09-23T09:27:34.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.082+0000 7fa88effd700 1 -- 172.21.15.45:0/1594218788 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6074 (secure 0 0 0) 0x7fa89409d130 con 0x7fa8980ffd30 2023-09-23T09:27:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:34.083 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8840a15f0 msgr2=0x7fa8840a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8840a15f0 0x7fa8840a3ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa888005950 tx=0x7fa88800b410 comp rx=0 tx=0).stop 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8980ffd30 msgr2=0x7fa898195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8980ffd30 0x7fa898195b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa894000f80 tx=0x7fa894008f10 comp rx=0 tx=0).stop 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 shutdown_connections 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8981018c0 0x7fa89819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8980ffd30 0x7fa898195b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8840a15f0 0x7fa8840a3ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 --2- 172.21.15.45:0/1594218788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898100f20 0x7fa898196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 >> 172.21.15.45:0/1594218788 conn(0x7fa8980fb280 msgr2=0x7fa8980fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 shutdown_connections 2023-09-23T09:27:34.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:34.084+0000 7fa89fd7f700 1 -- 172.21.15.45:0/1594218788 wait complete. 2023-09-23T09:27:34.087 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-09-23T09:27:34.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: Deploying daemon mds.h on smithi098 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6836/2272642085,v1:172.21.15.123:6837/2272642085] up:boot 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: fsmap 7 up:standby 2023-09-23T09:27:34.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:33 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:27:34.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:34.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:34.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:34.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:34.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:34.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: Deploying daemon mds.h on smithi098 2023-09-23T09:27:34.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6836/2272642085,v1:172.21.15.123:6837/2272642085] up:boot 2023-09-23T09:27:34.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: fsmap 7 up:standby 2023-09-23T09:27:34.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:33 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:27:34.468 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:27:34 smithi098 systemd[1]: Starting Ceph mds.h for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:35.025 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:27:34 smithi098 podman[137150]: 2023-09-23T09:27:35.026 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:34 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1594218788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:35.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:34 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1594218788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:35.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:34 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1594218788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:35.294 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:27:35 smithi098 ceph-mds[137190]: -- 172.21.15.98:0/1822044798 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561a22b2f1e0 con 0x561a22b04c00 2023-09-23T09:27:35.294 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:27:35 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-h[137166]: starting mds.h at 2023-09-23T09:27:35.294 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:27:35 smithi098 bash[137150]: 86817e815924f2646305355d1f87ab6e69c81d41b1e0c175f4633a2b18e08e71 2023-09-23T09:27:35.294 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:27:35 smithi098 systemd[1]: Started Ceph mds.h for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:36.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 -- 172.21.15.45:0/1161881967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 msgr2=0x7f2be8101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/1161881967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8101350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0099c0 tx=0x7f2bdc009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 -- 172.21.15.45:0/1161881967 shutdown_connections 2023-09-23T09:27:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/1161881967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be8101890 0x7f2be8108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/1161881967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2be80ffd00 0x7f2be8100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/1161881967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8101350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 -- 172.21.15.45:0/1161881967 >> 172.21.15.45:0/1161881967 conn(0x7f2be80fb250 msgr2=0x7f2be80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:36.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 -- 172.21.15.45:0/1161881967 shutdown_connections 2023-09-23T09:27:36.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.234+0000 7f2bf01aa700 1 -- 172.21.15.45:0/1161881967 wait complete. 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 Processor -- start 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 -- start start 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2be80ffd00 0x7f2be81959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be8101890 0x7f2be819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be819d530 con 0x7f2be8100ef0 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be819d670 con 0x7f2be80ffd00 2023-09-23T09:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.235+0000 7f2bf01aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be819d800 con 0x7f2be8101890 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58618/0 (socket says 172.21.15.45:58618) 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 -- 172.21.15.45:0/2587212668 learned_addr learned my addr 172.21.15.45:0/2587212668 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bedf46700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2be80ffd00 0x7f2be81959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bee747700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be8101890 0x7f2be819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 -- 172.21.15.45:0/2587212668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be8101890 msgr2=0x7f2be819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be8101890 0x7f2be819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 -- 172.21.15.45:0/2587212668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2be80ffd00 msgr2=0x7f2be81959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2be80ffd00 0x7f2be81959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 -- 172.21.15.45:0/2587212668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0095f0 con 0x7f2be8100ef0 2023-09-23T09:27:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bed745700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8195ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009fd0 tx=0x7f2bdc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:36.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bdaffd700 1 -- 172.21.15.45:0/2587212668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc01d070 con 0x7f2be8100ef0 2023-09-23T09:27:36.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bdaffd700 1 -- 172.21.15.45:0/2587212668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bdc0054f0 con 0x7f2be8100ef0 2023-09-23T09:27:36.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be819da80 con 0x7f2be8100ef0 2023-09-23T09:27:36.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bdaffd700 1 -- 172.21.15.45:0/2587212668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004bd0 con 0x7f2be8100ef0 2023-09-23T09:27:36.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.236+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be819df00 con 0x7f2be8100ef0 2023-09-23T09:27:36.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.237+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be810b510 con 0x7f2be8100ef0 2023-09-23T09:27:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.238+0000 7f2bdaffd700 1 -- 172.21.15.45:0/2587212668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f2bdc00fb80 con 0x7f2be8100ef0 2023-09-23T09:27:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.239+0000 7f2bdaffd700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2bd40a1670 0x7f2bd40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.239+0000 7f2bedf46700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2bd40a1670 0x7f2bd40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.239+0000 7f2bdaffd700 1 -- 172.21.15.45:0/2587212668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2bdc0cd8c0 con 0x7f2be8100ef0 2023-09-23T09:27:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.239+0000 7f2bedf46700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2bd40a1670 0x7f2bd40a3b30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2be8100d50 tx=0x7f2be400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:36.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.243+0000 7f2bdaffd700 1 -- 172.21.15.45:0/2587212668 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2bdc0973e0 con 0x7f2be8100ef0 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: Deploying daemon mds.i on smithi123 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] up:boot 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: fsmap 8 up:standby 2023-09-23T09:27:36.377 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:36 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:27:36.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.490+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2be8106be0 con 0x7f2be8100ef0 2023-09-23T09:27:36.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.491+0000 7f2bdaffd700 1 -- 172.21.15.45:0/2587212668 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6862 (secure 0 0 0) 0x7f2bdc09aa00 con 0x7f2be8100ef0 2023-09-23T09:27:36.494 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:36.494 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-23T09:27:36.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2bd40a1670 msgr2=0x7f2bd40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:36.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2bd40a1670 0x7f2bd40a3b30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2be8100d50 tx=0x7f2be400b410 comp rx=0 tx=0).stop 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 msgr2=0x7f2be8195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8195ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009fd0 tx=0x7f2bdc00f690 comp rx=0 tx=0).stop 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 shutdown_connections 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2bd40a1670 0x7f2bd40a3b30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be8101890 0x7f2be819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2be80ffd00 0x7f2be81959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 --2- 172.21.15.45:0/2587212668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be8100ef0 0x7f2be8195ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.493+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 >> 172.21.15.45:0/2587212668 conn(0x7f2be80fb250 msgr2=0x7f2be80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.494+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 shutdown_connections 2023-09-23T09:27:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:36.494+0000 7f2bf01aa700 1 -- 172.21.15.45:0/2587212668 wait complete. 2023-09-23T09:27:36.498 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: Deploying daemon mds.i on smithi123 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] up:boot 2023-09-23T09:27:36.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: fsmap 8 up:standby 2023-09-23T09:27:36.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:36 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: Deploying daemon mds.i on smithi123 2023-09-23T09:27:36.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] up:boot 2023-09-23T09:27:36.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: fsmap 8 up:standby 2023-09-23T09:27:36.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:36 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:27:36.839 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:27:36 smithi123 systemd[1]: Starting Ceph mds.i for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:37.193 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:27:36 smithi123 podman[139858]: 2023-09-23T09:27:37.194 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:37 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2587212668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:37.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:37 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2587212668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:37.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:37 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2587212668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:37.588 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:27:37 smithi123 ceph-mds[139899]: -- 172.21.15.123:0/125149425 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560457871e0 con 0x55604575c800 2023-09-23T09:27:37.588 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:27:37 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-i[139874]: starting mds.i at 2023-09-23T09:27:37.588 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:27:37 smithi123 bash[139858]: efc8dbc0421c2f895dc13078179323559befac9cb2bc1b7746f4b69299840dd4 2023-09-23T09:27:37.589 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:27:37 smithi123 systemd[1]: Started Ceph mds.i for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:38.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:38 smithi045 ceph-mon[98627]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:38.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.290 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:38.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:38.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:38 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:38.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:38.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:38 smithi098 ceph-mon[102386]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:38.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:38.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:38.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:38 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:38.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:38 smithi123 ceph-mon[103577]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:38.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:38.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:38.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:38.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:38 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:38.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 -- 172.21.15.45:0/5012489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb581018c0 msgr2=0x7fbb58103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:38.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 --2- 172.21.15.45:0/5012489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb581018c0 0x7fbb58103cb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009990 tx=0x7fbb54009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 -- 172.21.15.45:0/5012489 shutdown_connections 2023-09-23T09:27:38.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 --2- 172.21.15.45:0/5012489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58106b20 0x7fbb58108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 --2- 172.21.15.45:0/5012489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb581018c0 0x7fbb58103cb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 --2- 172.21.15.45:0/5012489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb581041f0 0x7fbb581065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 -- 172.21.15.45:0/5012489 >> 172.21.15.45:0/5012489 conn(0x7fbb580fb250 msgr2=0x7fbb580fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:38.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 -- 172.21.15.45:0/5012489 shutdown_connections 2023-09-23T09:27:38.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.640+0000 7fbb60093700 1 -- 172.21.15.45:0/5012489 wait complete. 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.641+0000 7fbb60093700 1 Processor -- start 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.641+0000 7fbb60093700 1 -- start start 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.641+0000 7fbb60093700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb581018c0 0x7fbb58196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb60093700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb581041f0 0x7fbb58197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb60093700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb58106b20 0x7fbb5819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb60093700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5819e680 con 0x7fbb581041f0 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb60093700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5819e7c0 con 0x7fbb58106b20 2023-09-23T09:27:38.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb60093700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5819e950 con 0x7fbb581018c0 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb581018c0 0x7fbb58196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb581018c0 0x7fbb58196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:60734/0 (socket says 172.21.15.45:60734) 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 -- 172.21.15.45:0/1787161939 learned_addr learned my addr 172.21.15.45:0/1787161939 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5e630700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb58106b20 0x7fbb5819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 -- 172.21.15.45:0/1787161939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb58106b20 msgr2=0x7fbb5819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb58106b20 0x7fbb5819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 -- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb581041f0 msgr2=0x7fbb58197040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb581041f0 0x7fbb58197040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 -- 172.21.15.45:0/1787161939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540095f0 con 0x7fbb581018c0 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.642+0000 7fbb5de2f700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb581018c0 0x7fbb58196b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005910 tx=0x7fbb5400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.643+0000 7fbb4effd700 1 -- 172.21.15.45:0/1787161939 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5401d070 con 0x7fbb581018c0 2023-09-23T09:27:38.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.643+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb580ff6b0 con 0x7fbb581018c0 2023-09-23T09:27:38.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.643+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb580ffbb0 con 0x7fbb581018c0 2023-09-23T09:27:38.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.643+0000 7fbb4effd700 1 -- 172.21.15.45:0/1787161939 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb5400fb80 con 0x7fbb581018c0 2023-09-23T09:27:38.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.643+0000 7fbb4effd700 1 -- 172.21.15.45:0/1787161939 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54017650 con 0x7fbb581018c0 2023-09-23T09:27:38.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.644+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c005320 con 0x7fbb581018c0 2023-09-23T09:27:38.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.645+0000 7fbb4effd700 1 -- 172.21.15.45:0/1787161939 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fbb5400fcf0 con 0x7fbb581018c0 2023-09-23T09:27:38.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.646+0000 7fbb4effd700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbb440a1670 0x7fbb440a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:38.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.646+0000 7fbb5d62e700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbb440a1670 0x7fbb440a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:38.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.646+0000 7fbb4effd700 1 -- 172.21.15.45:0/1787161939 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb540ce1a0 con 0x7fbb581018c0 2023-09-23T09:27:38.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.647+0000 7fbb5d62e700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbb440a1670 0x7fbb440a3b30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800ba10 tx=0x7fbb4800b3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:38.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.655+0000 7fbb4effd700 1 -- 172.21.15.45:0/1787161939 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbb54097c20 con 0x7fbb581018c0 2023-09-23T09:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.922+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb3c0058a0 con 0x7fbb581018c0 2023-09-23T09:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.923+0000 7fbb4effd700 1 -- 172.21.15.45:0/1787161939 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7658 (secure 0 0 0) 0x7fbb5409b240 con 0x7fbb581018c0 2023-09-23T09:27:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-23T09:27:38.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbb440a1670 msgr2=0x7fbb440a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:38.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbb440a1670 0x7fbb440a3b30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800ba10 tx=0x7fbb4800b3f0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb581018c0 msgr2=0x7fbb58196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:38.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb581018c0 0x7fbb58196b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005910 tx=0x7fbb5400f690 comp rx=0 tx=0).stop 2023-09-23T09:27:38.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 shutdown_connections 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb58106b20 0x7fbb5819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbb440a1670 0x7fbb440a3b30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb581018c0 0x7fbb58196b00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 --2- 172.21.15.45:0/1787161939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb581041f0 0x7fbb58197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 >> 172.21.15.45:0/1787161939 conn(0x7fbb580fb250 msgr2=0x7fbb58102530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 shutdown_connections 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:38.925+0000 7fbb60093700 1 -- 172.21.15.45:0/1787161939 wait complete. 2023-09-23T09:27:38.929 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2023-09-23T09:27:39.215 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:27:38 smithi045 systemd[1]: Starting Ceph mds.g for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:39.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:39 smithi045 ceph-mon[98627]: Deploying daemon mds.g on smithi045 2023-09-23T09:27:39.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:39 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] up:boot 2023-09-23T09:27:39.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:39 smithi045 ceph-mon[98627]: fsmap 9 up:standby 2023-09-23T09:27:39.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:39 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:27:39.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:39 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1787161939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:39.488 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:27:39 smithi045 podman[151352]: 2023-09-23T09:27:39.488 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:27:39 smithi045 bash[151352]: 1ca237c49e5efb1fe58dcf779554be8fba0e2dec09e5126c446af7a83e945e25 2023-09-23T09:27:39.488 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:27:39 smithi045 ceph-mds[151392]: -- 172.21.15.45:0/2146885829 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bea5d511e0 con 0x55bea5d27000 2023-09-23T09:27:39.489 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:27:39 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-g[151368]: starting mds.g at 2023-09-23T09:27:39.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:39 smithi098 ceph-mon[102386]: Deploying daemon mds.g on smithi045 2023-09-23T09:27:39.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:39 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] up:boot 2023-09-23T09:27:39.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:39 smithi098 ceph-mon[102386]: fsmap 9 up:standby 2023-09-23T09:27:39.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:39 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:27:39.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:39 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1787161939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:39.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:39 smithi123 ceph-mon[103577]: Deploying daemon mds.g on smithi045 2023-09-23T09:27:39.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:39 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] up:boot 2023-09-23T09:27:39.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:39 smithi123 ceph-mon[103577]: fsmap 9 up:standby 2023-09-23T09:27:39.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:39 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:27:39.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:39 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1787161939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:39.830 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:27:39 smithi045 systemd[1]: Started Ceph mds.g for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:40 smithi045 ceph-mon[98627]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:40.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:40.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:40 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:40.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:40 smithi098 ceph-mon[102386]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:40.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:40 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:40.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:40 smithi123 ceph-mon[103577]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:40.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:40.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:40 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:40.944 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 -- 172.21.15.45:0/2053072721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650104280 msgr2=0x7fd650106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 --2- 172.21.15.45:0/2053072721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650104280 0x7fd650106670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd644009990 tx=0x7fd644009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 -- 172.21.15.45:0/2053072721 shutdown_connections 2023-09-23T09:27:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 --2- 172.21.15.45:0/2053072721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd650106bb0 0x7fd650108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 --2- 172.21.15.45:0/2053072721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd650101950 0x7fd650103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 --2- 172.21.15.45:0/2053072721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650104280 0x7fd650106670 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 -- 172.21.15.45:0/2053072721 >> 172.21.15.45:0/2053072721 conn(0x7fd6500fb2a0 msgr2=0x7fd6500fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 -- 172.21.15.45:0/2053072721 shutdown_connections 2023-09-23T09:27:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.152+0000 7fd6549c9700 1 -- 172.21.15.45:0/2053072721 wait complete. 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 Processor -- start 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 -- start start 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd650101950 0x7fd650196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd650104280 0x7fd650196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650106bb0 0x7fd65019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65019e5b0 con 0x7fd650106bb0 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65019e6f0 con 0x7fd650101950 2023-09-23T09:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd6549c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65019e880 con 0x7fd650104280 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650106bb0 0x7fd65019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650106bb0 0x7fd65019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58658/0 (socket says 172.21.15.45:58658) 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 -- 172.21.15.45:0/169096959 learned_addr learned my addr 172.21.15.45:0/169096959 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64dd9b700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd650104280 0x7fd650196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64e59c700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd650101950 0x7fd650196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 -- 172.21.15.45:0/169096959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd650104280 msgr2=0x7fd650196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd650104280 0x7fd650196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 -- 172.21.15.45:0/169096959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd650101950 msgr2=0x7fd650196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd650101950 0x7fd650196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64ed9d700 1 -- 172.21.15.45:0/169096959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6440095f0 con 0x7fd650106bb0 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.153+0000 7fd64e59c700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd650101950 0x7fd650196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.154+0000 7fd64ed9d700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650106bb0 0x7fd65019dfe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd64800ef30 tx=0x7fd64800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.155+0000 7fd63f7fe700 1 -- 172.21.15.45:0/169096959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648018070 con 0x7fd650106bb0 2023-09-23T09:27:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.155+0000 7fd63f7fe700 1 -- 172.21.15.45:0/169096959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64800f040 con 0x7fd650106bb0 2023-09-23T09:27:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.155+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65019eb00 con 0x7fd650106bb0 2023-09-23T09:27:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.155+0000 7fd63f7fe700 1 -- 172.21.15.45:0/169096959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64800cdb0 con 0x7fd650106bb0 2023-09-23T09:27:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.155+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65019f060 con 0x7fd650106bb0 2023-09-23T09:27:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.156+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6500fce20 con 0x7fd650106bb0 2023-09-23T09:27:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.158+0000 7fd63f7fe700 1 -- 172.21.15.45:0/169096959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fd648003a00 con 0x7fd650106bb0 2023-09-23T09:27:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.159+0000 7fd63f7fe700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd6380a1670 0x7fd6380a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.159+0000 7fd63f7fe700 1 -- 172.21.15.45:0/169096959 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6480ccf30 con 0x7fd650106bb0 2023-09-23T09:27:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.159+0000 7fd64e59c700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd6380a1670 0x7fd6380a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.159+0000 7fd64e59c700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd6380a1670 0x7fd6380a3b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd640005950 tx=0x7fd6400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.162+0000 7fd63f7fe700 1 -- 172.21.15.45:0/169096959 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd648096a50 con 0x7fd650106bb0 2023-09-23T09:27:41.330 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:27:41 smithi045 systemd[1]: Starting Ceph mds.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:41 smithi045 ceph-mon[98627]: Deploying daemon mds.a on smithi045 2023-09-23T09:27:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:41 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.45:6838/581280967,v1:172.21.15.45:6839/581280967] up:boot 2023-09-23T09:27:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:41 smithi045 ceph-mon[98627]: fsmap 10 up:standby 2023-09-23T09:27:41.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:41 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T09:27:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.421+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd65010c610 con 0x7fd650106bb0 2023-09-23T09:27:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.421+0000 7fd63f7fe700 1 -- 172.21.15.45:0/169096959 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8447 (secure 0 0 0) 0x7fd648025380 con 0x7fd650106bb0 2023-09-23T09:27:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:41.424 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-23T09:27:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd6380a1670 msgr2=0x7fd6380a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd6380a1670 0x7fd6380a3b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd640005950 tx=0x7fd6400058e0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650106bb0 msgr2=0x7fd65019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650106bb0 0x7fd65019dfe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd64800ef30 tx=0x7fd64800c3a0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 shutdown_connections 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd650106bb0 0x7fd65019dfe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd650101950 0x7fd650196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd6380a1670 0x7fd6380a3b30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 --2- 172.21.15.45:0/169096959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd650104280 0x7fd650196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.424+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 >> 172.21.15.45:0/169096959 conn(0x7fd6500fb2a0 msgr2=0x7fd6501025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.425+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 shutdown_connections 2023-09-23T09:27:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:41.425+0000 7fd6549c9700 1 -- 172.21.15.45:0/169096959 wait complete. 2023-09-23T09:27:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 11 2023-09-23T09:27:41.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:41 smithi098 ceph-mon[102386]: Deploying daemon mds.a on smithi045 2023-09-23T09:27:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:41 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.45:6838/581280967,v1:172.21.15.45:6839/581280967] up:boot 2023-09-23T09:27:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:41 smithi098 ceph-mon[102386]: fsmap 10 up:standby 2023-09-23T09:27:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:41 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T09:27:41.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:41 smithi123 ceph-mon[103577]: Deploying daemon mds.a on smithi045 2023-09-23T09:27:41.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:41 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.45:6838/581280967,v1:172.21.15.45:6839/581280967] up:boot 2023-09-23T09:27:41.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:41 smithi123 ceph-mon[103577]: fsmap 10 up:standby 2023-09-23T09:27:41.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:41 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T09:27:41.613 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:27:41 smithi045 podman[151734]: 2023-09-23T09:27:42.080 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:27:41 smithi045 bash[151734]: 886d984b5a661451b90efaa2fb94d0548107a44dc98d7e1b7af959f36ebc2daa 2023-09-23T09:27:42.080 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:27:41 smithi045 ceph-mds[151776]: -- 172.21.15.45:0/3969298630 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5622f2f391e0 con 0x5622f2f0e800 2023-09-23T09:27:42.080 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:27:41 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-a[151750]: starting mds.a at 2023-09-23T09:27:42.081 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:27:41 smithi045 systemd[1]: Started Ceph mds.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:42.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/169096959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:42 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/169096959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:42.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:42.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:42 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:42.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:42.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/169096959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:42.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:42.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T09:27:42.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T09:27:42.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:42 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:43.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:43.474 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:43 smithi045 ceph-mon[98627]: Deploying daemon mds.b on smithi098 2023-09-23T09:27:43.474 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:43 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.45:6840/4027335319,v1:172.21.15.45:6841/4027335319] up:boot 2023-09-23T09:27:43.474 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:43 smithi045 ceph-mon[98627]: fsmap 11 up:standby 2023-09-23T09:27:43.474 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:43 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T09:27:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:43 smithi098 ceph-mon[102386]: Deploying daemon mds.b on smithi098 2023-09-23T09:27:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:43 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.45:6840/4027335319,v1:172.21.15.45:6841/4027335319] up:boot 2023-09-23T09:27:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:43 smithi098 ceph-mon[102386]: fsmap 11 up:standby 2023-09-23T09:27:43.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:43 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T09:27:43.544 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:27:43 smithi098 systemd[1]: Starting Ceph mds.b for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 -- 172.21.15.45:0/1644890799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10900ffee0 msgr2=0x7f1090100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 --2- 172.21.15.45:0/1644890799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10900ffee0 0x7f1090100360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1084009990 tx=0x7f1084009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 -- 172.21.15.45:0/1644890799 shutdown_connections 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 --2- 172.21.15.45:0/1644890799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10901008a0 0x7f1090100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 --2- 172.21.15.45:0/1644890799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900ff580 0x7f10900ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 --2- 172.21.15.45:0/1644890799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10900ffee0 0x7f1090100360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 -- 172.21.15.45:0/1644890799 >> 172.21.15.45:0/1644890799 conn(0x7f10900fb0e0 msgr2=0x7f10900fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 -- 172.21.15.45:0/1644890799 shutdown_connections 2023-09-23T09:27:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.583+0000 7f109519e700 1 -- 172.21.15.45:0/1644890799 wait complete. 2023-09-23T09:27:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 Processor -- start 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 -- start start 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900ff580 0x7f1090198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10900ffee0 0x7f1090198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10901008a0 0x7f109019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10901a05a0 con 0x7f10901008a0 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10901a06e0 con 0x7f10900ffee0 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109519e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10901a0870 con 0x7f10900ff580 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109499d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10901008a0 0x7f109019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109499d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10901008a0 0x7f109019ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58686/0 (socket says 172.21.15.45:58686) 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f109499d700 1 -- 172.21.15.45:0/3383502506 learned_addr learned my addr 172.21.15.45:0/3383502506 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.584+0000 7f108f7fe700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10900ffee0 0x7f1090198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:43.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f108ffff700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900ff580 0x7f1090198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109499d700 1 -- 172.21.15.45:0/3383502506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900ff580 msgr2=0x7f1090198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109499d700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900ff580 0x7f1090198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109499d700 1 -- 172.21.15.45:0/3383502506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10900ffee0 msgr2=0x7f1090198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109499d700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10900ffee0 0x7f1090198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109499d700 1 -- 172.21.15.45:0/3383502506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840095f0 con 0x7f10901008a0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109499d700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10901008a0 0x7f109019ffd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f108800b970 tx=0x7f108800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f108d7fa700 1 -- 172.21.15.45:0/3383502506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108800c930 con 0x7f10901008a0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f108d7fa700 1 -- 172.21.15.45:0/3383502506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1088014440 con 0x7f10901008a0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10901a0b50 con 0x7f10901008a0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f108d7fa700 1 -- 172.21.15.45:0/3383502506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880075d0 con 0x7f10901008a0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.585+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10901a1010 con 0x7f10901008a0 2023-09-23T09:27:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.586+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109010a360 con 0x7f10901008a0 2023-09-23T09:27:43.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:43 smithi123 ceph-mon[103577]: Deploying daemon mds.b on smithi098 2023-09-23T09:27:43.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:43 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.45:6840/4027335319,v1:172.21.15.45:6841/4027335319] up:boot 2023-09-23T09:27:43.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:43 smithi123 ceph-mon[103577]: fsmap 11 up:standby 2023-09-23T09:27:43.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:43 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T09:27:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.587+0000 7f108d7fa700 1 -- 172.21.15.45:0/3383502506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f10880145b0 con 0x7f10901008a0 2023-09-23T09:27:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.588+0000 7f108d7fa700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f10780a1690 0x7f10780a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.588+0000 7f108ffff700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f10780a1690 0x7f10780a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.588+0000 7f108d7fa700 1 -- 172.21.15.45:0/3383502506 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1088006e40 con 0x7f10901008a0 2023-09-23T09:27:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.588+0000 7f108ffff700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f10780a1690 0x7f10780a3b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1080005ea0 tx=0x7f1080005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:43.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.592+0000 7f108d7fa700 1 -- 172.21.15.45:0/3383502506 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f108804eec0 con 0x7f10901008a0 2023-09-23T09:27:43.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.837+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f109010e750 con 0x7f10901008a0 2023-09-23T09:27:43.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.838+0000 7f108d7fa700 1 -- 172.21.15.45:0/3383502506 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9239 (secure 0 0 0) 0x7f1088095b60 con 0x7f10901008a0 2023-09-23T09:27:43.839 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:43.840 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f10780a1690 msgr2=0x7f10780a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f10780a1690 0x7f10780a3b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1080005ea0 tx=0x7f1080005e30 comp rx=0 tx=0).stop 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10901008a0 msgr2=0x7f109019ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10901008a0 0x7f109019ffd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f108800b970 tx=0x7f108800bc80 comp rx=0 tx=0).stop 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 shutdown_connections 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10901008a0 0x7f109019ffd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f10780a1690 0x7f10780a3b50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900ff580 0x7f1090198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 --2- 172.21.15.45:0/3383502506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10900ffee0 0x7f1090198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 >> 172.21.15.45:0/3383502506 conn(0x7f10900fb0e0 msgr2=0x7f10900fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 shutdown_connections 2023-09-23T09:27:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:43.840+0000 7f109519e700 1 -- 172.21.15.45:0/3383502506 wait complete. 2023-09-23T09:27:43.843 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2023-09-23T09:27:43.928 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:27:43 smithi098 podman[137491]: 2023-09-23T09:27:44.293 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:27:43 smithi098 ceph-mds[137531]: -- 172.21.15.98:0/4293455902 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643935cd1e0 con 0x5643935a2800 2023-09-23T09:27:44.293 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:27:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b[137507]: starting mds.b at 2023-09-23T09:27:44.293 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:27:43 smithi098 bash[137491]: a1718a790ae630b5e0301190fb08428f84b92b9649c175b828f5813b7c9b7d2d 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:27:44 smithi098 systemd[1]: Started Ceph mds.b for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:44 smithi098 ceph-mon[102386]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:44 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3383502506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:44 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:44 smithi045 ceph-mon[98627]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:44 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3383502506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:44 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:44 smithi123 ceph-mon[103577]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:44.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:44 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3383502506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:44.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:44.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:44 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:45.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:45 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] up:boot 2023-09-23T09:27:45.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:45 smithi098 ceph-mon[102386]: fsmap 12 up:standby 2023-09-23T09:27:45.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:27:45.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:45.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:45 smithi098 ceph-mon[102386]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:45.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:45 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:45 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] up:boot 2023-09-23T09:27:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:45 smithi045 ceph-mon[98627]: fsmap 12 up:standby 2023-09-23T09:27:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:27:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:45 smithi045 ceph-mon[98627]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:45 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:45.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:45 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] up:boot 2023-09-23T09:27:45.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:45 smithi123 ceph-mon[103577]: fsmap 12 up:standby 2023-09-23T09:27:45.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:27:45.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:45.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:45 smithi123 ceph-mon[103577]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:45.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:45 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:45.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 -- 172.21.15.45:0/1618365225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21780ffd30 msgr2=0x7f2178100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 --2- 172.21.15.45:0/1618365225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21780ffd30 0x7f2178100150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21680099c0 tx=0x7f2168009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 -- 172.21.15.45:0/1618365225 shutdown_connections 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 --2- 172.21.15.45:0/1618365225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21781018c0 0x7f2178108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 --2- 172.21.15.45:0/1618365225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21780ffd30 0x7f2178100150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 --2- 172.21.15.45:0/1618365225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2178100f20 0x7f2178101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 -- 172.21.15.45:0/1618365225 >> 172.21.15.45:0/1618365225 conn(0x7f21780fb2a0 msgr2=0x7f21780fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 -- 172.21.15.45:0/1618365225 shutdown_connections 2023-09-23T09:27:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.983+0000 7f217e4ff700 1 -- 172.21.15.45:0/1618365225 wait complete. 2023-09-23T09:27:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 Processor -- start 2023-09-23T09:27:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 -- start start 2023-09-23T09:27:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21780ffd30 0x7f2178196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2178100f20 0x7f2178197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21781018c0 0x7f217819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217819e670 con 0x7f2178100f20 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217819e7b0 con 0x7f21781018c0 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217e4ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217819e940 con 0x7f21780ffd30 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f21777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2178100f20 0x7f2178197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f2177fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21780ffd30 0x7f2178196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f2177fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21780ffd30 0x7f2178196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:60800/0 (socket says 172.21.15.45:60800) 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f21777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2178100f20 0x7f2178197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58696/0 (socket says 172.21.15.45:58696) 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f2177fff700 1 -- 172.21.15.45:0/4195547180 learned_addr learned my addr 172.21.15.45:0/4195547180 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.984+0000 7f217ca9c700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21781018c0 0x7f217819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f2177fff700 1 -- 172.21.15.45:0/4195547180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21781018c0 msgr2=0x7f217819e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f2177fff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21781018c0 0x7f217819e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f2177fff700 1 -- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2178100f20 msgr2=0x7f2178197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f2177fff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2178100f20 0x7f2178197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f2177fff700 1 -- 172.21.15.45:0/4195547180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21680095f0 con 0x7f21780ffd30 2023-09-23T09:27:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f21777fe700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2178100f20 0x7f2178197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f2177fff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21780ffd30 0x7f2178196af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2168009990 tx=0x7f216801ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f21757fa700 1 -- 172.21.15.45:0/4195547180 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168021070 con 0x7f21780ffd30 2023-09-23T09:27:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f21757fa700 1 -- 172.21.15.45:0/4195547180 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21680054f0 con 0x7f21780ffd30 2023-09-23T09:27:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2178102140 con 0x7f21780ffd30 2023-09-23T09:27:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f21757fa700 1 -- 172.21.15.45:0/4195547180 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168004bb0 con 0x7f21780ffd30 2023-09-23T09:27:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.985+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2178102590 con 0x7f21780ffd30 2023-09-23T09:27:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.986+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217810b590 con 0x7f21780ffd30 2023-09-23T09:27:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.987+0000 7f21757fa700 1 -- 172.21.15.45:0/4195547180 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f2168025070 con 0x7f21780ffd30 2023-09-23T09:27:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.988+0000 7f21757fa700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f21600a15f0 0x7f21600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.988+0000 7f21777fe700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f21600a15f0 0x7f21600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.988+0000 7f21757fa700 1 -- 172.21.15.45:0/4195547180 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f21680d1bf0 con 0x7f21780ffd30 2023-09-23T09:27:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.989+0000 7f21777fe700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f21600a15f0 0x7f21600a3ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f216c011490 tx=0x7f216c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:45.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:45.996+0000 7f21757fa700 1 -- 172.21.15.45:0/4195547180 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f216809b710 con 0x7f21780ffd30 2023-09-23T09:27:46.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.247+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f217810b7a0 con 0x7f21780ffd30 2023-09-23T09:27:46.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.248+0000 7f21757fa700 1 -- 172.21.15.45:0/4195547180 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10031 (secure 0 0 0) 0x7f216809ed30 con 0x7f21780ffd30 2023-09-23T09:27:46.250 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:46.250 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-23T09:27:46.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f21600a15f0 msgr2=0x7f21600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f21600a15f0 0x7f21600a3ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f216c011490 tx=0x7f216c009260 comp rx=0 tx=0).stop 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21780ffd30 msgr2=0x7f2178196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21780ffd30 0x7f2178196af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2168009990 tx=0x7f216801ca40 comp rx=0 tx=0).stop 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 shutdown_connections 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21781018c0 0x7f217819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f21600a15f0 0x7f21600a3ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21780ffd30 0x7f2178196af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 --2- 172.21.15.45:0/4195547180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2178100f20 0x7f2178197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 >> 172.21.15.45:0/4195547180 conn(0x7f21780fb2a0 msgr2=0x7f21780fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 shutdown_connections 2023-09-23T09:27:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.251+0000 7f217e4ff700 1 -- 172.21.15.45:0/4195547180 wait complete. 2023-09-23T09:27:46.254 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2023-09-23T09:27:46.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:27:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 -- 172.21.15.45:0/3524166691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 msgr2=0x7f9478103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 --2- 172.21.15.45:0/3524166691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478103c70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9468009990 tx=0x7f9468009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 -- 172.21.15.45:0/3524166691 shutdown_connections 2023-09-23T09:27:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 --2- 172.21.15.45:0/3524166691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9478106ae0 0x7f9478108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 --2- 172.21.15.45:0/3524166691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478103c70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 --2- 172.21.15.45:0/3524166691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94781041b0 0x7f94781065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 -- 172.21.15.45:0/3524166691 >> 172.21.15.45:0/3524166691 conn(0x7f94780fb250 msgr2=0x7f94780fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 -- 172.21.15.45:0/3524166691 shutdown_connections 2023-09-23T09:27:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.398+0000 7f947f902700 1 -- 172.21.15.45:0/3524166691 wait complete. 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 Processor -- start 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 -- start start 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94781041b0 0x7f9478196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9478106ae0 0x7f947819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947819e5a0 con 0x7f9478101880 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947819e6e0 con 0x7f94781041b0 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947f902700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947819e870 con 0x7f9478106ae0 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58708/0 (socket says 172.21.15.45:58708) 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 -- 172.21.15.45:0/1048889367 learned_addr learned my addr 172.21.15.45:0/1048889367 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:46.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947de9f700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9478106ae0 0x7f947819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947ce9d700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94781041b0 0x7f9478196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 -- 172.21.15.45:0/1048889367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9478106ae0 msgr2=0x7f947819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9478106ae0 0x7f947819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 -- 172.21.15.45:0/1048889367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94781041b0 msgr2=0x7f9478196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94781041b0 0x7f9478196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 -- 172.21.15.45:0/1048889367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94680095f0 con 0x7f9478101880 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947ce9d700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94781041b0 0x7f9478196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.399+0000 7f947d69e700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478196a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9468004730 tx=0x7f9468004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.400+0000 7f946e7fc700 1 -- 172.21.15.45:0/1048889367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946801d070 con 0x7f9478101880 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.400+0000 7f946e7fc700 1 -- 172.21.15.45:0/1048889367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9468004c90 con 0x7f9478101880 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.400+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947819eaf0 con 0x7f9478101880 2023-09-23T09:27:46.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.400+0000 7f946e7fc700 1 -- 172.21.15.45:0/1048889367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468005570 con 0x7f9478101880 2023-09-23T09:27:46.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.400+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947819ef10 con 0x7f9478101880 2023-09-23T09:27:46.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.401+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947810c630 con 0x7f9478101880 2023-09-23T09:27:46.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.402+0000 7f946e7fc700 1 -- 172.21.15.45:0/1048889367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f946800fc70 con 0x7f9478101880 2023-09-23T09:27:46.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.402+0000 7f946e7fc700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f94640a1670 0x7f94640a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.402+0000 7f947ce9d700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f94640a1670 0x7f94640a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.402+0000 7f946e7fc700 1 -- 172.21.15.45:0/1048889367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f94680cdd70 con 0x7f9478101880 2023-09-23T09:27:46.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.403+0000 7f947ce9d700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f94640a1670 0x7f94640a3b30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9474005fd0 tx=0x7f9474005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:46.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.407+0000 7f946e7fc700 1 -- 172.21.15.45:0/1048889367 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f946805aec0 con 0x7f9478101880 2023-09-23T09:27:46.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:46 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/4195547180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:46.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:46 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/4195547180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:46.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:46 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/4195547180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.632+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f947819f160 con 0x7f9478101880 2023-09-23T09:27:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.633+0000 7f946e7fc700 1 -- 172.21.15.45:0/1048889367 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f9468097890 con 0x7f9478101880 2023-09-23T09:27:46.635 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:46.635 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:09.768315+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:27:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f94640a1670 msgr2=0x7f94640a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f94640a1670 0x7f94640a3b30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9474005fd0 tx=0x7f9474005e30 comp rx=0 tx=0).stop 2023-09-23T09:27:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 msgr2=0x7f9478196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478196a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9468004730 tx=0x7f9468004760 comp rx=0 tx=0).stop 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 shutdown_connections 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f94640a1670 0x7f94640a3b30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9478106ae0 0x7f947819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9478101880 0x7f9478196a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 --2- 172.21.15.45:0/1048889367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94781041b0 0x7f9478196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.635+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 >> 172.21.15.45:0/1048889367 conn(0x7f94780fb250 msgr2=0x7f9478102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.636+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 shutdown_connections 2023-09-23T09:27:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.636+0000 7f947f902700 1 -- 172.21.15.45:0/1048889367 wait complete. 2023-09-23T09:27:46.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:27:46.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 -- 172.21.15.45:0/613622654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc101890 msgr2=0x7f9adc103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 --2- 172.21.15.45:0/613622654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc101890 0x7f9adc103c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009990 tx=0x7f9ad0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 -- 172.21.15.45:0/613622654 shutdown_connections 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 --2- 172.21.15.45:0/613622654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9adc106af0 0x7f9adc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 --2- 172.21.15.45:0/613622654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc101890 0x7f9adc103c80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 --2- 172.21.15.45:0/613622654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9adc1041c0 0x7f9adc1065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 -- 172.21.15.45:0/613622654 >> 172.21.15.45:0/613622654 conn(0x7f9adc0fb280 msgr2=0x7f9adc0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 -- 172.21.15.45:0/613622654 shutdown_connections 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 -- 172.21.15.45:0/613622654 wait complete. 2023-09-23T09:27:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.782+0000 7f9ae126f700 1 Processor -- start 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ae126f700 1 -- start start 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ae126f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9adc101890 0x7f9adc196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ae126f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc1041c0 0x7f9adc197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ae126f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9adc106af0 0x7f9adc19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ae126f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19e690 con 0x7f9adc1041c0 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ae126f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19e7d0 con 0x7f9adc106af0 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ae126f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19e960 con 0x7f9adc101890 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc1041c0 0x7f9adc197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc1041c0 0x7f9adc197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58724/0 (socket says 172.21.15.45:58724) 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 -- 172.21.15.45:0/915335008 learned_addr learned my addr 172.21.15.45:0/915335008 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9adb7fe700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9adc106af0 0x7f9adc19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9adaffd700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9adc101890 0x7f9adc196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 -- 172.21.15.45:0/915335008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9adc101890 msgr2=0x7f9adc196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9adc101890 0x7f9adc196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 -- 172.21.15.45:0/915335008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9adc106af0 msgr2=0x7f9adc19e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9adc106af0 0x7f9adc19e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 -- 172.21.15.45:0/915335008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00095f0 con 0x7f9adc1041c0 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.783+0000 7f9ada7fc700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc1041c0 0x7f9adc197050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00c9b0 tx=0x7f9acc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.784+0000 7f9ac3fff700 1 -- 172.21.15.45:0/915335008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc007b00 con 0x7f9adc1041c0 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.784+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc19ec40 con 0x7f9adc1041c0 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.784+0000 7f9ac3fff700 1 -- 172.21.15.45:0/915335008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9acc00ce50 con 0x7f9adc1041c0 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.784+0000 7f9ac3fff700 1 -- 172.21.15.45:0/915335008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc018660 con 0x7f9adc1041c0 2023-09-23T09:27:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.784+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc19f150 con 0x7f9adc1041c0 2023-09-23T09:27:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.784+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc0fce00 con 0x7f9adc1041c0 2023-09-23T09:27:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.786+0000 7f9ac3fff700 1 -- 172.21.15.45:0/915335008 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f9acc00f450 con 0x7f9adc1041c0 2023-09-23T09:27:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.786+0000 7f9ac3fff700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9ac40a15a0 0x7f9ac40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.786+0000 7f9adaffd700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9ac40a15a0 0x7f9ac40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.786+0000 7f9ac3fff700 1 -- 172.21.15.45:0/915335008 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9acc0ccc00 con 0x7f9adc1041c0 2023-09-23T09:27:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.787+0000 7f9adaffd700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9ac40a15a0 0x7f9ac40a3a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b5c0 tx=0x7f9ad0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:46.790+0000 7f9ac3fff700 1 -- 172.21.15.45:0/915335008 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9acc096720 con 0x7f9adc1041c0 2023-09-23T09:27:47.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.010+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9adc10c850 con 0x7f9adc1041c0 2023-09-23T09:27:47.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.010+0000 7f9ac3fff700 1 -- 172.21.15.45:0/915335008 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9acc007e60 con 0x7f9adc1041c0 2023-09-23T09:27:47.012 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:27:47.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9ac40a15a0 msgr2=0x7f9ac40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9ac40a15a0 0x7f9ac40a3a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b5c0 tx=0x7f9ad0005fb0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc1041c0 msgr2=0x7f9adc197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc1041c0 0x7f9adc197050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00c9b0 tx=0x7f9acc00ccc0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 shutdown_connections 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9adc106af0 0x7f9adc19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f9ac40a15a0 0x7f9ac40a3a60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9adc101890 0x7f9adc196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 --2- 172.21.15.45:0/915335008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9adc1041c0 0x7f9adc197050 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.012+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 >> 172.21.15.45:0/915335008 conn(0x7f9adc0fb280 msgr2=0x7f9adc102500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.013+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 shutdown_connections 2023-09-23T09:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.013+0000 7f9ae126f700 1 -- 172.21.15.45:0/915335008 wait complete. 2023-09-23T09:27:47.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T09:27:47.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 -- 172.21.15.45:0/867809248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a0101880 msgr2=0x7ff8a0103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 --2- 172.21.15.45:0/867809248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a0101880 0x7ff8a0103c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff890009990 tx=0x7ff890009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 -- 172.21.15.45:0/867809248 shutdown_connections 2023-09-23T09:27:47.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 --2- 172.21.15.45:0/867809248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8a0106ae0 0x7ff8a0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 --2- 172.21.15.45:0/867809248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a0101880 0x7ff8a0103c70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 --2- 172.21.15.45:0/867809248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a01041b0 0x7ff8a01065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 -- 172.21.15.45:0/867809248 >> 172.21.15.45:0/867809248 conn(0x7ff8a00fb250 msgr2=0x7ff8a00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 -- 172.21.15.45:0/867809248 shutdown_connections 2023-09-23T09:27:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.157+0000 7ff8a588f700 1 -- 172.21.15.45:0/867809248 wait complete. 2023-09-23T09:27:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.158+0000 7ff8a588f700 1 Processor -- start 2023-09-23T09:27:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.158+0000 7ff8a588f700 1 -- start start 2023-09-23T09:27:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff8a588f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8a0101880 0x7ff8a0196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff8a588f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a01041b0 0x7ff8a0196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff8a588f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a0106ae0 0x7ff8a019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff8a588f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a019e5f0 con 0x7ff8a01041b0 2023-09-23T09:27:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff8a588f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a019e730 con 0x7ff8a0101880 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff8a588f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a019e8c0 con 0x7ff8a0106ae0 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a01041b0 0x7ff8a0196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a01041b0 0x7ff8a0196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58738/0 (socket says 172.21.15.45:58738) 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 -- 172.21.15.45:0/1419149292 learned_addr learned my addr 172.21.15.45:0/1419149292 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89effd700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8a0101880 0x7ff8a0196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89f7fe700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a0106ae0 0x7ff8a019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 -- 172.21.15.45:0/1419149292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a0106ae0 msgr2=0x7ff8a019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a0106ae0 0x7ff8a019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 -- 172.21.15.45:0/1419149292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8a0101880 msgr2=0x7ff8a0196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8a0101880 0x7ff8a0196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 -- 172.21.15.45:0/1419149292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900095f0 con 0x7ff8a01041b0 2023-09-23T09:27:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89f7fe700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a0106ae0 0x7ff8a019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89effd700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8a0101880 0x7ff8a0196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.159+0000 7ff89e7fc700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a01041b0 0x7ff8a0196fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff89400e9a0 tx=0x7ff89400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.160+0000 7ff8a488d700 1 -- 172.21.15.45:0/1419149292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894009790 con 0x7ff8a01041b0 2023-09-23T09:27:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.160+0000 7ff8a488d700 1 -- 172.21.15.45:0/1419149292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff894010430 con 0x7ff8a01041b0 2023-09-23T09:27:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.160+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a019eba0 con 0x7ff8a01041b0 2023-09-23T09:27:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.160+0000 7ff8a488d700 1 -- 172.21.15.45:0/1419149292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894018680 con 0x7ff8a01041b0 2023-09-23T09:27:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.160+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a019f0b0 con 0x7ff8a01041b0 2023-09-23T09:27:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.161+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a010c5e0 con 0x7ff8a01041b0 2023-09-23T09:27:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.162+0000 7ff8a488d700 1 -- 172.21.15.45:0/1419149292 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7ff8940105a0 con 0x7ff8a01041b0 2023-09-23T09:27:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.163+0000 7ff8a488d700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff8880a15f0 0x7ff8880a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.163+0000 7ff89effd700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff8880a15f0 0x7ff8880a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.163+0000 7ff8a488d700 1 -- 172.21.15.45:0/1419149292 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff894014070 con 0x7ff8a01041b0 2023-09-23T09:27:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.163+0000 7ff89effd700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff8880a15f0 0x7ff8880a3ab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff89000b2e0 tx=0x7ff890005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:47.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.168+0000 7ff8a488d700 1 -- 172.21.15.45:0/1419149292 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff894051ec0 con 0x7ff8a01041b0 2023-09-23T09:27:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1048889367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/915335008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:47.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:47 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.377+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff8a019f240 con 0x7ff8a01041b0 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.377+0000 7ff8a488d700 1 -- 172.21.15.45:0/1419149292 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff894096660 con 0x7ff8a01041b0 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:27:47.379 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-09-23T09:27:47.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff8880a15f0 msgr2=0x7ff8880a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff8880a15f0 0x7ff8880a3ab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff89000b2e0 tx=0x7ff890005fb0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a01041b0 msgr2=0x7ff8a0196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a01041b0 0x7ff8a0196fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff89400e9a0 tx=0x7ff89400ecb0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 shutdown_connections 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff8880a15f0 0x7ff8880a3ab0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a0106ae0 0x7ff8a019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8a0101880 0x7ff8a0196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 --2- 172.21.15.45:0/1419149292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8a01041b0 0x7ff8a0196fb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 >> 172.21.15.45:0/1419149292 conn(0x7ff8a00fb250 msgr2=0x7ff8a0102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 shutdown_connections 2023-09-23T09:27:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.379+0000 7ff8a588f700 1 -- 172.21.15.45:0/1419149292 wait complete. 2023-09-23T09:27:47.393 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T09:27:47.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T09:27:47.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 -- 172.21.15.45:0/2467992982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 msgr2=0x7fc65c101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2467992982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c101250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc658009990 tx=0x7fc658009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 -- 172.21.15.45:0/2467992982 shutdown_connections 2023-09-23T09:27:47.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2467992982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc65c102940 0x7fc65c109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2467992982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c101250 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2467992982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65c101f80 0x7fc65c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 -- 172.21.15.45:0/2467992982 >> 172.21.15.45:0/2467992982 conn(0x7fc65c0fc3a0 msgr2=0x7fc65c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:47.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 -- 172.21.15.45:0/2467992982 shutdown_connections 2023-09-23T09:27:47.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.524+0000 7fc663cfb700 1 -- 172.21.15.45:0/2467992982 wait complete. 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 Processor -- start 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 -- start start 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc65c101f80 0x7fc65c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65c102940 0x7fc65c073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c073ef0 con 0x7fc65c100e30 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c074060 con 0x7fc65c101f80 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc663cfb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c1a0b60 con 0x7fc65c102940 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc661a97700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc661a97700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58748/0 (socket says 172.21.15.45:58748) 2023-09-23T09:27:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc661a97700 1 -- 172.21.15.45:0/2045619744 learned_addr learned my addr 172.21.15.45:0/2045619744 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc661296700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc65c101f80 0x7fc65c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.525+0000 7fc662298700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65c102940 0x7fc65c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc661a97700 1 -- 172.21.15.45:0/2045619744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65c102940 msgr2=0x7fc65c073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc661a97700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65c102940 0x7fc65c073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc661a97700 1 -- 172.21.15.45:0/2045619744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc65c101f80 msgr2=0x7fc65c073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc661a97700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc65c101f80 0x7fc65c073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc661a97700 1 -- 172.21.15.45:0/2045619744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6580095f0 con 0x7fc65c100e30 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc661296700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc65c101f80 0x7fc65c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc661a97700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c074bd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc658004760 tx=0x7fc658004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc652ffd700 1 -- 172.21.15.45:0/2045619744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65801d070 con 0x7fc65c100e30 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc652ffd700 1 -- 172.21.15.45:0/2045619744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc658004cf0 con 0x7fc65c100e30 2023-09-23T09:27:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c1a0de0 con 0x7fc65c100e30 2023-09-23T09:27:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc652ffd700 1 -- 172.21.15.45:0/2045619744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658003d00 con 0x7fc65c100e30 2023-09-23T09:27:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.526+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c1a1180 con 0x7fc65c100e30 2023-09-23T09:27:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.527+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c108400 con 0x7fc65c100e30 2023-09-23T09:27:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.529+0000 7fc652ffd700 1 -- 172.21.15.45:0/2045619744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fc658005000 con 0x7fc65c100e30 2023-09-23T09:27:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.530+0000 7fc652ffd700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc6480a15a0 0x7fc6480a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.530+0000 7fc661296700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc6480a15a0 0x7fc6480a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.530+0000 7fc652ffd700 1 -- 172.21.15.45:0/2045619744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6580cdf20 con 0x7fc65c100e30 2023-09-23T09:27:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.531+0000 7fc661296700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc6480a15a0 0x7fc6480a3a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005950 tx=0x7fc64c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.534+0000 7fc652ffd700 1 -- 172.21.15.45:0/2045619744 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc658097b60 con 0x7fc65c100e30 2023-09-23T09:27:47.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1048889367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/915335008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:47 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1048889367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/915335008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:27:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:47 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:47.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:47.743+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc65c02d230 con 0x7fc65c100e30 2023-09-23T09:27:48.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:48 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1419149292' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-23T09:27:48.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:48 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2045619744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T09:27:48.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:48 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:48 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1419149292' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-23T09:27:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:48.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:48 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2045619744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T09:27:48.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:48 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:48.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:48 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1419149292' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-23T09:27:48.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:48.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:48.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:48.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:48 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2045619744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T09:27:48.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:48 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:27:48.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.636+0000 7fc652ffd700 1 -- 172.21.15.45:0/2045619744 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc6580300b0 con 0x7fc65c100e30 2023-09-23T09:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc6480a15a0 msgr2=0x7fc6480a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc6480a15a0 0x7fc6480a3a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005950 tx=0x7fc64c00b410 comp rx=0 tx=0).stop 2023-09-23T09:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 msgr2=0x7fc65c074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c074bd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc658004760 tx=0x7fc658004790 comp rx=0 tx=0).stop 2023-09-23T09:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 shutdown_connections 2023-09-23T09:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc6480a15a0 0x7fc6480a3a60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65c102940 0x7fc65c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc65c100e30 0x7fc65c074bd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 --2- 172.21.15.45:0/2045619744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc65c101f80 0x7fc65c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.639+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 >> 172.21.15.45:0/2045619744 conn(0x7fc65c0fc3a0 msgr2=0x7fc65c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.640+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 shutdown_connections 2023-09-23T09:27:48.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.641+0000 7fc663cfb700 1 -- 172.21.15.45:0/2045619744 wait complete. 2023-09-23T09:27:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-09-23T09:27:48.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 -- 172.21.15.45:0/2179474680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 msgr2=0x7f91981078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 --2- 172.21.15.45:0/2179474680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f91981078c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f919000c240 tx=0x7f919000c550 comp rx=0 tx=0).stop 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 -- 172.21.15.45:0/2179474680 shutdown_connections 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 --2- 172.21.15.45:0/2179474680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f91981078c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 --2- 172.21.15.45:0/2179474680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91980ff480 0x7f91980ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 --2- 172.21.15.45:0/2179474680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9198100670 0x7f9198100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 -- 172.21.15.45:0/2179474680 >> 172.21.15.45:0/2179474680 conn(0x7f91980fa9f0 msgr2=0x7f91980fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 -- 172.21.15.45:0/2179474680 shutdown_connections 2023-09-23T09:27:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.796+0000 7f919ceba700 1 -- 172.21.15.45:0/2179474680 wait complete. 2023-09-23T09:27:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 Processor -- start 2023-09-23T09:27:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 -- start start 2023-09-23T09:27:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91980ff480 0x7f9198194a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9198100670 0x7f9198194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f919819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919819c5a0 con 0x7f9198101010 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919819c6e0 con 0x7f9198100670 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f919ceba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919819c870 con 0x7f91980ff480 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f9196d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f919819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f9196d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f919819bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58766/0 (socket says 172.21.15.45:58766) 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f9196d9d700 1 -- 172.21.15.45:0/3315716652 learned_addr learned my addr 172.21.15.45:0/3315716652 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.797+0000 7f9195d9b700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9198100670 0x7f9198194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f9196d9d700 1 -- 172.21.15.45:0/3315716652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91980ff480 msgr2=0x7f9198194a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f9196d9d700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91980ff480 0x7f9198194a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f9196d9d700 1 -- 172.21.15.45:0/3315716652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9198100670 msgr2=0x7f9198194f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f9196d9d700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9198100670 0x7f9198194f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f9196d9d700 1 -- 172.21.15.45:0/3315716652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919000ae10 con 0x7f9198101010 2023-09-23T09:27:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f9196d9d700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f919819bfd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9190004dc0 tx=0x7f9190004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f91877fe700 1 -- 172.21.15.45:0/3315716652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000f040 con 0x7f9198101010 2023-09-23T09:27:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919819caf0 con 0x7f9198101010 2023-09-23T09:27:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f91877fe700 1 -- 172.21.15.45:0/3315716652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91900074e0 con 0x7f9198101010 2023-09-23T09:27:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f91877fe700 1 -- 172.21.15.45:0/3315716652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919001b720 con 0x7f9198101010 2023-09-23T09:27:48.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.798+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919819cf70 con 0x7f9198101010 2023-09-23T09:27:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.799+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919810a500 con 0x7f9198101010 2023-09-23T09:27:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.800+0000 7f91877fe700 1 -- 172.21.15.45:0/3315716652 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f9190007950 con 0x7f9198101010 2023-09-23T09:27:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.801+0000 7f91877fe700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91800a59b0 0x7f91800a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.801+0000 7f91877fe700 1 -- 172.21.15.45:0/3315716652 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f91900cddc0 con 0x7f9198101010 2023-09-23T09:27:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.801+0000 7f919659c700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91800a59b0 0x7f91800a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.802+0000 7f919659c700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91800a59b0 0x7f91800a7e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f91981004d0 tx=0x7f91880096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:48.805+0000 7f91877fe700 1 -- 172.21.15.45:0/3315716652 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9190097770 con 0x7f9198101010 2023-09-23T09:27:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.014+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9198106320 con 0x7f9198101010 2023-09-23T09:27:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.638+0000 7f91877fe700 1 -- 172.21.15.45:0/3315716652 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f919009ad90 con 0x7f9198101010 2023-09-23T09:27:49.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91800a59b0 msgr2=0x7f91800a7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:49.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91800a59b0 0x7f91800a7e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f91981004d0 tx=0x7f91880096c0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 msgr2=0x7f919819bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:49.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f919819bfd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9190004dc0 tx=0x7f9190004df0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 shutdown_connections 2023-09-23T09:27:49.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9198101010 0x7f919819bfd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91800a59b0 0x7f91800a7e70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91980ff480 0x7f9198194a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 --2- 172.21.15.45:0/3315716652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9198100670 0x7f9198194f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.642+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 >> 172.21.15.45:0/3315716652 conn(0x7f91980fa9f0 msgr2=0x7f91980fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.649+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 shutdown_connections 2023-09-23T09:27:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.650+0000 7f919ceba700 1 -- 172.21.15.45:0/3315716652 wait complete. 2023-09-23T09:27:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-09-23T09:27:49.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T09:27:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 -- 172.21.15.45:0/480663455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 msgr2=0x7f50dc1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 --2- 172.21.15.45:0/480663455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 0x7f50dc1017a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f50d400b0a0 tx=0x7f50d400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 -- 172.21.15.45:0/480663455 shutdown_connections 2023-09-23T09:27:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 --2- 172.21.15.45:0/480663455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50dc100900 0x7f50dc1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 --2- 172.21.15.45:0/480663455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 0x7f50dc1017a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 --2- 172.21.15.45:0/480663455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 0x7f50dc1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 -- 172.21.15.45:0/480663455 >> 172.21.15.45:0/480663455 conn(0x7f50dc0fa9f0 msgr2=0x7f50dc0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 -- 172.21.15.45:0/480663455 shutdown_connections 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.806+0000 7f50e4175700 1 -- 172.21.15.45:0/480663455 wait complete. 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 Processor -- start 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 -- start start 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 0x7f50dc103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50dc100900 0x7f50dc102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 0x7f50dc102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc103010 con 0x7f50dc101380 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc103180 con 0x7f50dc100900 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e4175700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc1a0ce0 con 0x7f50dc0fff40 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e1f11700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 0x7f50dc103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e1f11700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 0x7f50dc103cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:60882/0 (socket says 172.21.15.45:60882) 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e1f11700 1 -- 172.21.15.45:0/534768360 learned_addr learned my addr 172.21.15.45:0/534768360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e1710700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50dc100900 0x7f50dc102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e2712700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 0x7f50dc102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e1f11700 1 -- 172.21.15.45:0/534768360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50dc100900 msgr2=0x7f50dc102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e1f11700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50dc100900 0x7f50dc102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.807+0000 7f50e1f11700 1 -- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 msgr2=0x7f50dc102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50e1f11700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 0x7f50dc102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50e1f11700 1 -- 172.21.15.45:0/534768360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d4009d00 con 0x7f50dc0fff40 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50e1f11700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 0x7f50dc103cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50d4007a60 tx=0x7f50d400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50daffd700 1 -- 172.21.15.45:0/534768360 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400e040 con 0x7f50dc0fff40 2023-09-23T09:27:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc1a0f60 con 0x7f50dc0fff40 2023-09-23T09:27:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50daffd700 1 -- 172.21.15.45:0/534768360 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d40094c0 con 0x7f50dc0fff40 2023-09-23T09:27:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50daffd700 1 -- 172.21.15.45:0/534768360 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d40052a0 con 0x7f50dc0fff40 2023-09-23T09:27:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.808+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc1a1330 con 0x7f50dc0fff40 2023-09-23T09:27:49.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.809+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc10b5b0 con 0x7f50dc0fff40 2023-09-23T09:27:49.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.810+0000 7f50daffd700 1 -- 172.21.15.45:0/534768360 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f50d401d470 con 0x7f50dc0fff40 2023-09-23T09:27:49.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.811+0000 7f50daffd700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f50c40a15a0 0x7f50c40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:49.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.811+0000 7f50e1710700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f50c40a15a0 0x7f50c40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:49.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.811+0000 7f50daffd700 1 -- 172.21.15.45:0/534768360 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f50d40cea90 con 0x7f50dc0fff40 2023-09-23T09:27:49.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.811+0000 7f50e1710700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f50c40a15a0 0x7f50c40a3a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009c60 tx=0x7f50d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:49.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:49.814+0000 7f50daffd700 1 -- 172.21.15.45:0/534768360 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f50d40982c0 con 0x7f50dc0fff40 2023-09-23T09:27:50.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:49 smithi098 ceph-mon[102386]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:50.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:49 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2045619744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-23T09:27:50.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:49 smithi098 ceph-mon[102386]: osdmap e80: 12 total, 12 up, 12 in 2023-09-23T09:27:50.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:49 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3315716652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T09:27:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.071+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f50dc10b740 con 0x7f50dc0fff40 2023-09-23T09:27:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:49 smithi045 ceph-mon[98627]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:49 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2045619744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-23T09:27:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:49 smithi045 ceph-mon[98627]: osdmap e80: 12 total, 12 up, 12 in 2023-09-23T09:27:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:49 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3315716652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T09:27:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:49 smithi123 ceph-mon[103577]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:49 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2045619744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-23T09:27:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:49 smithi123 ceph-mon[103577]: osdmap e80: 12 total, 12 up, 12 in 2023-09-23T09:27:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:49 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3315716652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T09:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.656+0000 7f50daffd700 1 -- 172.21.15.45:0/534768360 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f50d409b8e0 con 0x7f50dc0fff40 2023-09-23T09:27:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.660+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f50c40a15a0 msgr2=0x7f50c40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.660+0000 7f50e4175700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f50c40a15a0 0x7f50c40a3a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009c60 tx=0x7f50d0009260 comp rx=0 tx=0).stop 2023-09-23T09:27:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.660+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 msgr2=0x7f50dc103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.660+0000 7f50e4175700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 0x7f50dc103cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50d4007a60 tx=0x7f50d400bd60 comp rx=0 tx=0).stop 2023-09-23T09:27:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 shutdown_connections 2023-09-23T09:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50dc101380 0x7f50dc102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f50c40a15a0 0x7f50c40a3a60 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009c60 tx=0x7f50d0009260 comp rx=0 tx=0).stop 2023-09-23T09:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50dc0fff40 0x7f50dc103cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 --2- 172.21.15.45:0/534768360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50dc100900 0x7f50dc102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 >> 172.21.15.45:0/534768360 conn(0x7f50dc0fa9f0 msgr2=0x7f50dc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 shutdown_connections 2023-09-23T09:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.661+0000 7f50e4175700 1 -- 172.21.15.45:0/534768360 wait complete. 2023-09-23T09:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2023-09-23T09:27:50.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:27:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 -- 172.21.15.45:0/1558230093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af8104200 msgr2=0x7f4af81065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/1558230093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af8104200 0x7f4af81065f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009990 tx=0x7f4aec009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 -- 172.21.15.45:0/1558230093 shutdown_connections 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/1558230093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4af8106b30 0x7f4af8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/1558230093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4af81018d0 0x7f4af8103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/1558230093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af8104200 0x7f4af81065f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 -- 172.21.15.45:0/1558230093 >> 172.21.15.45:0/1558230093 conn(0x7f4af80fb240 msgr2=0x7f4af80fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 -- 172.21.15.45:0/1558230093 shutdown_connections 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.815+0000 7f4afe4a5700 1 -- 172.21.15.45:0/1558230093 wait complete. 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 Processor -- start 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 -- start start 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af81018d0 0x7f4af8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4af8104200 0x7f4af8196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4af8106b30 0x7f4af819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af819e620 con 0x7f4af81018d0 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af819e760 con 0x7f4af8104200 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afe4a5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af819e8f0 con 0x7f4af8106b30 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af81018d0 0x7f4af8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af81018d0 0x7f4af8196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58798/0 (socket says 172.21.15.45:58798) 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 -- 172.21.15.45:0/351334867 learned_addr learned my addr 172.21.15.45:0/351334867 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af77fe700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4af8104200 0x7f4af8196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4afca42700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4af8106b30 0x7f4af819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 -- 172.21.15.45:0/351334867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4af8106b30 msgr2=0x7f4af819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4af8106b30 0x7f4af819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 -- 172.21.15.45:0/351334867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4af8104200 msgr2=0x7f4af8196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4af8104200 0x7f4af8196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.816+0000 7f4af7fff700 1 -- 172.21.15.45:0/351334867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0095f0 con 0x7f4af81018d0 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.817+0000 7f4afca42700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4af8106b30 0x7f4af819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.817+0000 7f4af7fff700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af81018d0 0x7f4af8196aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800e9a0 tx=0x7f4ae800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.817+0000 7f4af57fa700 1 -- 172.21.15.45:0/351334867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8009790 con 0x7f4af81018d0 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.817+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af819ebd0 con 0x7f4af81018d0 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.817+0000 7f4af57fa700 1 -- 172.21.15.45:0/351334867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ae800cbd0 con 0x7f4af81018d0 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.817+0000 7f4af57fa700 1 -- 172.21.15.45:0/351334867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8018680 con 0x7f4af81018d0 2023-09-23T09:27:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.817+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af819f130 con 0x7f4af81018d0 2023-09-23T09:27:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.818+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af80fcdc0 con 0x7f4af81018d0 2023-09-23T09:27:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.819+0000 7f4af57fa700 1 -- 172.21.15.45:0/351334867 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f4ae800cd40 con 0x7f4af81018d0 2023-09-23T09:27:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.820+0000 7f4af57fa700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.820+0000 7f4af77fe700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.820+0000 7f4af57fa700 1 -- 172.21.15.45:0/351334867 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4ae8014070 con 0x7f4af81018d0 2023-09-23T09:27:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.820+0000 7f4af77fe700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00b2e0 tx=0x7f4aec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:50.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:50.825+0000 7f4af57fa700 1 -- 172.21.15.45:0/351334867 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4ae8051ec0 con 0x7f4af81018d0 2023-09-23T09:27:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.040+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4af810c660 con 0x7f4af81018d0 2023-09-23T09:27:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.041+0000 7f4af57fa700 1 -- 172.21.15.45:0/351334867 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22072 (secure 0 0 0) 0x7f4af810c660 con 0x7f4af81018d0 2023-09-23T09:27:51.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:51.044 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:50.073219+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:27:51.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:50 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3315716652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-23T09:27:51.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:50 smithi098 ceph-mon[102386]: osdmap e81: 12 total, 12 up, 12 in 2023-09-23T09:27:51.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:50 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/534768360' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T09:27:51.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:50 smithi098 ceph-mon[102386]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T09:27:51.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:50 smithi098 ceph-mon[102386]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-23T09:27:51.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:50 smithi098 ceph-mon[102386]: osdmap e82: 12 total, 12 up, 12 in 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4ae00a15f0 msgr2=0x7f4ae00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00b2e0 tx=0x7f4aec005fb0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af81018d0 msgr2=0x7f4af8196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af81018d0 0x7f4af8196aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800e9a0 tx=0x7f4ae800ecb0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 shutdown_connections 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4af8106b30 0x7f4af819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4af81018d0 0x7f4af8196aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 --2- 172.21.15.45:0/351334867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4af8104200 0x7f4af8196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 >> 172.21.15.45:0/351334867 conn(0x7f4af80fb240 msgr2=0x7f4af8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 shutdown_connections 2023-09-23T09:27:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.044+0000 7f4afe4a5700 1 -- 172.21.15.45:0/351334867 wait complete. 2023-09-23T09:27:51.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:50 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3315716652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-23T09:27:51.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:50 smithi045 ceph-mon[98627]: osdmap e81: 12 total, 12 up, 12 in 2023-09-23T09:27:51.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:50 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/534768360' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T09:27:51.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:50 smithi045 ceph-mon[98627]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T09:27:51.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:50 smithi045 ceph-mon[98627]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-23T09:27:51.058 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:50 smithi045 ceph-mon[98627]: osdmap e82: 12 total, 12 up, 12 in 2023-09-23T09:27:51.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:27:51.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:50 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3315716652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-23T09:27:51.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:50 smithi123 ceph-mon[103577]: osdmap e81: 12 total, 12 up, 12 in 2023-09-23T09:27:51.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:50 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/534768360' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T09:27:51.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:50 smithi123 ceph-mon[103577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T09:27:51.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:50 smithi123 ceph-mon[103577]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-23T09:27:51.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:50 smithi123 ceph-mon[103577]: osdmap e82: 12 total, 12 up, 12 in 2023-09-23T09:27:51.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 -- 172.21.15.45:0/2431813968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 msgr2=0x7fa7e0108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 --2- 172.21.15.45:0/2431813968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e0108f90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800b0a0 tx=0x7fa7d800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 -- 172.21.15.45:0/2431813968 shutdown_connections 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 --2- 172.21.15.45:0/2431813968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e0108f90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 --2- 172.21.15.45:0/2431813968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e0101940 0x7fa7e0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 --2- 172.21.15.45:0/2431813968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e0104270 0x7fa7e0106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 -- 172.21.15.45:0/2431813968 >> 172.21.15.45:0/2431813968 conn(0x7fa7e00fb250 msgr2=0x7fa7e00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 -- 172.21.15.45:0/2431813968 shutdown_connections 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 -- 172.21.15.45:0/2431813968 wait complete. 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 Processor -- start 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.188+0000 7fa7e6042700 1 -- start start 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e0101940 0x7fa7e0196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e0104270 0x7fa7e0197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e019e6d0 con 0x7fa7e0106ba0 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e019e810 con 0x7fa7e0101940 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e019e9a0 con 0x7fa7e0104270 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e019e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58816/0 (socket says 172.21.15.45:58816) 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 -- 172.21.15.45:0/3167414667 learned_addr learned my addr 172.21.15.45:0/3167414667 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7deffd700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e0104270 0x7fa7e0197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7df7fe700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e0101940 0x7fa7e0196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 -- 172.21.15.45:0/3167414667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e0104270 msgr2=0x7fa7e0197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e0104270 0x7fa7e0197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 -- 172.21.15.45:0/3167414667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e0101940 msgr2=0x7fa7e0196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e0101940 0x7fa7e0196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 -- 172.21.15.45:0/3167414667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8009d00 con 0x7fa7e0106ba0 2023-09-23T09:27:51.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7df7fe700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e0101940 0x7fa7e0196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:27:51.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dffff700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e019e100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800b070 tx=0x7fa7d8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:51.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dcff9700 1 -- 172.21.15.45:0/3167414667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800e040 con 0x7fa7e0106ba0 2023-09-23T09:27:51.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e019ec20 con 0x7fa7e0106ba0 2023-09-23T09:27:51.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dcff9700 1 -- 172.21.15.45:0/3167414667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d8004e80 con 0x7fa7e0106ba0 2023-09-23T09:27:51.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7dcff9700 1 -- 172.21.15.45:0/3167414667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d801b6b0 con 0x7fa7e0106ba0 2023-09-23T09:27:51.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.189+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e019f0a0 con 0x7fa7e0106ba0 2023-09-23T09:27:51.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.190+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e00fcdd0 con 0x7fa7e0106ba0 2023-09-23T09:27:51.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.192+0000 7fa7dcff9700 1 -- 172.21.15.45:0/3167414667 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fa7d801b810 con 0x7fa7e0106ba0 2023-09-23T09:27:51.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.193+0000 7fa7dcff9700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa7cc0a15f0 0x7fa7cc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.193+0000 7fa7df7fe700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa7cc0a15f0 0x7fa7cc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.193+0000 7fa7dcff9700 1 -- 172.21.15.45:0/3167414667 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7d80ce2c0 con 0x7fa7e0106ba0 2023-09-23T09:27:51.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.193+0000 7fa7df7fe700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa7cc0a15f0 0x7fa7cc0a3ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005fd0 tx=0x7fa7c8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:51.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.196+0000 7fa7dcff9700 1 -- 172.21.15.45:0/3167414667 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa7d8097a90 con 0x7fa7e0106ba0 2023-09-23T09:27:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.401+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7e010c840 con 0x7fa7e0106ba0 2023-09-23T09:27:51.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.402+0000 7fa7dcff9700 1 -- 172.21.15.45:0/3167414667 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7fa7d809b0b0 con 0x7fa7e0106ba0 2023-09-23T09:27:51.405 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:51.405 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:51.253687+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:27:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa7cc0a15f0 msgr2=0x7fa7cc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa7cc0a15f0 0x7fa7cc0a3ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005fd0 tx=0x7fa7c8005e20 comp rx=0 tx=0).stop 2023-09-23T09:27:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 msgr2=0x7fa7e019e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e019e100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800b070 tx=0x7fa7d8004a50 comp rx=0 tx=0).stop 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 shutdown_connections 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e0106ba0 0x7fa7e019e100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e0101940 0x7fa7e0196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa7cc0a15f0 0x7fa7cc0a3ab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 --2- 172.21.15.45:0/3167414667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e0104270 0x7fa7e0197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.404+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 >> 172.21.15.45:0/3167414667 conn(0x7fa7e00fb250 msgr2=0x7fa7e01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.405+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 shutdown_connections 2023-09-23T09:27:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.405+0000 7fa7e6042700 1 -- 172.21.15.45:0/3167414667 wait complete. 2023-09-23T09:27:51.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/269696845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc100f10 msgr2=0x7fa3bc101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/269696845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc100f10 0x7fa3bc101370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0099c0 tx=0x7fa3ac009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/269696845 shutdown_connections 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/269696845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3bc1018b0 0x7fa3bc108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/269696845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3bc0ffd20 0x7fa3bc100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/269696845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc100f10 0x7fa3bc101370 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/269696845 >> 172.21.15.45:0/269696845 conn(0x7fa3bc0fb2b0 msgr2=0x7fa3bc0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:51.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/269696845 shutdown_connections 2023-09-23T09:27:51.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.546+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/269696845 wait complete. 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 Processor -- start 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 -- start start 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc0ffd20 0x7fa3bc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3bc100f10 0x7fa3bc19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3bc1018b0 0x7fa3bc19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc19e630 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc19e7a0 con 0x7fa3bc1018b0 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3c1cd5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc19e960 con 0x7fa3bc100f10 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc0ffd20 0x7fa3bc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc0ffd20 0x7fa3bc069b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58826/0 (socket says 172.21.15.45:58826) 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 -- 172.21.15.45:0/3518307823 learned_addr learned my addr 172.21.15.45:0/3518307823 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bbfff700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3bc1018b0 0x7fa3bc19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3baffd700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3bc100f10 0x7fa3bc19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 -- 172.21.15.45:0/3518307823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3bc100f10 msgr2=0x7fa3bc19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3bc100f10 0x7fa3bc19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 -- 172.21.15.45:0/3518307823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3bc1018b0 msgr2=0x7fa3bc19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3bc1018b0 0x7fa3bc19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.547+0000 7fa3bb7fe700 1 -- 172.21.15.45:0/3518307823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0095f0 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.548+0000 7fa3bb7fe700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc0ffd20 0x7fa3bc069b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000e970 tx=0x7fa3b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.548+0000 7fa3b8ff9700 1 -- 172.21.15.45:0/3518307823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0009790 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.548+0000 7fa3b8ff9700 1 -- 172.21.15.45:0/3518307823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3b000c7e0 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.548+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc19ec40 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.548+0000 7fa3b8ff9700 1 -- 172.21.15.45:0/3518307823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0003c70 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.548+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc19f170 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.549+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc106c50 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.550+0000 7fa3b8ff9700 1 -- 172.21.15.45:0/3518307823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7fa3b000cc50 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.551+0000 7fa3b8ff9700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa3a40a1620 0x7fa3a40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:51.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.551+0000 7fa3b8ff9700 1 -- 172.21.15.45:0/3518307823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa3b0014070 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.551+0000 7fa3baffd700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa3a40a1620 0x7fa3a40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:51.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.551+0000 7fa3baffd700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa3a40a1620 0x7fa3a40a3ae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b580 tx=0x7fa3ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:51.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.555+0000 7fa3b8ff9700 1 -- 172.21.15.45:0/3518307823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa3b0096620 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:51.806+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa3bc02d1a0 con 0x7fa3bc0ffd20 2023-09-23T09:27:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: pgmap v250: 97 pgs: 5 active+clean, 32 creating+peering, 60 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-23T09:27:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-23T09:27:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-23T09:27:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/351334867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: osdmap e83: 12 total, 12 up, 12 in 2023-09-23T09:27:51.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:51 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3167414667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: pgmap v250: 97 pgs: 5 active+clean, 32 creating+peering, 60 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-23T09:27:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-23T09:27:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-23T09:27:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/351334867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: osdmap e83: 12 total, 12 up, 12 in 2023-09-23T09:27:52.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:51 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3167414667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: pgmap v250: 97 pgs: 5 active+clean, 32 creating+peering, 60 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:27:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:27:52.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/351334867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:52.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: osdmap e83: 12 total, 12 up, 12 in 2023-09-23T09:27:52.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:51 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3167414667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.672+0000 7fa3b8ff9700 1 -- 172.21.15.45:0/3518307823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa3b0003dd0 con 0x7fa3bc0ffd20 2023-09-23T09:27:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa3a40a1620 msgr2=0x7fa3a40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa3a40a1620 0x7fa3a40a3ae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b580 tx=0x7fa3ac005fb0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc0ffd20 msgr2=0x7fa3bc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc0ffd20 0x7fa3bc069b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000e970 tx=0x7fa3b000ec80 comp rx=0 tx=0).stop 2023-09-23T09:27:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 shutdown_connections 2023-09-23T09:27:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3bc1018b0 0x7fa3bc19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3bc0ffd20 0x7fa3bc069b20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa3a40a1620 0x7fa3a40a3ae0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 --2- 172.21.15.45:0/3518307823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3bc100f10 0x7fa3bc19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 >> 172.21.15.45:0/3518307823 conn(0x7fa3bc0fb2b0 msgr2=0x7fa3bc0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 shutdown_connections 2023-09-23T09:27:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.675+0000 7fa3c1cd5700 1 -- 172.21.15.45:0/3518307823 wait complete. 2023-09-23T09:27:52.692 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-09-23T09:27:52.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-09-23T09:27:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 -- 172.21.15.45:0/3281635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45901018d0 msgr2=0x7f4590108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 --2- 172.21.15.45:0/3281635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45901018d0 0x7f4590108180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f458800b0d0 tx=0x7f458800b3e0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 -- 172.21.15.45:0/3281635640 shutdown_connections 2023-09-23T09:27:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 --2- 172.21.15.45:0/3281635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45901018d0 0x7f4590108180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 --2- 172.21.15.45:0/3281635640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45900ffd40 0x7f4590100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 --2- 172.21.15.45:0/3281635640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4590100f30 0x7f4590101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 -- 172.21.15.45:0/3281635640 >> 172.21.15.45:0/3281635640 conn(0x7f45900fb2b0 msgr2=0x7f45900fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 -- 172.21.15.45:0/3281635640 shutdown_connections 2023-09-23T09:27:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.826+0000 7f4597f39700 1 -- 172.21.15.45:0/3281635640 wait complete. 2023-09-23T09:27:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 Processor -- start 2023-09-23T09:27:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 -- start start 2023-09-23T09:27:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45900ffd40 0x7f4590196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590100f30 0x7f4590197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45901018d0 0x7f459019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019e740 con 0x7f4590100f30 2023-09-23T09:27:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019e880 con 0x7f45900ffd40 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4597f39700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019ea10 con 0x7f45901018d0 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f45954d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590100f30 0x7f4590197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f45954d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590100f30 0x7f4590197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58842/0 (socket says 172.21.15.45:58842) 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f45954d4700 1 -- 172.21.15.45:0/94621342 learned_addr learned my addr 172.21.15.45:0/94621342 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f4595cd5700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45900ffd40 0x7f4590196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f45964d6700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45901018d0 0x7f459019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f45954d4700 1 -- 172.21.15.45:0/94621342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45901018d0 msgr2=0x7f459019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.827+0000 7f45954d4700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45901018d0 0x7f459019e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f45954d4700 1 -- 172.21.15.45:0/94621342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45900ffd40 msgr2=0x7f4590196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f45954d4700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45900ffd40 0x7f4590196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f45954d4700 1 -- 172.21.15.45:0/94621342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4588009d00 con 0x7f4590100f30 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f45954d4700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590100f30 0x7f4590197100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d950 tx=0x7f458c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f4586ffd700 1 -- 172.21.15.45:0/94621342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c00f890 con 0x7f4590100f30 2023-09-23T09:27:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f4586ffd700 1 -- 172.21.15.45:0/94621342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458c010460 con 0x7f4590100f30 2023-09-23T09:27:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459019ecf0 con 0x7f4590100f30 2023-09-23T09:27:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f4586ffd700 1 -- 172.21.15.45:0/94621342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c00e610 con 0x7f4590100f30 2023-09-23T09:27:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.828+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459019f1b0 con 0x7f4590100f30 2023-09-23T09:27:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.829+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459010b560 con 0x7f4590100f30 2023-09-23T09:27:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.830+0000 7f4586ffd700 1 -- 172.21.15.45:0/94621342 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f458c0108d0 con 0x7f4590100f30 2023-09-23T09:27:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.830+0000 7f4586ffd700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a15a0 0x7f457c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.831+0000 7f4595cd5700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a15a0 0x7f457c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.831+0000 7f4586ffd700 1 -- 172.21.15.45:0/94621342 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f458c0ccbe0 con 0x7f4590100f30 2023-09-23T09:27:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.831+0000 7f4595cd5700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a15a0 0x7f457c0a3a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4590100d90 tx=0x7f458000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:52.834+0000 7f4586ffd700 1 -- 172.21.15.45:0/94621342 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f458c096280 con 0x7f4590100f30 2023-09-23T09:27:53.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:52 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] up:active 2023-09-23T09:27:53.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:52 smithi098 ceph-mon[102386]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:53.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:52 smithi098 ceph-mon[102386]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:53.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:52 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3518307823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T09:27:53.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:52 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] up:active 2023-09-23T09:27:53.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:52 smithi045 ceph-mon[98627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:53.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:52 smithi045 ceph-mon[98627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:53.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:52 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3518307823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T09:27:53.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.082+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f459019f3a0 con 0x7f4590100f30 2023-09-23T09:27:53.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:52 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] up:active 2023-09-23T09:27:53.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:52 smithi123 ceph-mon[103577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:53.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:52 smithi123 ceph-mon[103577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:53.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:52 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3518307823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T09:27:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.673+0000 7f4586ffd700 1 -- 172.21.15.45:0/94621342 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f458c0160b0 con 0x7f4590100f30 2023-09-23T09:27:53.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a15a0 msgr2=0x7f457c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:53.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a15a0 0x7f457c0a3a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4590100d90 tx=0x7f458000b410 comp rx=0 tx=0).stop 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590100f30 msgr2=0x7f4590197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590100f30 0x7f4590197100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d950 tx=0x7f458c00dc60 comp rx=0 tx=0).stop 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 shutdown_connections 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a15a0 0x7f457c0a3a60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45901018d0 0x7f459019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45900ffd40 0x7f4590196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.676+0000 7f4597f39700 1 --2- 172.21.15.45:0/94621342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590100f30 0x7f4590197100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.677+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 >> 172.21.15.45:0/94621342 conn(0x7f45900fb2b0 msgr2=0x7f4590068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.677+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 shutdown_connections 2023-09-23T09:27:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.677+0000 7f4597f39700 1 -- 172.21.15.45:0/94621342 wait complete. 2023-09-23T09:27:53.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-23T09:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 -- 172.21.15.45:0/4151858808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0ff130 msgr2=0x7f0cdc0ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/4151858808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0ff130 0x7f0cdc0ff550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009990 tx=0x7f0ccc009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 -- 172.21.15.45:0/4151858808 shutdown_connections 2023-09-23T09:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/4151858808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cdc102b40 0x7f0cdc1093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/4151858808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0ff130 0x7f0cdc0ff550 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/4151858808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cdc1021a0 0x7f0cdc102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 -- 172.21.15.45:0/4151858808 >> 172.21.15.45:0/4151858808 conn(0x7f0cdc0faa40 msgr2=0x7f0cdc0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 -- 172.21.15.45:0/4151858808 shutdown_connections 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.824+0000 7f0ce13b2700 1 -- 172.21.15.45:0/4151858808 wait complete. 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 Processor -- start 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 -- start start 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cdc0ff130 0x7f0cdc196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc1021a0 0x7f0cdc197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cdc102b40 0x7f0cdc19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19e6b0 con 0x7f0cdc1021a0 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19e7f0 con 0x7f0cdc0ff130 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0ce13b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19e980 con 0x7f0cdc102b40 2023-09-23T09:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0cda7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc1021a0 0x7f0cdc197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0cda7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc1021a0 0x7f0cdc197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58856/0 (socket says 172.21.15.45:58856) 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0cda7fc700 1 -- 172.21.15.45:0/558050982 learned_addr learned my addr 172.21.15.45:0/558050982 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0cdaffd700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cdc0ff130 0x7f0cdc196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.825+0000 7f0cdb7fe700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cdc102b40 0x7f0cdc19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cda7fc700 1 -- 172.21.15.45:0/558050982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cdc102b40 msgr2=0x7f0cdc19e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cda7fc700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cdc102b40 0x7f0cdc19e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cda7fc700 1 -- 172.21.15.45:0/558050982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cdc0ff130 msgr2=0x7f0cdc196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cda7fc700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cdc0ff130 0x7f0cdc196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cda7fc700 1 -- 172.21.15.45:0/558050982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc0095f0 con 0x7f0cdc1021a0 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cda7fc700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc1021a0 0x7f0cdc197070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400d950 tx=0x7f0cc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cd3fff700 1 -- 172.21.15.45:0/558050982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400f890 con 0x7f0cdc1021a0 2023-09-23T09:27:53.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cd3fff700 1 -- 172.21.15.45:0/558050982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cc4010460 con 0x7f0cdc1021a0 2023-09-23T09:27:53.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc19ec60 con 0x7f0cdc1021a0 2023-09-23T09:27:53.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0cd3fff700 1 -- 172.21.15.45:0/558050982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400e610 con 0x7f0cdc1021a0 2023-09-23T09:27:53.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.826+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc19f170 con 0x7f0cdc1021a0 2023-09-23T09:27:53.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.827+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc1085b0 con 0x7f0cdc1021a0 2023-09-23T09:27:53.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.828+0000 7f0cd3fff700 1 -- 172.21.15.45:0/558050982 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f0cc40108d0 con 0x7f0cdc1021a0 2023-09-23T09:27:53.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.829+0000 7f0cd3fff700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0cc80a15a0 0x7f0cc80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:53.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.829+0000 7f0cdaffd700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0cc80a15a0 0x7f0cc80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:53.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.829+0000 7f0cd3fff700 1 -- 172.21.15.45:0/558050982 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0cc40ccaf0 con 0x7f0cdc1021a0 2023-09-23T09:27:53.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.829+0000 7f0cdaffd700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0cc80a15a0 0x7f0cc80a3a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00b5c0 tx=0x7f0ccc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:53.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:53.832+0000 7f0cd3fff700 1 -- 172.21.15.45:0/558050982 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0cc4096230 con 0x7f0cdc1021a0 2023-09-23T09:27:54.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:53 smithi098 ceph-mon[102386]: pgmap v252: 97 pgs: 32 active+clean, 32 creating+peering, 33 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2023-09-23T09:27:54.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:53 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3518307823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-23T09:27:54.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:53 smithi098 ceph-mon[102386]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:54.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:53 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/94621342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-23T09:27:54.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:53 smithi045 ceph-mon[98627]: pgmap v252: 97 pgs: 32 active+clean, 32 creating+peering, 33 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2023-09-23T09:27:54.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:53 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3518307823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-23T09:27:54.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:53 smithi045 ceph-mon[98627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:54.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:53 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/94621342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-23T09:27:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.080+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f0cdc10c640 con 0x7f0cdc1021a0 2023-09-23T09:27:54.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:53 smithi123 ceph-mon[103577]: pgmap v252: 97 pgs: 32 active+clean, 32 creating+peering, 33 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2023-09-23T09:27:54.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:53 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3518307823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-23T09:27:54.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:53 smithi123 ceph-mon[103577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:54.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:53 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/94621342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-23T09:27:54.543 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b[137507]: 2023-09-23T09:27:54.263+0000 7fb9f079f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T09:27:54.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.684+0000 7f0cd3fff700 1 -- 172.21.15.45:0/558050982 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0cc40160c0 con 0x7f0cdc1021a0 2023-09-23T09:27:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0cc80a15a0 msgr2=0x7f0cc80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0cc80a15a0 0x7f0cc80a3a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00b5c0 tx=0x7f0ccc005fd0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc1021a0 msgr2=0x7f0cdc197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc1021a0 0x7f0cdc197070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400d950 tx=0x7f0cc400dc60 comp rx=0 tx=0).stop 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 shutdown_connections 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0cc80a15a0 0x7f0cc80a3a60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cdc102b40 0x7f0cdc19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cdc0ff130 0x7f0cdc196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 --2- 172.21.15.45:0/558050982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc1021a0 0x7f0cdc197070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.687+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 >> 172.21.15.45:0/558050982 conn(0x7f0cdc0faa40 msgr2=0x7f0cdc1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.688+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 shutdown_connections 2023-09-23T09:27:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.688+0000 7f0ce13b2700 1 -- 172.21.15.45:0/558050982 wait complete. 2023-09-23T09:27:54.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.844+0000 7f101273a700 1 -- 172.21.15.45:0/1770218652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c0ffd10 msgr2=0x7f100c100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.844+0000 7f101273a700 1 --2- 172.21.15.45:0/1770218652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c0ffd10 0x7f100c100130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40099c0 tx=0x7f0ff4009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 -- 172.21.15.45:0/1770218652 shutdown_connections 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 --2- 172.21.15.45:0/1770218652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f100c1018a0 0x7f100c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 --2- 172.21.15.45:0/1770218652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c0ffd10 0x7f100c100130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 --2- 172.21.15.45:0/1770218652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f100c100f00 0x7f100c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 -- 172.21.15.45:0/1770218652 >> 172.21.15.45:0/1770218652 conn(0x7f100c0fb280 msgr2=0x7f100c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 -- 172.21.15.45:0/1770218652 shutdown_connections 2023-09-23T09:27:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 -- 172.21.15.45:0/1770218652 wait complete. 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 Processor -- start 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.845+0000 7f101273a700 1 -- start start 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f100c0ffd10 0x7f100c068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c100f00 0x7f100c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f100c1018a0 0x7f100c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c19e730 con 0x7f100c100f00 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c19e8a0 con 0x7f100c0ffd10 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c19ea60 con 0x7f100c1018a0 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c100f00 0x7f100c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c100f00 0x7f100c19b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58870/0 (socket says 172.21.15.45:58870) 2023-09-23T09:27:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 -- 172.21.15.45:0/787021562 learned_addr learned my addr 172.21.15.45:0/787021562 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 -- 172.21.15.45:0/787021562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f100c1018a0 msgr2=0x7f100c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100bfff700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f100c0ffd10 0x7f100c068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f1010cd7700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f100c1018a0 0x7f100c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f100c1018a0 0x7f100c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 -- 172.21.15.45:0/787021562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f100c0ffd10 msgr2=0x7f100c068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f100c0ffd10 0x7f100c068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 -- 172.21.15.45:0/787021562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40095f0 con 0x7f100c100f00 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100bfff700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f100c0ffd10 0x7f100c068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f100b7fe700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c100f00 0x7f100c19b950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b530 tx=0x7f0ffc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f10097fa700 1 -- 172.21.15.45:0/787021562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011830 con 0x7f100c100f00 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c19ed40 con 0x7f100c100f00 2023-09-23T09:27:54.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f10097fa700 1 -- 172.21.15.45:0/787021562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ffc011e50 con 0x7f100c100f00 2023-09-23T09:27:54.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f10097fa700 1 -- 172.21.15.45:0/787021562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00f2e0 con 0x7f100c100f00 2023-09-23T09:27:54.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.846+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c19f270 con 0x7f100c100f00 2023-09-23T09:27:54.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.847+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c106c90 con 0x7f100c100f00 2023-09-23T09:27:54.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.848+0000 7f10097fa700 1 -- 172.21.15.45:0/787021562 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f0ffc010580 con 0x7f100c100f00 2023-09-23T09:27:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.849+0000 7f10097fa700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ff80a15a0 0x7f0ff80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.849+0000 7f100bfff700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ff80a15a0 0x7f0ff80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.849+0000 7f10097fa700 1 -- 172.21.15.45:0/787021562 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ffc0cce90 con 0x7f100c100f00 2023-09-23T09:27:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.849+0000 7f100bfff700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ff80a15a0 0x7f0ff80a3a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006010 tx=0x7f0ff4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:54.852+0000 7f10097fa700 1 -- 172.21.15.45:0/787021562 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0ffc096530 con 0x7f100c100f00 2023-09-23T09:27:55.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-mon[102386]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-23T09:27:55.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/94621342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-23T09:27:55.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-mon[102386]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-23T09:27:55.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-mon[102386]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:55.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-mon[102386]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-09-23T09:27:55.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-mon[102386]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-09-23T09:27:55.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:54 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/558050982' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-23T09:27:55.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:54 smithi045 ceph-mon[98627]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-23T09:27:55.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:54 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/94621342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-23T09:27:55.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:54 smithi045 ceph-mon[98627]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-23T09:27:55.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:54 smithi045 ceph-mon[98627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:55.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:54 smithi045 ceph-mon[98627]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-09-23T09:27:55.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:54 smithi045 ceph-mon[98627]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-09-23T09:27:55.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:54 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/558050982' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-23T09:27:55.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:54 smithi123 ceph-mon[103577]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-23T09:27:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:54 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/94621342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-23T09:27:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:54 smithi123 ceph-mon[103577]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-23T09:27:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:54 smithi123 ceph-mon[103577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-23T09:27:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:54 smithi123 ceph-mon[103577]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-09-23T09:27:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:54 smithi123 ceph-mon[103577]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-09-23T09:27:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:54 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/558050982' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-23T09:27:55.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.105+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f100c10b870 con 0x7f100c100f00 2023-09-23T09:27:55.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.253+0000 7f10097fa700 1 -- 172.21.15.45:0/787021562 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0ffc099b50 con 0x7f100c100f00 2023-09-23T09:27:55.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ff80a15a0 msgr2=0x7f0ff80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ff80a15a0 0x7f0ff80a3a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006010 tx=0x7f0ff4000bc0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c100f00 msgr2=0x7f100c19b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c100f00 0x7f100c19b950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b530 tx=0x7f0ffc00b8f0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 shutdown_connections 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ff80a15a0 0x7f0ff80a3a60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f100c1018a0 0x7f100c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f100c0ffd10 0x7f100c068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 --2- 172.21.15.45:0/787021562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f100c100f00 0x7f100c19b950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.256+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 >> 172.21.15.45:0/787021562 conn(0x7f100c0fb280 msgr2=0x7f100c0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.257+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 shutdown_connections 2023-09-23T09:27:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.257+0000 7f101273a700 1 -- 172.21.15.45:0/787021562 wait complete. 2023-09-23T09:27:55.272 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-09-23T09:27:55.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 -- 172.21.15.45:0/2097178604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc101920 msgr2=0x7ff3fc103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2097178604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc101920 0x7ff3fc103d10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009990 tx=0x7ff3ec009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 -- 172.21.15.45:0/2097178604 shutdown_connections 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2097178604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3fc106b80 0x7ff3fc108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2097178604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc101920 0x7ff3fc103d10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2097178604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3fc104250 0x7ff3fc106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 -- 172.21.15.45:0/2097178604 >> 172.21.15.45:0/2097178604 conn(0x7ff3fc0fb2b0 msgr2=0x7ff3fc0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 -- 172.21.15.45:0/2097178604 shutdown_connections 2023-09-23T09:27:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 -- 172.21.15.45:0/2097178604 wait complete. 2023-09-23T09:27:55.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.401+0000 7ff4044a3700 1 Processor -- start 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff4044a3700 1 -- start start 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff4044a3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3fc101920 0x7ff3fc196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff4044a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3fc104250 0x7ff3fc197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff4044a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc106b80 0x7ff3fc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff4044a3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc19e6d0 con 0x7ff3fc106b80 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff4044a3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc19e810 con 0x7ff3fc101920 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff4044a3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc19e9a0 con 0x7ff3fc104250 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc106b80 0x7ff3fc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc106b80 0x7ff3fc19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58886/0 (socket says 172.21.15.45:58886) 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 -- 172.21.15.45:0/2571559039 learned_addr learned my addr 172.21.15.45:0/2571559039 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff40223f700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3fc101920 0x7ff3fc196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff401a3e700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3fc104250 0x7ff3fc197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 -- 172.21.15.45:0/2571559039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3fc104250 msgr2=0x7ff3fc197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3fc104250 0x7ff3fc197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 -- 172.21.15.45:0/2571559039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3fc101920 msgr2=0x7ff3fc196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3fc101920 0x7ff3fc196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 -- 172.21.15.45:0/2571559039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0095f0 con 0x7ff3fc106b80 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff402a40700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc106b80 0x7ff3fc19e100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007970 tx=0x7ff3f4007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.402+0000 7ff3f37fe700 1 -- 172.21.15.45:0/2571559039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff3fc106b80 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.403+0000 7ff3f37fe700 1 -- 172.21.15.45:0/2571559039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3f400f040 con 0x7ff3fc106b80 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.403+0000 7ff3f37fe700 1 -- 172.21.15.45:0/2571559039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4013a20 con 0x7ff3fc106b80 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.403+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc19ec80 con 0x7ff3fc106b80 2023-09-23T09:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.403+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc19f190 con 0x7ff3fc106b80 2023-09-23T09:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.404+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc0fce30 con 0x7ff3fc106b80 2023-09-23T09:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.405+0000 7ff3f37fe700 1 -- 172.21.15.45:0/2571559039 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7ff3f400dd30 con 0x7ff3fc106b80 2023-09-23T09:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.405+0000 7ff3f37fe700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3e80a1620 0x7ff3e80a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.405+0000 7ff40223f700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3e80a1620 0x7ff3e80a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.405+0000 7ff3f37fe700 1 -- 172.21.15.45:0/2571559039 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3f40cca20 con 0x7ff3fc106b80 2023-09-23T09:27:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.406+0000 7ff40223f700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3e80a1620 0x7ff3e80a3ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b5c0 tx=0x7ff3ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.409+0000 7ff3f37fe700 1 -- 172.21.15.45:0/2571559039 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff3f4096160 con 0x7ff3fc106b80 2023-09-23T09:27:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.648+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3fc02d2a0 con 0x7ff3e80a1620 2023-09-23T09:27:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.692+0000 7ff3f37fe700 1 -- 172.21.15.45:0/2571559039 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3fc02d2a0 con 0x7ff3e80a1620 2023-09-23T09:27:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3e80a1620 msgr2=0x7ff3e80a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3e80a1620 0x7ff3e80a3ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b5c0 tx=0x7ff3ec005fb0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc106b80 msgr2=0x7ff3fc19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc106b80 0x7ff3fc19e100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007970 tx=0x7ff3f4007c80 comp rx=0 tx=0).stop 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 shutdown_connections 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3fc106b80 0x7ff3fc19e100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3fc101920 0x7ff3fc196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff3e80a1620 0x7ff3e80a3ae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 --2- 172.21.15.45:0/2571559039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3fc104250 0x7ff3fc197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.694+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 >> 172.21.15.45:0/2571559039 conn(0x7ff3fc0fb2b0 msgr2=0x7ff3fc102590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.695+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 shutdown_connections 2023-09-23T09:27:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.695+0000 7ff4044a3700 1 -- 172.21.15.45:0/2571559039 wait complete. 2023-09-23T09:27:55.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-09-23T09:27:55.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 -- 172.21.15.45:0/1990054435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 msgr2=0x7f0e5c106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 --2- 172.21.15.45:0/1990054435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c106610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009990 tx=0x7f0e50009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 -- 172.21.15.45:0/1990054435 shutdown_connections 2023-09-23T09:27:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 --2- 172.21.15.45:0/1990054435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e5c106b50 0x7f0e5c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 --2- 172.21.15.45:0/1990054435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e5c1018f0 0x7f0e5c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 --2- 172.21.15.45:0/1990054435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c106610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 -- 172.21.15.45:0/1990054435 >> 172.21.15.45:0/1990054435 conn(0x7f0e5c0fb280 msgr2=0x7f0e5c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 -- 172.21.15.45:0/1990054435 shutdown_connections 2023-09-23T09:27:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.836+0000 7f0e61e88700 1 -- 172.21.15.45:0/1990054435 wait complete. 2023-09-23T09:27:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 Processor -- start 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 -- start start 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e5c1018f0 0x7f0e5c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e5c106b50 0x7f0e5c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c19e660 con 0x7f0e5c104220 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c19e7a0 con 0x7f0e5c1018f0 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e61e88700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c19e930 con 0x7f0e5c106b50 2023-09-23T09:27:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e5affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e5affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58916/0 (socket says 172.21.15.45:58916) 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e5affd700 1 -- 172.21.15.45:0/4268717718 learned_addr learned my addr 172.21.15.45:0/4268717718 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.837+0000 7f0e5bfff700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e5c106b50 0x7f0e5c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e5b7fe700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e5c1018f0 0x7f0e5c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e5affd700 1 -- 172.21.15.45:0/4268717718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e5c106b50 msgr2=0x7f0e5c19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e5affd700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e5c106b50 0x7f0e5c19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e5affd700 1 -- 172.21.15.45:0/4268717718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e5c1018f0 msgr2=0x7f0e5c196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e5affd700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e5c1018f0 0x7f0e5c196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e5affd700 1 -- 172.21.15.45:0/4268717718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500095f0 con 0x7f0e5c104220 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e5affd700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c197020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000c00 tx=0x7f0e5000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e58ff9700 1 -- 172.21.15.45:0/4268717718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5001d070 con 0x7f0e5c104220 2023-09-23T09:27:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c19ebb0 con 0x7f0e5c104220 2023-09-23T09:27:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e58ff9700 1 -- 172.21.15.45:0/4268717718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e5000fb50 con 0x7f0e5c104220 2023-09-23T09:27:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e58ff9700 1 -- 172.21.15.45:0/4268717718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50017650 con 0x7f0e5c104220 2023-09-23T09:27:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.838+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c19f0b0 con 0x7f0e5c104220 2023-09-23T09:27:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.839+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c0fce00 con 0x7f0e5c104220 2023-09-23T09:27:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.840+0000 7f0e58ff9700 1 -- 172.21.15.45:0/4268717718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f0e5000fcc0 con 0x7f0e5c104220 2023-09-23T09:27:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.841+0000 7f0e58ff9700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0e440a15a0 0x7f0e440a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.841+0000 7f0e58ff9700 1 -- 172.21.15.45:0/4268717718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e500ce430 con 0x7f0e5c104220 2023-09-23T09:27:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.841+0000 7f0e5b7fe700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0e440a15a0 0x7f0e440a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.841+0000 7f0e5b7fe700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0e440a15a0 0x7f0e440a3a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c005fd0 tx=0x7f0e4c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:55.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:55.844+0000 7f0e58ff9700 1 -- 172.21.15.45:0/4268717718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0e50097c00 con 0x7f0e5c104220 2023-09-23T09:27:56.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: pgmap v253: 97 pgs: 76 active+clean, 21 creating+peering; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/558050982' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] up:active 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: Cluster is now healthy 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/787021562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/787021562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-23T09:27:56.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] up:active 2023-09-23T09:27:56.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:55 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:27:56.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:27:55.654+0000 7f7b93976700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:27:56.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:27:55.654+0000 7f7b93976700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:27:56.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:27:55.654+0000 7f7b93976700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:27:56.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:27:55.654+0000 7f7b93976700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:27:56.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:27:55.654+0000 7f7b93976700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:27:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: pgmap v253: 97 pgs: 76 active+clean, 21 creating+peering; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2023-09-23T09:27:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/558050982' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] up:active 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: Cluster is now healthy 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/787021562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/787021562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] up:active 2023-09-23T09:27:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:55 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:27:56.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.080+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e5c10c600 con 0x7f0e440a15a0 2023-09-23T09:27:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.085+0000 7f0e58ff9700 1 -- 172.21.15.45:0/4268717718 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f0e5c10c600 con 0x7f0e440a15a0 2023-09-23T09:27:56.086 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/_nogroup/sv_0/a2cd32f7-65e5-483f-b9a5-a9f35acd8b0c 2023-09-23T09:27:56.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: pgmap v253: 97 pgs: 76 active+clean, 21 creating+peering; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2023-09-23T09:27:56.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/558050982' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-23T09:27:56.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] up:active 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: Cluster is now healthy 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/787021562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/787021562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] up:active 2023-09-23T09:27:56.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:55 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:27:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0e440a15a0 msgr2=0x7f0e440a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0e440a15a0 0x7f0e440a3a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c005fd0 tx=0x7f0e4c00a560 comp rx=0 tx=0).stop 2023-09-23T09:27:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 msgr2=0x7f0e5c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c197020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000c00 tx=0x7f0e5000f710 comp rx=0 tx=0).stop 2023-09-23T09:27:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 shutdown_connections 2023-09-23T09:27:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0e440a15a0 0x7f0e440a3a60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e5c106b50 0x7f0e5c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e5c1018f0 0x7f0e5c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 --2- 172.21.15.45:0/4268717718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5c104220 0x7f0e5c197020 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 >> 172.21.15.45:0/4268717718 conn(0x7f0e5c0fb280 msgr2=0x7f0e5c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:56.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 shutdown_connections 2023-09-23T09:27:56.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.087+0000 7f0e61e88700 1 -- 172.21.15.45:0/4268717718 wait complete. 2023-09-23T09:27:56.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 -- 172.21.15.45:0/2989145227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 msgr2=0x7f8a481013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 --2- 172.21.15.45:0/2989145227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a481013a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a440099c0 tx=0x7f8a44009cd0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 -- 172.21.15.45:0/2989145227 shutdown_connections 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 --2- 172.21.15.45:0/2989145227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a481018e0 0x7f8a48108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 --2- 172.21.15.45:0/2989145227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a480ffd50 0x7f8a48100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 --2- 172.21.15.45:0/2989145227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a481013a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 -- 172.21.15.45:0/2989145227 >> 172.21.15.45:0/2989145227 conn(0x7f8a480fb2a0 msgr2=0x7f8a480fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 -- 172.21.15.45:0/2989145227 shutdown_connections 2023-09-23T09:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 -- 172.21.15.45:0/2989145227 wait complete. 2023-09-23T09:27:56.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 Processor -- start 2023-09-23T09:27:56.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.230+0000 7f8a506d3700 1 -- start start 2023-09-23T09:27:56.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a480ffd50 0x7f8a481959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a48195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a481018e0 0x7f8a4819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819d570 con 0x7f8a48100f40 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819d6b0 con 0x7f8a481018e0 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819d840 con 0x7f8a480ffd50 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a48195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a48195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58930/0 (socket says 172.21.15.45:58930) 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 -- 172.21.15.45:0/50298779 learned_addr learned my addr 172.21.15.45:0/50298779 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4e46f700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a480ffd50 0x7f8a481959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 -- 172.21.15.45:0/50298779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a480ffd50 msgr2=0x7f8a481959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a480ffd50 0x7f8a481959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 -- 172.21.15.45:0/50298779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a481018e0 msgr2=0x7f8a4819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4ec70700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a481018e0 0x7f8a4819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a481018e0 0x7f8a4819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 -- 172.21.15.45:0/50298779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a440095f0 con 0x7f8a48100f40 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a4dc6e700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a48195f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009fd0 tx=0x7f8a4400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a3f7fe700 1 -- 172.21.15.45:0/50298779 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401d070 con 0x7f8a48100f40 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a3f7fe700 1 -- 172.21.15.45:0/50298779 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a440054f0 con 0x7f8a48100f40 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4819dac0 con 0x7f8a48100f40 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a3f7fe700 1 -- 172.21.15.45:0/50298779 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44004bd0 con 0x7f8a48100f40 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.231+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4819df40 con 0x7f8a48100f40 2023-09-23T09:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.232+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4810b560 con 0x7f8a48100f40 2023-09-23T09:27:56.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.233+0000 7f8a3f7fe700 1 -- 172.21.15.45:0/50298779 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f8a4400fb80 con 0x7f8a48100f40 2023-09-23T09:27:56.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.234+0000 7f8a3f7fe700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8a340a1620 0x7f8a340a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.234+0000 7f8a3f7fe700 1 -- 172.21.15.45:0/50298779 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8a440cdf90 con 0x7f8a48100f40 2023-09-23T09:27:56.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.234+0000 7f8a4e46f700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8a340a1620 0x7f8a340a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.234+0000 7f8a4e46f700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8a340a1620 0x7f8a340a3ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a48100da0 tx=0x7f8a3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:56.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.238+0000 7f8a3f7fe700 1 -- 172.21.15.45:0/50298779 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8a44097760 con 0x7f8a48100f40 2023-09-23T09:27:56.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.473+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a48061260 con 0x7f8a340a1620 2023-09-23T09:27:56.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.498+0000 7f8a3f7fe700 1 -- 172.21.15.45:0/50298779 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8a48061260 con 0x7f8a340a1620 2023-09-23T09:27:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8a340a1620 msgr2=0x7f8a340a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8a340a1620 0x7f8a340a3ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a48100da0 tx=0x7f8a3800b410 comp rx=0 tx=0).stop 2023-09-23T09:27:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 msgr2=0x7f8a48195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a48195f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009fd0 tx=0x7f8a4400f690 comp rx=0 tx=0).stop 2023-09-23T09:27:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 shutdown_connections 2023-09-23T09:27:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a481018e0 0x7f8a4819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8a340a1620 0x7f8a340a3ae0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a480ffd50 0x7f8a481959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 --2- 172.21.15.45:0/50298779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a48100f40 0x7f8a48195f30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 >> 172.21.15.45:0/50298779 conn(0x7f8a480fb2a0 msgr2=0x7f8a480fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 shutdown_connections 2023-09-23T09:27:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.500+0000 7f8a506d3700 1 -- 172.21.15.45:0/50298779 wait complete. 2023-09-23T09:27:56.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-09-23T09:27:56.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2477064211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c641018a0 msgr2=0x7f2c64108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2477064211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c641018a0 0x7f2c64108150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00b0d0 tx=0x7f2c5c00b3e0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2477064211 shutdown_connections 2023-09-23T09:27:56.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2477064211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c641018a0 0x7f2c64108150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2477064211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c640ffd10 0x7f2c64100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2477064211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c64100f00 0x7f2c64101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2477064211 >> 172.21.15.45:0/2477064211 conn(0x7f2c640fb280 msgr2=0x7f2c640fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:56.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2477064211 shutdown_connections 2023-09-23T09:27:56.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.640+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2477064211 wait complete. 2023-09-23T09:27:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 Processor -- start 2023-09-23T09:27:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 -- start start 2023-09-23T09:27:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c640ffd10 0x7f2c64196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64100f00 0x7f2c641971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c641018a0 0x7f2c6419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6419e800 con 0x7f2c64100f00 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6419e970 con 0x7f2c640ffd10 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c6a0dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6419eb30 con 0x7f2c641018a0 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64100f00 0x7f2c641971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64100f00 0x7f2c641971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58946/0 (socket says 172.21.15.45:58946) 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.641+0000 7f2c62ffd700 1 -- 172.21.15.45:0/2112318189 learned_addr learned my addr 172.21.15.45:0/2112318189 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c637fe700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c640ffd10 0x7f2c64196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c63fff700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c641018a0 0x7f2c6419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c62ffd700 1 -- 172.21.15.45:0/2112318189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c641018a0 msgr2=0x7f2c6419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c62ffd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c641018a0 0x7f2c6419e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c62ffd700 1 -- 172.21.15.45:0/2112318189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c640ffd10 msgr2=0x7f2c64196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c62ffd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c640ffd10 0x7f2c64196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c62ffd700 1 -- 172.21.15.45:0/2112318189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009d00 con 0x7f2c64100f00 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c637fe700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c640ffd10 0x7f2c64196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c62ffd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64100f00 0x7f2c641971d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d950 tx=0x7f2c5800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58009b40 con 0x7f2c64100f00 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c58009ca0 con 0x7f2c64100f00 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6419ee10 con 0x7f2c64100f00 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800f620 con 0x7f2c64100f00 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.642+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6419f360 con 0x7f2c64100f00 2023-09-23T09:27:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.643+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6410b5c0 con 0x7f2c64100f00 2023-09-23T09:27:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.644+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 136192+0+0 (secure 0 0 0) 0x7f2c58010460 con 0x7f2c64100f00 2023-09-23T09:27:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.645+0000 7f2c60ff9700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2c4c0a1620 0x7f2c4c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.645+0000 7f2c637fe700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2c4c0a1620 0x7f2c4c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.645+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c580cd100 con 0x7f2c64100f00 2023-09-23T09:27:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.645+0000 7f2c637fe700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2c4c0a1620 0x7f2c4c0a3ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c64197b60 tx=0x7f2c54008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.649+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2c580967a0 con 0x7f2c64100f00 2023-09-23T09:27:56.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.701+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f2c58096b70 con 0x7f2c64100f00 2023-09-23T09:27:56.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:56 smithi045 ceph-mon[98627]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:56.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:56 smithi045 ceph-mon[98627]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:56.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.895+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c640686b0 con 0x7f2c4c0a1620 2023-09-23T09:27:56.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.900+0000 7f2c60ff9700 1 -- 172.21.15.45:0/2112318189 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2c640686b0 con 0x7f2c4c0a1620 2023-09-23T09:27:56.901 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/_nogroup/sv_1/03909e3e-9314-4e2a-91e9-07669c01496d 2023-09-23T09:27:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2c4c0a1620 msgr2=0x7f2c4c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2c4c0a1620 0x7f2c4c0a3ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c64197b60 tx=0x7f2c54008040 comp rx=0 tx=0).stop 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64100f00 msgr2=0x7f2c641971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64100f00 0x7f2c641971d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d950 tx=0x7f2c5800dd10 comp rx=0 tx=0).stop 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 shutdown_connections 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2c4c0a1620 0x7f2c4c0a3ae0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c641018a0 0x7f2c6419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c640ffd10 0x7f2c64196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 --2- 172.21.15.45:0/2112318189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64100f00 0x7f2c641971d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 >> 172.21.15.45:0/2112318189 conn(0x7f2c640fb280 msgr2=0x7f2c64067fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 shutdown_connections 2023-09-23T09:27:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:56.902+0000 7f2c6a0dd700 1 -- 172.21.15.45:0/2112318189 wait complete. 2023-09-23T09:27:56.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:27:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 -- 172.21.15.45:0/1189689983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 msgr2=0x7f7bec101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/1189689983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec101260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009990 tx=0x7f7be8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 -- 172.21.15.45:0/1189689983 shutdown_connections 2023-09-23T09:27:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/1189689983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bec102950 0x7f7bec109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/1189689983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec101260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/1189689983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec101f90 0x7f7bec102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 -- 172.21.15.45:0/1189689983 >> 172.21.15.45:0/1189689983 conn(0x7f7bec0fc3d0 msgr2=0x7f7bec0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 -- 172.21.15.45:0/1189689983 shutdown_connections 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 -- 172.21.15.45:0/1189689983 wait complete. 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 Processor -- start 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.039+0000 7f7bf44a2700 1 -- start start 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bec101f90 0x7f7bec197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec102950 0x7f7bec19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec19e640 con 0x7f7bec100e40 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec19e780 con 0x7f7bec101f90 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec19e910 con 0x7f7bec102950 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58958/0 (socket says 172.21.15.45:58958) 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 -- 172.21.15.45:0/250118943 learned_addr learned my addr 172.21.15.45:0/250118943 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 -- 172.21.15.45:0/250118943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec102950 msgr2=0x7f7bec19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec102950 0x7f7bec19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 -- 172.21.15.45:0/250118943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bec101f90 msgr2=0x7f7bec197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bec101f90 0x7f7bec197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 -- 172.21.15.45:0/250118943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be80095f0 con 0x7f7bec100e40 2023-09-23T09:27:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf223e700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec196ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7be8004760 tx=0x7f7be8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7be37fe700 1 -- 172.21.15.45:0/250118943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be801d070 con 0x7f7bec100e40 2023-09-23T09:27:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7be37fe700 1 -- 172.21.15.45:0/250118943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be8004200 con 0x7f7bec100e40 2023-09-23T09:27:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec19eb90 con 0x7f7bec100e40 2023-09-23T09:27:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7be37fe700 1 -- 172.21.15.45:0/250118943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80049f0 con 0x7f7bec100e40 2023-09-23T09:27:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.040+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec19f090 con 0x7f7bec100e40 2023-09-23T09:27:57.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:56 smithi098 ceph-mon[102386]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:57.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:56 smithi098 ceph-mon[102386]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.041+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec107d80 con 0x7f7bec100e40 2023-09-23T09:27:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.042+0000 7f7be37fe700 1 -- 172.21.15.45:0/250118943 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f7be8003d40 con 0x7f7bec100e40 2023-09-23T09:27:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.043+0000 7f7be37fe700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7bd80a16f0 0x7f7bd80a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.043+0000 7f7bf1a3d700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7bd80a16f0 0x7f7bd80a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.043+0000 7f7be37fe700 1 -- 172.21.15.45:0/250118943 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7be80cddd0 con 0x7f7bec100e40 2023-09-23T09:27:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.043+0000 7f7bf1a3d700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7bd80a16f0 0x7f7bd80a3bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00a990 tx=0x7f7bdc005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.048+0000 7f7be37fe700 1 -- 172.21.15.45:0/250118943 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7be80974a0 con 0x7f7bec100e40 2023-09-23T09:27:57.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:56 smithi123 ceph-mon[103577]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:57.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:56 smithi123 ceph-mon[103577]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.297+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bec10c640 con 0x7f7bec100e40 2023-09-23T09:27:57.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.297+0000 7f7be37fe700 1 -- 172.21.15.45:0/250118943 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7f7bec10c640 con 0x7f7bec100e40 2023-09-23T09:27:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:57.299 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:27:57.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7bd80a16f0 msgr2=0x7f7bd80a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7bd80a16f0 0x7f7bd80a3bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00a990 tx=0x7f7bdc005c80 comp rx=0 tx=0).stop 2023-09-23T09:27:57.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 msgr2=0x7f7bec196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec196ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7be8004760 tx=0x7f7be8009f90 comp rx=0 tx=0).stop 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 shutdown_connections 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7bd80a16f0 0x7f7bd80a3bb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec102950 0x7f7bec19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bec100e40 0x7f7bec196ac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 --2- 172.21.15.45:0/250118943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bec101f90 0x7f7bec197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.299+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 >> 172.21.15.45:0/250118943 conn(0x7f7bec0fc3d0 msgr2=0x7f7bec0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.300+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 shutdown_connections 2023-09-23T09:27:57.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.300+0000 7f7bf44a2700 1 -- 172.21.15.45:0/250118943 wait complete. 2023-09-23T09:27:57.303 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:27:57.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:27:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 -- 172.21.15.45:0/2320259283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630102930 msgr2=0x7fe630109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 --2- 172.21.15.45:0/2320259283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630102930 0x7fe630109250 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b0a0 tx=0x7fe62800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 -- 172.21.15.45:0/2320259283 shutdown_connections 2023-09-23T09:27:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 --2- 172.21.15.45:0/2320259283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630102930 0x7fe630109250 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 --2- 172.21.15.45:0/2320259283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe630100e20 0x7fe630101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 --2- 172.21.15.45:0/2320259283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630101f70 0x7fe6301023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 -- 172.21.15.45:0/2320259283 >> 172.21.15.45:0/2320259283 conn(0x7fe6300fc370 msgr2=0x7fe6300fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 -- 172.21.15.45:0/2320259283 shutdown_connections 2023-09-23T09:27:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 -- 172.21.15.45:0/2320259283 wait complete. 2023-09-23T09:27:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.439+0000 7fe635a3f700 1 Processor -- start 2023-09-23T09:27:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe635a3f700 1 -- start start 2023-09-23T09:27:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe635a3f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe630100e20 0x7fe630074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe635a3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630101f70 0x7fe630073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe635a3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630102930 0x7fe630073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe635a3f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe630073ef0 con 0x7fe630101f70 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe635a3f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe630074060 con 0x7fe630100e20 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe635a3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6301a0c30 con 0x7fe630102930 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630101f70 0x7fe630073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630101f70 0x7fe630073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58974/0 (socket says 172.21.15.45:58974) 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 -- 172.21.15.45:0/3379279953 learned_addr learned my addr 172.21.15.45:0/3379279953 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62effd700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe630100e20 0x7fe630074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62f7fe700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630102930 0x7fe630073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 -- 172.21.15.45:0/3379279953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630102930 msgr2=0x7fe630073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630102930 0x7fe630073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 -- 172.21.15.45:0/3379279953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe630100e20 msgr2=0x7fe630074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe630100e20 0x7fe630074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 -- 172.21.15.45:0/3379279953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628009d00 con 0x7fe630101f70 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe62e7fc700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630101f70 0x7fe630073210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe62400d980 tx=0x7fe62400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.440+0000 7fe634a3d700 1 -- 172.21.15.45:0/3379279953 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624004a30 con 0x7fe630101f70 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.441+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6301a0f10 con 0x7fe630101f70 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.441+0000 7fe634a3d700 1 -- 172.21.15.45:0/3379279953 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6240054e0 con 0x7fe630101f70 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.441+0000 7fe634a3d700 1 -- 172.21.15.45:0/3379279953 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400fbc0 con 0x7fe630101f70 2023-09-23T09:27:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.441+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6301a1390 con 0x7fe630101f70 2023-09-23T09:27:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.441+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63010c650 con 0x7fe630101f70 2023-09-23T09:27:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.442+0000 7fe634a3d700 1 -- 172.21.15.45:0/3379279953 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fe624004c20 con 0x7fe630101f70 2023-09-23T09:27:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.443+0000 7fe634a3d700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe6180a16f0 0x7fe6180a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.443+0000 7fe62effd700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe6180a16f0 0x7fe6180a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.443+0000 7fe634a3d700 1 -- 172.21.15.45:0/3379279953 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe6240ccc60 con 0x7fe630101f70 2023-09-23T09:27:57.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.444+0000 7fe62effd700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe6180a16f0 0x7fe6180a3bb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe620009c60 tx=0x7fe620009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.448+0000 7fe634a3d700 1 -- 172.21.15.45:0/3379279953 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe6240962a0 con 0x7fe630101f70 2023-09-23T09:27:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.654+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe630107c70 con 0x7fe630101f70 2023-09-23T09:27:57.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.655+0000 7fe634a3d700 1 -- 172.21.15.45:0/3379279953 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7fe624017020 con 0x7fe630101f70 2023-09-23T09:27:57.657 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:57.657 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:51.253687+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:27:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.657+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe6180a16f0 msgr2=0x7fe6180a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.657+0000 7fe635a3f700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe6180a16f0 0x7fe6180a3bb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe620009c60 tx=0x7fe620009260 comp rx=0 tx=0).stop 2023-09-23T09:27:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.657+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630101f70 msgr2=0x7fe630073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.657+0000 7fe635a3f700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630101f70 0x7fe630073210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe62400d980 tx=0x7fe62400dc90 comp rx=0 tx=0).stop 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.657+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 shutdown_connections 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.658+0000 7fe635a3f700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe6180a16f0 0x7fe6180a3bb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.658+0000 7fe635a3f700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630102930 0x7fe630073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.658+0000 7fe635a3f700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe630100e20 0x7fe630074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.658+0000 7fe635a3f700 1 --2- 172.21.15.45:0/3379279953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe630101f70 0x7fe630073210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.658+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 >> 172.21.15.45:0/3379279953 conn(0x7fe6300fc370 msgr2=0x7fe6300fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.658+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 shutdown_connections 2023-09-23T09:27:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.658+0000 7fe635a3f700 1 -- 172.21.15.45:0/3379279953 wait complete. 2023-09-23T09:27:57.672 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T09:27:57.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T09:27:57.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 -- 172.21.15.45:0/1146338752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c1018e0 msgr2=0x7f215c108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 --2- 172.21.15.45:0/1146338752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c1018e0 0x7f215c108190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f215400b0d0 tx=0x7f215400b3e0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 -- 172.21.15.45:0/1146338752 shutdown_connections 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 --2- 172.21.15.45:0/1146338752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c1018e0 0x7f215c108190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 --2- 172.21.15.45:0/1146338752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ffd50 0x7f215c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 --2- 172.21.15.45:0/1146338752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c100f40 0x7f215c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 -- 172.21.15.45:0/1146338752 >> 172.21.15.45:0/1146338752 conn(0x7f215c0fb2a0 msgr2=0x7f215c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 -- 172.21.15.45:0/1146338752 shutdown_connections 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.797+0000 7f2162d05700 1 -- 172.21.15.45:0/1146338752 wait complete. 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 Processor -- start 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 -- start start 2023-09-23T09:27:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ffd50 0x7f215c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c100f40 0x7f215c19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c1018e0 0x7f215c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c19e6f0 con 0x7f215c100f40 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c19e860 con 0x7f215c0ffd50 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2162d05700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c19ea20 con 0x7f215c1018e0 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c100f40 0x7f215c19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c100f40 0x7f215c19b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58988/0 (socket says 172.21.15.45:58988) 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 -- 172.21.15.45:0/2207726967 learned_addr learned my addr 172.21.15.45:0/2207726967 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f21612a2700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c1018e0 0x7f215c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2160aa1700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ffd50 0x7f215c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 -- 172.21.15.45:0/2207726967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c1018e0 msgr2=0x7f215c19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c1018e0 0x7f215c19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 -- 172.21.15.45:0/2207726967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ffd50 msgr2=0x7f215c069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ffd50 0x7f215c069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 -- 172.21.15.45:0/2207726967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154009d00 con 0x7f215c100f40 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f2160aa1700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ffd50 0x7f215c069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.798+0000 7f215bfff700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c100f40 0x7f215c19b910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9a0 tx=0x7f214800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.799+0000 7f2159ffb700 1 -- 172.21.15.45:0/2207726967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148009790 con 0x7f215c100f40 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.799+0000 7f2159ffb700 1 -- 172.21.15.45:0/2207726967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f214800cbd0 con 0x7f215c100f40 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.799+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c19ed00 con 0x7f215c100f40 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.799+0000 7f2159ffb700 1 -- 172.21.15.45:0/2207726967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148018680 con 0x7f215c100f40 2023-09-23T09:27:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.799+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c19f200 con 0x7f215c100f40 2023-09-23T09:27:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.800+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c1071f0 con 0x7f215c100f40 2023-09-23T09:27:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.801+0000 7f2159ffb700 1 -- 172.21.15.45:0/2207726967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f214800cd40 con 0x7f215c100f40 2023-09-23T09:27:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.801+0000 7f2159ffb700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f214c0a16f0 0x7f214c0a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.801+0000 7f2160aa1700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f214c0a16f0 0x7f214c0a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:57.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.801+0000 7f2159ffb700 1 -- 172.21.15.45:0/2207726967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2148014070 con 0x7f215c100f40 2023-09-23T09:27:57.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.802+0000 7f2160aa1700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f214c0a16f0 0x7f214c0a3bb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2150005fd0 tx=0x7f2150005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:57.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:57.806+0000 7f2159ffb700 1 -- 172.21.15.45:0/2207726967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2148051ec0 con 0x7f215c100f40 2023-09-23T09:27:58.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.004+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f215c02d2e0 con 0x7f214c0a16f0 2023-09-23T09:27:58.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.017+0000 7f2159ffb700 1 -- 172.21.15.45:0/2207726967 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335925 (secure 0 0 0) 0x7f215c02d2e0 con 0x7f214c0a16f0 2023-09-23T09:27:58.022 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:58.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f214c0a16f0 msgr2=0x7f214c0a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:58.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f214c0a16f0 0x7f214c0a3bb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2150005fd0 tx=0x7f2150005e40 comp rx=0 tx=0).stop 2023-09-23T09:27:58.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c100f40 msgr2=0x7f215c19b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:58.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c100f40 0x7f215c19b910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9a0 tx=0x7f214800ecb0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 shutdown_connections 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f214c0a16f0 0x7f214c0a3bb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c1018e0 0x7f215c19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ffd50 0x7f215c069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 --2- 172.21.15.45:0/2207726967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f215c100f40 0x7f215c19b910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 >> 172.21.15.45:0/2207726967 conn(0x7f215c0fb2a0 msgr2=0x7f215c104330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 shutdown_connections 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.020+0000 7f2162d05700 1 -- 172.21.15.45:0/2207726967 wait complete. 2023-09-23T09:27:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-09-23T09:27:58.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:57 smithi045 ceph-mon[98627]: pgmap v254: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2023-09-23T09:27:58.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:57 smithi045 ceph-mon[98627]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:58.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:57 smithi045 ceph-mon[98627]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-23T09:27:58.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:57 smithi045 ceph-mon[98627]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:58.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:57 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/250118943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:58.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:57 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3379279953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:58.035 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-09-23T09:27:56.322409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594878,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":143,"num_read_kb":126,"num_write":276,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":238,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334964,"kb_used_data":10708,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606860,"statfs":{"total":1151940427776,"available":1151597424640,"internally_reserved":0,"allocated":10964992,"data_stored":6900928,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4510,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":95,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.682123"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735132+0000","last_change":"2023-09-23T09:27:50.270599+0000","last_active":"2023-09-23T09:27:51.735132+0000","last_peered":"2023-09-23T09:27:51.735132+0000","last_clean":"2023-09-23T09:27:51.735132+0000","last_became_active":"2023-09-23T09:27:50.269926+0000","last_became_peered":"2023-09-23T09:27:50.269926+0000","last_unstale":"2023-09-23T09:27:51.735132+0000","last_undegraded":"2023-09-23T09:27:51.735132+0000","last_fullsized":"2023-09-23T09:27:51.735132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:12:08.908188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733803+0000","last_change":"2023-09-23T09:27:50.270451+0000","last_active":"2023-09-23T09:27:51.733803+0000","last_peered":"2023-09-23T09:27:51.733803+0000","last_clean":"2023-09-23T09:27:51.733803+0000","last_became_active":"2023-09-23T09:27:50.270055+0000","last_became_peered":"2023-09-23T09:27:50.270055+0000","last_unstale":"2023-09-23T09:27:51.733803+0000","last_undegraded":"2023-09-23T09:27:51.733803+0000","last_fullsized":"2023-09-23T09:27:51.733803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:56:43.388109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734671+0000","last_change":"2023-09-23T09:27:50.278101+0000","last_active":"2023-09-23T09:27:51.734671+0000","last_peered":"2023-09-23T09:27:51.734671+0000","last_clean":"2023-09-23T09:27:51.734671+0000","last_became_active":"2023-09-23T09:27:50.277739+0000","last_became_peered":"2023-09-23T09:27:50.277739+0000","last_unstale":"2023-09-23T09:27:51.734671+0000","last_undegraded":"2023-09-23T09:27:51.734671+0000","last_fullsized":"2023-09-23T09:27:51.734671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:52:31.582623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733930+0000","last_change":"2023-09-23T09:27:50.265962+0000","last_active":"2023-09-23T09:27:51.733930+0000","last_peered":"2023-09-23T09:27:51.733930+0000","last_clean":"2023-09-23T09:27:51.733930+0000","last_became_active":"2023-09-23T09:27:50.265681+0000","last_became_peered":"2023-09-23T09:27:50.265681+0000","last_unstale":"2023-09-23T09:27:51.733930+0000","last_undegraded":"2023-09-23T09:27:51.733930+0000","last_fullsized":"2023-09-23T09:27:51.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:12.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734705+0000","last_change":"2023-09-23T09:27:50.270356+0000","last_active":"2023-09-23T09:27:51.734705+0000","last_peered":"2023-09-23T09:27:51.734705+0000","last_clean":"2023-09-23T09:27:51.734705+0000","last_became_active":"2023-09-23T09:27:50.270124+0000","last_became_peered":"2023-09-23T09:27:50.270124+0000","last_unstale":"2023-09-23T09:27:51.734705+0000","last_undegraded":"2023-09-23T09:27:51.734705+0000","last_fullsized":"2023-09-23T09:27:51.734705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:14:00.508926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398274+0000","last_change":"2023-09-23T09:27:50.276122+0000","last_active":"2023-09-23T09:27:51.398274+0000","last_peered":"2023-09-23T09:27:51.398274+0000","last_clean":"2023-09-23T09:27:51.398274+0000","last_became_active":"2023-09-23T09:27:50.275847+0000","last_became_peered":"2023-09-23T09:27:50.275847+0000","last_unstale":"2023-09-23T09:27:51.398274+0000","last_undegraded":"2023-09-23T09:27:51.398274+0000","last_fullsized":"2023-09-23T09:27:51.398274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:04:03.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734273+0000","last_change":"2023-09-23T09:27:50.279420+0000","last_active":"2023-09-23T09:27:51.734273+0000","last_peered":"2023-09-23T09:27:51.734273+0000","last_clean":"2023-09-23T09:27:51.734273+0000","last_became_active":"2023-09-23T09:27:50.279120+0000","last_became_peered":"2023-09-23T09:27:50.279120+0000","last_unstale":"2023-09-23T09:27:51.734273+0000","last_undegraded":"2023-09-23T09:27:51.734273+0000","last_fullsized":"2023-09-23T09:27:51.734273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:48:03.461868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734300+0000","last_change":"2023-09-23T09:27:50.276364+0000","last_active":"2023-09-23T09:27:51.734300+0000","last_peered":"2023-09-23T09:27:51.734300+0000","last_clean":"2023-09-23T09:27:51.734300+0000","last_became_active":"2023-09-23T09:27:50.276060+0000","last_became_peered":"2023-09-23T09:27:50.276060+0000","last_unstale":"2023-09-23T09:27:51.734300+0000","last_undegraded":"2023-09-23T09:27:51.734300+0000","last_fullsized":"2023-09-23T09:27:51.734300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:26.998609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259923+0000","last_change":"2023-09-23T09:27:50.280302+0000","last_active":"2023-09-23T09:27:51.259923+0000","last_peered":"2023-09-23T09:27:51.259923+0000","last_clean":"2023-09-23T09:27:51.259923+0000","last_became_active":"2023-09-23T09:27:50.279295+0000","last_became_peered":"2023-09-23T09:27:50.279295+0000","last_unstale":"2023-09-23T09:27:51.259923+0000","last_undegraded":"2023-09-23T09:27:51.259923+0000","last_fullsized":"2023-09-23T09:27:51.259923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:15.329483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398526+0000","last_change":"2023-09-23T09:27:50.271876+0000","last_active":"2023-09-23T09:27:51.398526+0000","last_peered":"2023-09-23T09:27:51.398526+0000","last_clean":"2023-09-23T09:27:51.398526+0000","last_became_active":"2023-09-23T09:27:50.271576+0000","last_became_peered":"2023-09-23T09:27:50.271576+0000","last_unstale":"2023-09-23T09:27:51.398526+0000","last_undegraded":"2023-09-23T09:27:51.398526+0000","last_fullsized":"2023-09-23T09:27:51.398526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:17:22.656828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734649+0000","last_change":"2023-09-23T09:27:50.271405+0000","last_active":"2023-09-23T09:27:51.734649+0000","last_peered":"2023-09-23T09:27:51.734649+0000","last_clean":"2023-09-23T09:27:51.734649+0000","last_became_active":"2023-09-23T09:27:50.271087+0000","last_became_peered":"2023-09-23T09:27:50.271087+0000","last_unstale":"2023-09-23T09:27:51.734649+0000","last_undegraded":"2023-09-23T09:27:51.734649+0000","last_fullsized":"2023-09-23T09:27:51.734649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:09.363894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733876+0000","last_change":"2023-09-23T09:27:50.265855+0000","last_active":"2023-09-23T09:27:51.733876+0000","last_peered":"2023-09-23T09:27:51.733876+0000","last_clean":"2023-09-23T09:27:51.733876+0000","last_became_active":"2023-09-23T09:27:50.265101+0000","last_became_peered":"2023-09-23T09:27:50.265101+0000","last_unstale":"2023-09-23T09:27:51.733876+0000","last_undegraded":"2023-09-23T09:27:51.733876+0000","last_fullsized":"2023-09-23T09:27:51.733876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:52:40.456013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734201+0000","last_change":"2023-09-23T09:27:50.274405+0000","last_active":"2023-09-23T09:27:51.734201+0000","last_peered":"2023-09-23T09:27:51.734201+0000","last_clean":"2023-09-23T09:27:51.734201+0000","last_became_active":"2023-09-23T09:27:50.274036+0000","last_became_peered":"2023-09-23T09:27:50.274036+0000","last_unstale":"2023-09-23T09:27:51.734201+0000","last_undegraded":"2023-09-23T09:27:51.734201+0000","last_fullsized":"2023-09-23T09:27:51.734201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:38.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733331+0000","last_change":"2023-09-23T09:27:50.272237+0000","last_active":"2023-09-23T09:27:51.733331+0000","last_peered":"2023-09-23T09:27:51.733331+0000","last_clean":"2023-09-23T09:27:51.733331+0000","last_became_active":"2023-09-23T09:27:50.271897+0000","last_became_peered":"2023-09-23T09:27:50.271897+0000","last_unstale":"2023-09-23T09:27:51.733331+0000","last_undegraded":"2023-09-23T09:27:51.733331+0000","last_fullsized":"2023-09-23T09:27:51.733331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:31:31.115205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733452+0000","last_change":"2023-09-23T09:27:50.268902+0000","last_active":"2023-09-23T09:27:51.733452+0000","last_peered":"2023-09-23T09:27:51.733452+0000","last_clean":"2023-09-23T09:27:51.733452+0000","last_became_active":"2023-09-23T09:27:50.268590+0000","last_became_peered":"2023-09-23T09:27:50.268590+0000","last_unstale":"2023-09-23T09:27:51.733452+0000","last_undegraded":"2023-09-23T09:27:51.733452+0000","last_fullsized":"2023-09-23T09:27:51.733452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:54:39.384835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259359+0000","last_change":"2023-09-23T09:27:50.276934+0000","last_active":"2023-09-23T09:27:51.259359+0000","last_peered":"2023-09-23T09:27:51.259359+0000","last_clean":"2023-09-23T09:27:51.259359+0000","last_became_active":"2023-09-23T09:27:50.276645+0000","last_became_peered":"2023-09-23T09:27:50.276645+0000","last_unstale":"2023-09-23T09:27:51.259359+0000","last_undegraded":"2023-09-23T09:27:51.259359+0000","last_fullsized":"2023-09-23T09:27:51.259359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:07.100550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735229+0000","last_change":"2023-09-23T09:27:50.270781+0000","last_active":"2023-09-23T09:27:51.735229+0000","last_peered":"2023-09-23T09:27:51.735229+0000","last_clean":"2023-09-23T09:27:51.735229+0000","last_became_active":"2023-09-23T09:27:50.270319+0000","last_became_peered":"2023-09-23T09:27:50.270319+0000","last_unstale":"2023-09-23T09:27:51.735229+0000","last_undegraded":"2023-09-23T09:27:51.735229+0000","last_fullsized":"2023-09-23T09:27:51.735229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:42:44.882122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259692+0000","last_change":"2023-09-23T09:27:50.278775+0000","last_active":"2023-09-23T09:27:51.259692+0000","last_peered":"2023-09-23T09:27:51.259692+0000","last_clean":"2023-09-23T09:27:51.259692+0000","last_became_active":"2023-09-23T09:27:50.278375+0000","last_became_peered":"2023-09-23T09:27:50.278375+0000","last_unstale":"2023-09-23T09:27:51.259692+0000","last_undegraded":"2023-09-23T09:27:51.259692+0000","last_fullsized":"2023-09-23T09:27:51.259692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:14:00.076810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264221+0000","last_change":"2023-09-23T09:27:50.270141+0000","last_active":"2023-09-23T09:27:51.264221+0000","last_peered":"2023-09-23T09:27:51.264221+0000","last_clean":"2023-09-23T09:27:51.264221+0000","last_became_active":"2023-09-23T09:27:50.269849+0000","last_became_peered":"2023-09-23T09:27:50.269849+0000","last_unstale":"2023-09-23T09:27:51.264221+0000","last_undegraded":"2023-09-23T09:27:51.264221+0000","last_fullsized":"2023-09-23T09:27:51.264221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:49:41.837886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259449+0000","last_change":"2023-09-23T09:27:49.654458+0000","last_active":"2023-09-23T09:27:51.259449+0000","last_peered":"2023-09-23T09:27:51.259449+0000","last_clean":"2023-09-23T09:27:51.259449+0000","last_became_active":"2023-09-23T09:27:49.654084+0000","last_became_peered":"2023-09-23T09:27:49.654084+0000","last_unstale":"2023-09-23T09:27:51.259449+0000","last_undegraded":"2023-09-23T09:27:51.259449+0000","last_fullsized":"2023-09-23T09:27:51.259449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:47:32.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734956+0000","last_change":"2023-09-23T09:27:50.276558+0000","last_active":"2023-09-23T09:27:51.734956+0000","last_peered":"2023-09-23T09:27:51.734956+0000","last_clean":"2023-09-23T09:27:51.734956+0000","last_became_active":"2023-09-23T09:27:50.276255+0000","last_became_peered":"2023-09-23T09:27:50.276255+0000","last_unstale":"2023-09-23T09:27:51.734956+0000","last_undegraded":"2023-09-23T09:27:51.734956+0000","last_fullsized":"2023-09-23T09:27:51.734956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:13:09.870383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735166+0000","last_change":"2023-09-23T09:27:49.659256+0000","last_active":"2023-09-23T09:27:51.735166+0000","last_peered":"2023-09-23T09:27:51.735166+0000","last_clean":"2023-09-23T09:27:51.735166+0000","last_became_active":"2023-09-23T09:27:49.658793+0000","last_became_peered":"2023-09-23T09:27:49.658793+0000","last_unstale":"2023-09-23T09:27:51.735166+0000","last_undegraded":"2023-09-23T09:27:51.735166+0000","last_fullsized":"2023-09-23T09:27:51.735166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:03.416666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733674+0000","last_change":"2023-09-23T09:27:50.277338+0000","last_active":"2023-09-23T09:27:51.733674+0000","last_peered":"2023-09-23T09:27:51.733674+0000","last_clean":"2023-09-23T09:27:51.733674+0000","last_became_active":"2023-09-23T09:27:50.277011+0000","last_became_peered":"2023-09-23T09:27:50.277011+0000","last_unstale":"2023-09-23T09:27:51.733674+0000","last_undegraded":"2023-09-23T09:27:51.733674+0000","last_fullsized":"2023-09-23T09:27:51.733674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:52:23.028794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702688+0000","last_change":"2023-09-23T09:27:49.660679+0000","last_active":"2023-09-23T09:27:54.702688+0000","last_peered":"2023-09-23T09:27:54.702688+0000","last_clean":"2023-09-23T09:27:54.702688+0000","last_became_active":"2023-09-23T09:27:49.660201+0000","last_became_peered":"2023-09-23T09:27:49.660201+0000","last_unstale":"2023-09-23T09:27:54.702688+0000","last_undegraded":"2023-09-23T09:27:54.702688+0000","last_fullsized":"2023-09-23T09:27:54.702688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:33:56.205322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259494+0000","last_change":"2023-09-23T09:27:50.278373+0000","last_active":"2023-09-23T09:27:51.259494+0000","last_peered":"2023-09-23T09:27:51.259494+0000","last_clean":"2023-09-23T09:27:51.259494+0000","last_became_active":"2023-09-23T09:27:50.277837+0000","last_became_peered":"2023-09-23T09:27:50.277837+0000","last_unstale":"2023-09-23T09:27:51.259494+0000","last_undegraded":"2023-09-23T09:27:51.259494+0000","last_fullsized":"2023-09-23T09:27:51.259494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:24:26.914509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.117099+0000","last_change":"2023-09-23T09:27:49.657578+0000","last_active":"2023-09-23T09:27:51.117099+0000","last_peered":"2023-09-23T09:27:51.117099+0000","last_clean":"2023-09-23T09:27:51.117099+0000","last_became_active":"2023-09-23T09:27:49.657278+0000","last_became_peered":"2023-09-23T09:27:49.657278+0000","last_unstale":"2023-09-23T09:27:51.117099+0000","last_undegraded":"2023-09-23T09:27:51.117099+0000","last_fullsized":"2023-09-23T09:27:51.117099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:53.440997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734804+0000","last_change":"2023-09-23T09:27:50.279784+0000","last_active":"2023-09-23T09:27:51.734804+0000","last_peered":"2023-09-23T09:27:51.734804+0000","last_clean":"2023-09-23T09:27:51.734804+0000","last_became_active":"2023-09-23T09:27:50.279473+0000","last_became_peered":"2023-09-23T09:27:50.279473+0000","last_unstale":"2023-09-23T09:27:51.734804+0000","last_undegraded":"2023-09-23T09:27:51.734804+0000","last_fullsized":"2023-09-23T09:27:51.734804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:30:13.010556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690814+0000","last_change":"2023-09-23T09:27:49.655391+0000","last_active":"2023-09-23T09:27:53.690814+0000","last_peered":"2023-09-23T09:27:53.690814+0000","last_clean":"2023-09-23T09:27:53.690814+0000","last_became_active":"2023-09-23T09:27:49.655067+0000","last_became_peered":"2023-09-23T09:27:49.655067+0000","last_unstale":"2023-09-23T09:27:53.690814+0000","last_undegraded":"2023-09-23T09:27:53.690814+0000","last_fullsized":"2023-09-23T09:27:53.690814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.701471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733903+0000","last_change":"2023-09-23T09:27:50.275688+0000","last_active":"2023-09-23T09:27:51.733903+0000","last_peered":"2023-09-23T09:27:51.733903+0000","last_clean":"2023-09-23T09:27:51.733903+0000","last_became_active":"2023-09-23T09:27:50.275356+0000","last_became_peered":"2023-09-23T09:27:50.275356+0000","last_unstale":"2023-09-23T09:27:51.733903+0000","last_undegraded":"2023-09-23T09:27:51.733903+0000","last_fullsized":"2023-09-23T09:27:51.733903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:43:31.829209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.663153+0000","last_change":"2023-09-23T09:27:49.657066+0000","last_active":"2023-09-23T09:27:50.663153+0000","last_peered":"2023-09-23T09:27:50.663153+0000","last_clean":"2023-09-23T09:27:50.663153+0000","last_became_active":"2023-09-23T09:27:49.656754+0000","last_became_peered":"2023-09-23T09:27:49.656754+0000","last_unstale":"2023-09-23T09:27:50.663153+0000","last_undegraded":"2023-09-23T09:27:50.663153+0000","last_fullsized":"2023-09-23T09:27:50.663153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:52.277474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734458+0000","last_change":"2023-09-23T09:27:50.273705+0000","last_active":"2023-09-23T09:27:51.734458+0000","last_peered":"2023-09-23T09:27:51.734458+0000","last_clean":"2023-09-23T09:27:51.734458+0000","last_became_active":"2023-09-23T09:27:50.272245+0000","last_became_peered":"2023-09-23T09:27:50.272245+0000","last_unstale":"2023-09-23T09:27:51.734458+0000","last_undegraded":"2023-09-23T09:27:51.734458+0000","last_fullsized":"2023-09-23T09:27:51.734458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:45:57.318326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.255097+0000","last_change":"2023-09-23T09:27:49.651958+0000","last_active":"2023-09-23T09:27:50.255097+0000","last_peered":"2023-09-23T09:27:50.255097+0000","last_clean":"2023-09-23T09:27:50.255097+0000","last_became_active":"2023-09-23T09:27:49.651058+0000","last_became_peered":"2023-09-23T09:27:49.651058+0000","last_unstale":"2023-09-23T09:27:50.255097+0000","last_undegraded":"2023-09-23T09:27:50.255097+0000","last_fullsized":"2023-09-23T09:27:50.255097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:26:56.360566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734149+0000","last_change":"2023-09-23T09:27:50.270804+0000","last_active":"2023-09-23T09:27:51.734149+0000","last_peered":"2023-09-23T09:27:51.734149+0000","last_clean":"2023-09-23T09:27:51.734149+0000","last_became_active":"2023-09-23T09:27:50.270044+0000","last_became_peered":"2023-09-23T09:27:50.270044+0000","last_unstale":"2023-09-23T09:27:51.734149+0000","last_undegraded":"2023-09-23T09:27:51.734149+0000","last_fullsized":"2023-09-23T09:27:51.734149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:59:20.821949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734563+0000","last_change":"2023-09-23T09:27:49.657030+0000","last_active":"2023-09-23T09:27:51.734563+0000","last_peered":"2023-09-23T09:27:51.734563+0000","last_clean":"2023-09-23T09:27:51.734563+0000","last_became_active":"2023-09-23T09:27:49.656710+0000","last_became_peered":"2023-09-23T09:27:49.656710+0000","last_unstale":"2023-09-23T09:27:51.734563+0000","last_undegraded":"2023-09-23T09:27:51.734563+0000","last_fullsized":"2023-09-23T09:27:51.734563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:24:59.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259415+0000","last_change":"2023-09-23T09:27:50.271688+0000","last_active":"2023-09-23T09:27:51.259415+0000","last_peered":"2023-09-23T09:27:51.259415+0000","last_clean":"2023-09-23T09:27:51.259415+0000","last_became_active":"2023-09-23T09:27:50.271276+0000","last_became_peered":"2023-09-23T09:27:50.271276+0000","last_unstale":"2023-09-23T09:27:51.259415+0000","last_undegraded":"2023-09-23T09:27:51.259415+0000","last_fullsized":"2023-09-23T09:27:51.259415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:40:48.333279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.692028+0000","last_change":"2023-09-23T09:27:49.655183+0000","last_active":"2023-09-23T09:27:53.692028+0000","last_peered":"2023-09-23T09:27:53.692028+0000","last_clean":"2023-09-23T09:27:53.692028+0000","last_became_active":"2023-09-23T09:27:49.654875+0000","last_became_peered":"2023-09-23T09:27:49.654875+0000","last_unstale":"2023-09-23T09:27:53.692028+0000","last_undegraded":"2023-09-23T09:27:53.692028+0000","last_fullsized":"2023-09-23T09:27:53.692028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:12:49.944732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259843+0000","last_change":"2023-09-23T09:27:50.276347+0000","last_active":"2023-09-23T09:27:51.259843+0000","last_peered":"2023-09-23T09:27:51.259843+0000","last_clean":"2023-09-23T09:27:51.259843+0000","last_became_active":"2023-09-23T09:27:50.276095+0000","last_became_peered":"2023-09-23T09:27:50.276095+0000","last_unstale":"2023-09-23T09:27:51.259843+0000","last_undegraded":"2023-09-23T09:27:51.259843+0000","last_fullsized":"2023-09-23T09:27:51.259843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:42.952808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.692327+0000","last_change":"2023-09-23T09:27:49.657115+0000","last_active":"2023-09-23T09:27:53.692327+0000","last_peered":"2023-09-23T09:27:53.692327+0000","last_clean":"2023-09-23T09:27:53.692327+0000","last_became_active":"2023-09-23T09:27:49.656816+0000","last_became_peered":"2023-09-23T09:27:49.656816+0000","last_unstale":"2023-09-23T09:27:53.692327+0000","last_undegraded":"2023-09-23T09:27:53.692327+0000","last_fullsized":"2023-09-23T09:27:53.692327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:37.205221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733619+0000","last_change":"2023-09-23T09:27:50.277477+0000","last_active":"2023-09-23T09:27:51.733619+0000","last_peered":"2023-09-23T09:27:51.733619+0000","last_clean":"2023-09-23T09:27:51.733619+0000","last_became_active":"2023-09-23T09:27:50.277200+0000","last_became_peered":"2023-09-23T09:27:50.277200+0000","last_unstale":"2023-09-23T09:27:51.733619+0000","last_undegraded":"2023-09-23T09:27:51.733619+0000","last_fullsized":"2023-09-23T09:27:51.733619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:45:03.167471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734861+0000","last_change":"2023-09-23T09:27:49.663678+0000","last_active":"2023-09-23T09:27:51.734861+0000","last_peered":"2023-09-23T09:27:51.734861+0000","last_clean":"2023-09-23T09:27:51.734861+0000","last_became_active":"2023-09-23T09:27:49.663142+0000","last_became_peered":"2023-09-23T09:27:49.663142+0000","last_unstale":"2023-09-23T09:27:51.734861+0000","last_undegraded":"2023-09-23T09:27:51.734861+0000","last_fullsized":"2023-09-23T09:27:51.734861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:14.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259656+0000","last_change":"2023-09-23T09:27:50.275044+0000","last_active":"2023-09-23T09:27:51.259656+0000","last_peered":"2023-09-23T09:27:51.259656+0000","last_clean":"2023-09-23T09:27:51.259656+0000","last_became_active":"2023-09-23T09:27:50.274752+0000","last_became_peered":"2023-09-23T09:27:50.274752+0000","last_unstale":"2023-09-23T09:27:51.259656+0000","last_undegraded":"2023-09-23T09:27:51.259656+0000","last_fullsized":"2023-09-23T09:27:51.259656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:19.914752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.691494+0000","last_change":"2023-09-23T09:27:49.653694+0000","last_active":"2023-09-23T09:27:53.691494+0000","last_peered":"2023-09-23T09:27:53.691494+0000","last_clean":"2023-09-23T09:27:53.691494+0000","last_became_active":"2023-09-23T09:27:49.653355+0000","last_became_peered":"2023-09-23T09:27:49.653355+0000","last_unstale":"2023-09-23T09:27:53.691494+0000","last_undegraded":"2023-09-23T09:27:53.691494+0000","last_fullsized":"2023-09-23T09:27:53.691494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:19:53.282723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.272849+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.272411+0000","last_became_peered":"2023-09-23T09:27:50.272411+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:01:41.972357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702769+0000","last_change":"2023-09-23T09:27:49.652769+0000","last_active":"2023-09-23T09:27:54.702769+0000","last_peered":"2023-09-23T09:27:54.702769+0000","last_clean":"2023-09-23T09:27:54.702769+0000","last_became_active":"2023-09-23T09:27:49.652440+0000","last_became_peered":"2023-09-23T09:27:49.652440+0000","last_unstale":"2023-09-23T09:27:54.702769+0000","last_undegraded":"2023-09-23T09:27:54.702769+0000","last_fullsized":"2023-09-23T09:27:54.702769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:02.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734000+0000","last_change":"2023-09-23T09:27:50.272105+0000","last_active":"2023-09-23T09:27:51.734000+0000","last_peered":"2023-09-23T09:27:51.734000+0000","last_clean":"2023-09-23T09:27:51.734000+0000","last_became_active":"2023-09-23T09:27:50.271766+0000","last_became_peered":"2023-09-23T09:27:50.271766+0000","last_unstale":"2023-09-23T09:27:51.734000+0000","last_undegraded":"2023-09-23T09:27:51.734000+0000","last_fullsized":"2023-09-23T09:27:51.734000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:46:46.012381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690285+0000","last_change":"2023-09-23T09:27:49.663473+0000","last_active":"2023-09-23T09:27:53.690285+0000","last_peered":"2023-09-23T09:27:53.690285+0000","last_clean":"2023-09-23T09:27:53.690285+0000","last_became_active":"2023-09-23T09:27:49.663085+0000","last_became_peered":"2023-09-23T09:27:49.663085+0000","last_unstale":"2023-09-23T09:27:53.690285+0000","last_undegraded":"2023-09-23T09:27:53.690285+0000","last_fullsized":"2023-09-23T09:27:53.690285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:28:05.246720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734657+0000","last_change":"2023-09-23T09:27:50.274172+0000","last_active":"2023-09-23T09:27:51.734657+0000","last_peered":"2023-09-23T09:27:51.734657+0000","last_clean":"2023-09-23T09:27:51.734657+0000","last_became_active":"2023-09-23T09:27:50.273874+0000","last_became_peered":"2023-09-23T09:27:50.273874+0000","last_unstale":"2023-09-23T09:27:51.734657+0000","last_undegraded":"2023-09-23T09:27:51.734657+0000","last_fullsized":"2023-09-23T09:27:51.734657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:04:06.864055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733787+0000","last_change":"2023-09-23T09:27:49.654341+0000","last_active":"2023-09-23T09:27:51.733787+0000","last_peered":"2023-09-23T09:27:51.733787+0000","last_clean":"2023-09-23T09:27:51.733787+0000","last_became_active":"2023-09-23T09:27:49.653936+0000","last_became_peered":"2023-09-23T09:27:49.653936+0000","last_unstale":"2023-09-23T09:27:51.733787+0000","last_undegraded":"2023-09-23T09:27:51.733787+0000","last_fullsized":"2023-09-23T09:27:51.733787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:48.876752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734194+0000","last_change":"2023-09-23T09:27:50.278620+0000","last_active":"2023-09-23T09:27:51.734194+0000","last_peered":"2023-09-23T09:27:51.734194+0000","last_clean":"2023-09-23T09:27:51.734194+0000","last_became_active":"2023-09-23T09:27:50.278289+0000","last_became_peered":"2023-09-23T09:27:50.278289+0000","last_unstale":"2023-09-23T09:27:51.734194+0000","last_undegraded":"2023-09-23T09:27:51.734194+0000","last_fullsized":"2023-09-23T09:27:51.734194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:20:44.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704421+0000","last_change":"2023-09-23T09:27:49.662970+0000","last_active":"2023-09-23T09:27:54.704421+0000","last_peered":"2023-09-23T09:27:54.704421+0000","last_clean":"2023-09-23T09:27:54.704421+0000","last_became_active":"2023-09-23T09:27:49.662664+0000","last_became_peered":"2023-09-23T09:27:49.662664+0000","last_unstale":"2023-09-23T09:27:54.704421+0000","last_undegraded":"2023-09-23T09:27:54.704421+0000","last_fullsized":"2023-09-23T09:27:54.704421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:13.189686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734127+0000","last_change":"2023-09-23T09:27:50.278475+0000","last_active":"2023-09-23T09:27:51.734127+0000","last_peered":"2023-09-23T09:27:51.734127+0000","last_clean":"2023-09-23T09:27:51.734127+0000","last_became_active":"2023-09-23T09:27:50.277802+0000","last_became_peered":"2023-09-23T09:27:50.277802+0000","last_unstale":"2023-09-23T09:27:51.734127+0000","last_undegraded":"2023-09-23T09:27:51.734127+0000","last_fullsized":"2023-09-23T09:27:51.734127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:44.608172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734574+0000","last_change":"2023-09-23T09:27:49.657191+0000","last_active":"2023-09-23T09:27:51.734574+0000","last_peered":"2023-09-23T09:27:51.734574+0000","last_clean":"2023-09-23T09:27:51.734574+0000","last_became_active":"2023-09-23T09:27:49.656863+0000","last_became_peered":"2023-09-23T09:27:49.656863+0000","last_unstale":"2023-09-23T09:27:51.734574+0000","last_undegraded":"2023-09-23T09:27:51.734574+0000","last_fullsized":"2023-09-23T09:27:51.734574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:10.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734984+0000","last_change":"2023-09-23T09:27:50.274609+0000","last_active":"2023-09-23T09:27:51.734984+0000","last_peered":"2023-09-23T09:27:51.734984+0000","last_clean":"2023-09-23T09:27:51.734984+0000","last_became_active":"2023-09-23T09:27:50.274316+0000","last_became_peered":"2023-09-23T09:27:50.274316+0000","last_unstale":"2023-09-23T09:27:51.734984+0000","last_undegraded":"2023-09-23T09:27:51.734984+0000","last_fullsized":"2023-09-23T09:27:51.734984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:10:51.284306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259542+0000","last_change":"2023-09-23T09:27:49.659410+0000","last_active":"2023-09-23T09:27:51.259542+0000","last_peered":"2023-09-23T09:27:51.259542+0000","last_clean":"2023-09-23T09:27:51.259542+0000","last_became_active":"2023-09-23T09:27:49.659087+0000","last_became_peered":"2023-09-23T09:27:49.659087+0000","last_unstale":"2023-09-23T09:27:51.259542+0000","last_undegraded":"2023-09-23T09:27:51.259542+0000","last_fullsized":"2023-09-23T09:27:51.259542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:23:41.677416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398391+0000","last_change":"2023-09-23T09:27:50.268568+0000","last_active":"2023-09-23T09:27:51.398391+0000","last_peered":"2023-09-23T09:27:51.398391+0000","last_clean":"2023-09-23T09:27:51.398391+0000","last_became_active":"2023-09-23T09:27:50.268268+0000","last_became_peered":"2023-09-23T09:27:50.268268+0000","last_unstale":"2023-09-23T09:27:51.398391+0000","last_undegraded":"2023-09-23T09:27:51.398391+0000","last_fullsized":"2023-09-23T09:27:51.398391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:59:22.516794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734411+0000","last_change":"2023-09-23T09:27:49.657160+0000","last_active":"2023-09-23T09:27:51.734411+0000","last_peered":"2023-09-23T09:27:51.734411+0000","last_clean":"2023-09-23T09:27:51.734411+0000","last_became_active":"2023-09-23T09:27:49.656853+0000","last_became_peered":"2023-09-23T09:27:49.656853+0000","last_unstale":"2023-09-23T09:27:51.734411+0000","last_undegraded":"2023-09-23T09:27:51.734411+0000","last_fullsized":"2023-09-23T09:27:51.734411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:17.489250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398429+0000","last_change":"2023-09-23T09:27:50.272495+0000","last_active":"2023-09-23T09:27:51.398429+0000","last_peered":"2023-09-23T09:27:51.398429+0000","last_clean":"2023-09-23T09:27:51.398429+0000","last_became_active":"2023-09-23T09:27:50.272207+0000","last_became_peered":"2023-09-23T09:27:50.272207+0000","last_unstale":"2023-09-23T09:27:51.398429+0000","last_undegraded":"2023-09-23T09:27:51.398429+0000","last_fullsized":"2023-09-23T09:27:51.398429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:24.952253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734722+0000","last_change":"2023-09-23T09:27:50.275551+0000","last_active":"2023-09-23T09:27:51.734722+0000","last_peered":"2023-09-23T09:27:51.734722+0000","last_clean":"2023-09-23T09:27:51.734722+0000","last_became_active":"2023-09-23T09:27:50.275235+0000","last_became_peered":"2023-09-23T09:27:50.275235+0000","last_unstale":"2023-09-23T09:27:51.734722+0000","last_undegraded":"2023-09-23T09:27:51.734722+0000","last_fullsized":"2023-09-23T09:27:51.734722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:41.009566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734928+0000","last_change":"2023-09-23T09:27:49.652889+0000","last_active":"2023-09-23T09:27:51.734928+0000","last_peered":"2023-09-23T09:27:51.734928+0000","last_clean":"2023-09-23T09:27:51.734928+0000","last_became_active":"2023-09-23T09:27:49.652571+0000","last_became_peered":"2023-09-23T09:27:49.652571+0000","last_unstale":"2023-09-23T09:27:51.734928+0000","last_undegraded":"2023-09-23T09:27:51.734928+0000","last_fullsized":"2023-09-23T09:27:51.734928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:22.484052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734814+0000","last_change":"2023-09-23T09:27:50.278419+0000","last_active":"2023-09-23T09:27:51.734814+0000","last_peered":"2023-09-23T09:27:51.734814+0000","last_clean":"2023-09-23T09:27:51.734814+0000","last_became_active":"2023-09-23T09:27:50.278071+0000","last_became_peered":"2023-09-23T09:27:50.278071+0000","last_unstale":"2023-09-23T09:27:51.734814+0000","last_undegraded":"2023-09-23T09:27:51.734814+0000","last_fullsized":"2023-09-23T09:27:51.734814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:28:45.896479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733837+0000","last_change":"2023-09-23T09:27:49.651251+0000","last_active":"2023-09-23T09:27:51.733837+0000","last_peered":"2023-09-23T09:27:51.733837+0000","last_clean":"2023-09-23T09:27:51.733837+0000","last_became_active":"2023-09-23T09:27:49.650933+0000","last_became_peered":"2023-09-23T09:27:49.650933+0000","last_unstale":"2023-09-23T09:27:51.733837+0000","last_undegraded":"2023-09-23T09:27:51.733837+0000","last_fullsized":"2023-09-23T09:27:51.733837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:00:54.100400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.264684+0000","last_change":"2023-09-23T09:26:51.296669+0000","last_active":"2023-09-23T09:27:50.264684+0000","last_peered":"2023-09-23T09:27:50.264684+0000","last_clean":"2023-09-23T09:27:50.264684+0000","last_became_active":"2023-09-23T09:26:51.296229+0000","last_became_peered":"2023-09-23T09:26:51.296229+0000","last_unstale":"2023-09-23T09:27:50.264684+0000","last_undegraded":"2023-09-23T09:27:50.264684+0000","last_fullsized":"2023-09-23T09:27:50.264684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_clean_scrub_stamp":"2023-09-23T09:24:01.204422+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:31:24.888152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734832+0000","last_change":"2023-09-23T09:27:50.267662+0000","last_active":"2023-09-23T09:27:51.734832+0000","last_peered":"2023-09-23T09:27:51.734832+0000","last_clean":"2023-09-23T09:27:51.734832+0000","last_became_active":"2023-09-23T09:27:50.267374+0000","last_became_peered":"2023-09-23T09:27:50.267374+0000","last_unstale":"2023-09-23T09:27:51.734832+0000","last_undegraded":"2023-09-23T09:27:51.734832+0000","last_fullsized":"2023-09-23T09:27:51.734832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:22.797939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.701373+0000","last_change":"2023-09-23T09:27:49.659773+0000","last_active":"2023-09-23T09:27:54.701373+0000","last_peered":"2023-09-23T09:27:54.701373+0000","last_clean":"2023-09-23T09:27:54.701373+0000","last_became_active":"2023-09-23T09:27:49.659526+0000","last_became_peered":"2023-09-23T09:27:49.659526+0000","last_unstale":"2023-09-23T09:27:54.701373+0000","last_undegraded":"2023-09-23T09:27:54.701373+0000","last_fullsized":"2023-09-23T09:27:54.701373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:27.423680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689202+0000","last_change":"2023-09-23T09:27:49.661336+0000","last_active":"2023-09-23T09:27:53.689202+0000","last_peered":"2023-09-23T09:27:53.689202+0000","last_clean":"2023-09-23T09:27:53.689202+0000","last_became_active":"2023-09-23T09:27:49.661038+0000","last_became_peered":"2023-09-23T09:27:49.661038+0000","last_unstale":"2023-09-23T09:27:53.689202+0000","last_undegraded":"2023-09-23T09:27:53.689202+0000","last_fullsized":"2023-09-23T09:27:53.689202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:20.634410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735024+0000","last_change":"2023-09-23T09:27:50.272485+0000","last_active":"2023-09-23T09:27:51.735024+0000","last_peered":"2023-09-23T09:27:51.735024+0000","last_clean":"2023-09-23T09:27:51.735024+0000","last_became_active":"2023-09-23T09:27:50.272164+0000","last_became_peered":"2023-09-23T09:27:50.272164+0000","last_unstale":"2023-09-23T09:27:51.735024+0000","last_undegraded":"2023-09-23T09:27:51.735024+0000","last_fullsized":"2023-09-23T09:27:51.735024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:48:00.596034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259600+0000","last_change":"2023-09-23T09:27:49.657563+0000","last_active":"2023-09-23T09:27:51.259600+0000","last_peered":"2023-09-23T09:27:51.259600+0000","last_clean":"2023-09-23T09:27:51.259600+0000","last_became_active":"2023-09-23T09:27:49.657232+0000","last_became_peered":"2023-09-23T09:27:49.657232+0000","last_unstale":"2023-09-23T09:27:51.259600+0000","last_undegraded":"2023-09-23T09:27:51.259600+0000","last_fullsized":"2023-09-23T09:27:51.259600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:29:10.640013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735007+0000","last_change":"2023-09-23T09:27:50.274959+0000","last_active":"2023-09-23T09:27:51.735007+0000","last_peered":"2023-09-23T09:27:51.735007+0000","last_clean":"2023-09-23T09:27:51.735007+0000","last_became_active":"2023-09-23T09:27:50.274614+0000","last_became_peered":"2023-09-23T09:27:50.274614+0000","last_unstale":"2023-09-23T09:27:51.735007+0000","last_undegraded":"2023-09-23T09:27:51.735007+0000","last_fullsized":"2023-09-23T09:27:51.735007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:00:09.831103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733844+0000","last_change":"2023-09-23T09:27:49.654673+0000","last_active":"2023-09-23T09:27:51.733844+0000","last_peered":"2023-09-23T09:27:51.733844+0000","last_clean":"2023-09-23T09:27:51.733844+0000","last_became_active":"2023-09-23T09:27:49.654356+0000","last_became_peered":"2023-09-23T09:27:49.654356+0000","last_unstale":"2023-09-23T09:27:51.733844+0000","last_undegraded":"2023-09-23T09:27:51.733844+0000","last_fullsized":"2023-09-23T09:27:51.733844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:39:50.628346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.267368+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.266962+0000","last_became_peered":"2023-09-23T09:27:50.266962+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:24.660228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690757+0000","last_change":"2023-09-23T09:27:49.661200+0000","last_active":"2023-09-23T09:27:53.690757+0000","last_peered":"2023-09-23T09:27:53.690757+0000","last_clean":"2023-09-23T09:27:53.690757+0000","last_became_active":"2023-09-23T09:27:49.660902+0000","last_became_peered":"2023-09-23T09:27:49.660902+0000","last_unstale":"2023-09-23T09:27:53.690757+0000","last_undegraded":"2023-09-23T09:27:53.690757+0000","last_fullsized":"2023-09-23T09:27:53.690757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:41.036216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735005+0000","last_change":"2023-09-23T09:27:50.271834+0000","last_active":"2023-09-23T09:27:51.735005+0000","last_peered":"2023-09-23T09:27:51.735005+0000","last_clean":"2023-09-23T09:27:51.735005+0000","last_became_active":"2023-09-23T09:27:50.271519+0000","last_became_peered":"2023-09-23T09:27:50.271519+0000","last_unstale":"2023-09-23T09:27:51.735005+0000","last_undegraded":"2023-09-23T09:27:51.735005+0000","last_fullsized":"2023-09-23T09:27:51.735005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:08.036615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689622+0000","last_change":"2023-09-23T09:27:49.656794+0000","last_active":"2023-09-23T09:27:53.689622+0000","last_peered":"2023-09-23T09:27:53.689622+0000","last_clean":"2023-09-23T09:27:53.689622+0000","last_became_active":"2023-09-23T09:27:49.656480+0000","last_became_peered":"2023-09-23T09:27:49.656480+0000","last_unstale":"2023-09-23T09:27:53.689622+0000","last_undegraded":"2023-09-23T09:27:53.689622+0000","last_fullsized":"2023-09-23T09:27:53.689622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:35:46.948703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734135+0000","last_change":"2023-09-23T09:27:50.271805+0000","last_active":"2023-09-23T09:27:51.734135+0000","last_peered":"2023-09-23T09:27:51.734135+0000","last_clean":"2023-09-23T09:27:51.734135+0000","last_became_active":"2023-09-23T09:27:50.271463+0000","last_became_peered":"2023-09-23T09:27:50.271463+0000","last_unstale":"2023-09-23T09:27:51.734135+0000","last_undegraded":"2023-09-23T09:27:51.734135+0000","last_fullsized":"2023-09-23T09:27:51.734135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:00:03.608504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264108+0000","last_change":"2023-09-23T09:27:49.656988+0000","last_active":"2023-09-23T09:27:51.264108+0000","last_peered":"2023-09-23T09:27:51.264108+0000","last_clean":"2023-09-23T09:27:51.264108+0000","last_became_active":"2023-09-23T09:27:49.656655+0000","last_became_peered":"2023-09-23T09:27:49.656655+0000","last_unstale":"2023-09-23T09:27:51.264108+0000","last_undegraded":"2023-09-23T09:27:51.264108+0000","last_fullsized":"2023-09-23T09:27:51.264108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:07:33.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734679+0000","last_change":"2023-09-23T09:27:50.275021+0000","last_active":"2023-09-23T09:27:51.734679+0000","last_peered":"2023-09-23T09:27:51.734679+0000","last_clean":"2023-09-23T09:27:51.734679+0000","last_became_active":"2023-09-23T09:27:50.274551+0000","last_became_peered":"2023-09-23T09:27:50.274551+0000","last_unstale":"2023-09-23T09:27:51.734679+0000","last_undegraded":"2023-09-23T09:27:51.734679+0000","last_fullsized":"2023-09-23T09:27:51.734679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:50:45.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690216+0000","last_change":"2023-09-23T09:27:49.659603+0000","last_active":"2023-09-23T09:27:53.690216+0000","last_peered":"2023-09-23T09:27:53.690216+0000","last_clean":"2023-09-23T09:27:53.690216+0000","last_became_active":"2023-09-23T09:27:49.659293+0000","last_became_peered":"2023-09-23T09:27:49.659293+0000","last_unstale":"2023-09-23T09:27:53.690216+0000","last_undegraded":"2023-09-23T09:27:53.690216+0000","last_fullsized":"2023-09-23T09:27:53.690216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:31:25.165888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734478+0000","last_change":"2023-09-23T09:27:50.275069+0000","last_active":"2023-09-23T09:27:51.734478+0000","last_peered":"2023-09-23T09:27:51.734478+0000","last_clean":"2023-09-23T09:27:51.734478+0000","last_became_active":"2023-09-23T09:27:50.274731+0000","last_became_peered":"2023-09-23T09:27:50.274731+0000","last_unstale":"2023-09-23T09:27:51.734478+0000","last_undegraded":"2023-09-23T09:27:51.734478+0000","last_fullsized":"2023-09-23T09:27:51.734478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:45:23.794774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.703349+0000","last_change":"2023-09-23T09:27:49.659914+0000","last_active":"2023-09-23T09:27:54.703349+0000","last_peered":"2023-09-23T09:27:54.703349+0000","last_clean":"2023-09-23T09:27:54.703349+0000","last_became_active":"2023-09-23T09:27:49.659604+0000","last_became_peered":"2023-09-23T09:27:49.659604+0000","last_unstale":"2023-09-23T09:27:54.703349+0000","last_undegraded":"2023-09-23T09:27:54.703349+0000","last_fullsized":"2023-09-23T09:27:54.703349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:20.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734661+0000","last_change":"2023-09-23T09:27:50.271309+0000","last_active":"2023-09-23T09:27:51.734661+0000","last_peered":"2023-09-23T09:27:51.734661+0000","last_clean":"2023-09-23T09:27:51.734661+0000","last_became_active":"2023-09-23T09:27:50.270951+0000","last_became_peered":"2023-09-23T09:27:50.270951+0000","last_unstale":"2023-09-23T09:27:51.734661+0000","last_undegraded":"2023-09-23T09:27:51.734661+0000","last_fullsized":"2023-09-23T09:27:51.734661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:18.811428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.664667+0000","last_change":"2023-09-23T09:27:49.663849+0000","last_active":"2023-09-23T09:27:50.664667+0000","last_peered":"2023-09-23T09:27:50.664667+0000","last_clean":"2023-09-23T09:27:50.664667+0000","last_became_active":"2023-09-23T09:27:49.663499+0000","last_became_peered":"2023-09-23T09:27:49.663499+0000","last_unstale":"2023-09-23T09:27:50.664667+0000","last_undegraded":"2023-09-23T09:27:50.664667+0000","last_fullsized":"2023-09-23T09:27:50.664667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:31:08.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734512+0000","last_change":"2023-09-23T09:27:50.275681+0000","last_active":"2023-09-23T09:27:51.734512+0000","last_peered":"2023-09-23T09:27:51.734512+0000","last_clean":"2023-09-23T09:27:51.734512+0000","last_became_active":"2023-09-23T09:27:50.275370+0000","last_became_peered":"2023-09-23T09:27:50.275370+0000","last_unstale":"2023-09-23T09:27:51.734512+0000","last_undegraded":"2023-09-23T09:27:51.734512+0000","last_fullsized":"2023-09-23T09:27:51.734512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:53:01.710377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.665421+0000","last_change":"2023-09-23T09:27:49.652127+0000","last_active":"2023-09-23T09:27:50.665421+0000","last_peered":"2023-09-23T09:27:50.665421+0000","last_clean":"2023-09-23T09:27:50.665421+0000","last_became_active":"2023-09-23T09:27:49.651792+0000","last_became_peered":"2023-09-23T09:27:49.651792+0000","last_unstale":"2023-09-23T09:27:50.665421+0000","last_undegraded":"2023-09-23T09:27:50.665421+0000","last_fullsized":"2023-09-23T09:27:50.665421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:23:39.299203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734734+0000","last_change":"2023-09-23T09:27:50.271063+0000","last_active":"2023-09-23T09:27:51.734734+0000","last_peered":"2023-09-23T09:27:51.734734+0000","last_clean":"2023-09-23T09:27:51.734734+0000","last_became_active":"2023-09-23T09:27:50.270487+0000","last_became_peered":"2023-09-23T09:27:50.270487+0000","last_unstale":"2023-09-23T09:27:51.734734+0000","last_undegraded":"2023-09-23T09:27:51.734734+0000","last_fullsized":"2023-09-23T09:27:51.734734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:11:05.621855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734264+0000","last_change":"2023-09-23T09:27:50.275900+0000","last_active":"2023-09-23T09:27:51.734264+0000","last_peered":"2023-09-23T09:27:51.734264+0000","last_clean":"2023-09-23T09:27:51.734264+0000","last_became_active":"2023-09-23T09:27:50.275565+0000","last_became_peered":"2023-09-23T09:27:50.275565+0000","last_unstale":"2023-09-23T09:27:51.734264+0000","last_undegraded":"2023-09-23T09:27:51.734264+0000","last_fullsized":"2023-09-23T09:27:51.734264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:57:37.004604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264041+0000","last_change":"2023-09-23T09:27:50.275911+0000","last_active":"2023-09-23T09:27:51.264041+0000","last_peered":"2023-09-23T09:27:51.264041+0000","last_clean":"2023-09-23T09:27:51.264041+0000","last_became_active":"2023-09-23T09:27:50.275626+0000","last_became_peered":"2023-09-23T09:27:50.275626+0000","last_unstale":"2023-09-23T09:27:51.264041+0000","last_undegraded":"2023-09-23T09:27:51.264041+0000","last_fullsized":"2023-09-23T09:27:51.264041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:46.790012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264037+0000","last_change":"2023-09-23T09:27:50.269483+0000","last_active":"2023-09-23T09:27:51.264037+0000","last_peered":"2023-09-23T09:27:51.264037+0000","last_clean":"2023-09-23T09:27:51.264037+0000","last_became_active":"2023-09-23T09:27:50.269199+0000","last_became_peered":"2023-09-23T09:27:50.269199+0000","last_unstale":"2023-09-23T09:27:51.264037+0000","last_undegraded":"2023-09-23T09:27:51.264037+0000","last_fullsized":"2023-09-23T09:27:51.264037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:58:34.304439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733479+0000","last_change":"2023-09-23T09:27:50.267917+0000","last_active":"2023-09-23T09:27:51.733479+0000","last_peered":"2023-09-23T09:27:51.733479+0000","last_clean":"2023-09-23T09:27:51.733479+0000","last_became_active":"2023-09-23T09:27:50.267036+0000","last_became_peered":"2023-09-23T09:27:50.267036+0000","last_unstale":"2023-09-23T09:27:51.733479+0000","last_undegraded":"2023-09-23T09:27:51.733479+0000","last_fullsized":"2023-09-23T09:27:51.733479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:42.866260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735091+0000","last_change":"2023-09-23T09:27:50.269934+0000","last_active":"2023-09-23T09:27:51.735091+0000","last_peered":"2023-09-23T09:27:51.735091+0000","last_clean":"2023-09-23T09:27:51.735091+0000","last_became_active":"2023-09-23T09:27:50.269060+0000","last_became_peered":"2023-09-23T09:27:50.269060+0000","last_unstale":"2023-09-23T09:27:51.735091+0000","last_undegraded":"2023-09-23T09:27:51.735091+0000","last_fullsized":"2023-09-23T09:27:51.735091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:47:31.191479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733829+0000","last_change":"2023-09-23T09:27:50.277885+0000","last_active":"2023-09-23T09:27:51.733829+0000","last_peered":"2023-09-23T09:27:51.733829+0000","last_clean":"2023-09-23T09:27:51.733829+0000","last_became_active":"2023-09-23T09:27:50.277576+0000","last_became_peered":"2023-09-23T09:27:50.277576+0000","last_unstale":"2023-09-23T09:27:51.733829+0000","last_undegraded":"2023-09-23T09:27:51.733829+0000","last_fullsized":"2023-09-23T09:27:51.733829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:03:43.338185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264326+0000","last_change":"2023-09-23T09:27:50.278225+0000","last_active":"2023-09-23T09:27:51.264326+0000","last_peered":"2023-09-23T09:27:51.264326+0000","last_clean":"2023-09-23T09:27:51.264326+0000","last_became_active":"2023-09-23T09:27:50.277891+0000","last_became_peered":"2023-09-23T09:27:50.277891+0000","last_unstale":"2023-09-23T09:27:51.264326+0000","last_undegraded":"2023-09-23T09:27:51.264326+0000","last_fullsized":"2023-09-23T09:27:51.264326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:27:08.961439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734854+0000","last_change":"2023-09-23T09:27:50.273445+0000","last_active":"2023-09-23T09:27:51.734854+0000","last_peered":"2023-09-23T09:27:51.734854+0000","last_clean":"2023-09-23T09:27:51.734854+0000","last_became_active":"2023-09-23T09:27:50.273124+0000","last_became_peered":"2023-09-23T09:27:50.273124+0000","last_unstale":"2023-09-23T09:27:51.734854+0000","last_undegraded":"2023-09-23T09:27:51.734854+0000","last_fullsized":"2023-09-23T09:27:51.734854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:46.030212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734647+0000","last_change":"2023-09-23T09:27:50.276164+0000","last_active":"2023-09-23T09:27:51.734647+0000","last_peered":"2023-09-23T09:27:51.734647+0000","last_clean":"2023-09-23T09:27:51.734647+0000","last_became_active":"2023-09-23T09:27:50.275860+0000","last_became_peered":"2023-09-23T09:27:50.275860+0000","last_unstale":"2023-09-23T09:27:51.734647+0000","last_undegraded":"2023-09-23T09:27:51.734647+0000","last_fullsized":"2023-09-23T09:27:51.734647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:53:50.521872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734518+0000","last_change":"2023-09-23T09:27:50.275221+0000","last_active":"2023-09-23T09:27:51.734518+0000","last_peered":"2023-09-23T09:27:51.734518+0000","last_clean":"2023-09-23T09:27:51.734518+0000","last_became_active":"2023-09-23T09:27:50.274920+0000","last_became_peered":"2023-09-23T09:27:50.274920+0000","last_unstale":"2023-09-23T09:27:51.734518+0000","last_undegraded":"2023-09-23T09:27:51.734518+0000","last_fullsized":"2023-09-23T09:27:51.734518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:45:35.023433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259534+0000","last_change":"2023-09-23T09:27:50.277776+0000","last_active":"2023-09-23T09:27:51.259534+0000","last_peered":"2023-09-23T09:27:51.259534+0000","last_clean":"2023-09-23T09:27:51.259534+0000","last_became_active":"2023-09-23T09:27:50.277291+0000","last_became_peered":"2023-09-23T09:27:50.277291+0000","last_unstale":"2023-09-23T09:27:51.259534+0000","last_undegraded":"2023-09-23T09:27:51.259534+0000","last_fullsized":"2023-09-23T09:27:51.259534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:23:46.984450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734180+0000","last_change":"2023-09-23T09:27:50.271219+0000","last_active":"2023-09-23T09:27:51.734180+0000","last_peered":"2023-09-23T09:27:51.734180+0000","last_clean":"2023-09-23T09:27:51.734180+0000","last_became_active":"2023-09-23T09:27:50.270747+0000","last_became_peered":"2023-09-23T09:27:50.270747+0000","last_unstale":"2023-09-23T09:27:51.734180+0000","last_undegraded":"2023-09-23T09:27:51.734180+0000","last_fullsized":"2023-09-23T09:27:51.734180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:19.237260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733913+0000","last_change":"2023-09-23T09:27:50.263794+0000","last_active":"2023-09-23T09:27:51.733913+0000","last_peered":"2023-09-23T09:27:51.733913+0000","last_clean":"2023-09-23T09:27:51.733913+0000","last_became_active":"2023-09-23T09:27:50.263459+0000","last_became_peered":"2023-09-23T09:27:50.263459+0000","last_unstale":"2023-09-23T09:27:51.733913+0000","last_undegraded":"2023-09-23T09:27:51.733913+0000","last_fullsized":"2023-09-23T09:27:51.733913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:05:37.618959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250071,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4510,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":95,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":31067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":426682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":1019618,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57599999999999996}]},{"osd":9,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":427264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.72599999999999998}]},{"osd":1,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.753}]},{"osd":2,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.88}]},{"osd":3,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.86299999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.92500000000000004}]},{"osd":5,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":1.016},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.95699999999999996}]},{"osd":6,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.94399999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.97499999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.98499999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.054},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.0660000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":427420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72399999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.78000000000000003}]},{"osd":3,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.749}]},{"osd":4,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.90400000000000003}]},{"osd":6,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.871},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.80400000000000005}]},{"osd":7,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.006}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.89100000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":1017981,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41199999999999998,"15min":0.40400000000000003},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.55400000000000005}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59299999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.311,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.745},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80900000000000005}]},{"osd":8,"last update":"Sat Sep 23 09:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69699999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":426682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.41199999999999998},"min":{"1min":0.33300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.30099999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.499,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.51200000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63200000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.57099999999999995},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98399999999999999,"5min":1.006,"15min":1.006},"last":0.65000000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.36299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.79200000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.747,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":1.077,"15min":1.077},"last":0.46600000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.61299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.56699999999999995,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73999999999999999}]},{"osd":7,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.61899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.77100000000000002}]},{"osd":8,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75800000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":428294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.37,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.74299999999999999}]},{"osd":2,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83199999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.29499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39900000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.42199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91900000000000004}]},{"osd":6,"last update":"Sat Sep 23 09:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89400000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81499999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.624},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85799999999999998}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72999999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":426772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 09:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.55100000000000005,"15min":0.51300000000000001},"min":{"1min":0.46400000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.31},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.439,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.32800000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":1.054,"15min":1.054},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.53100000000000003,"15min":0.504},"min":{"1min":0.376,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Sep 23 09:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.26400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81100000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.40200000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.57099999999999995,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.442}]},{"osd":5,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.52600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.308,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.55000000000000004,"15min":0.52000000000000002},"min":{"1min":0.40200000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41799999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.718,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.718},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.64100000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.47899999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.57199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":429644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.505,"15min":0.50700000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.754,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.5,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.97199999999999998,"5min":1.655,"15min":1.655},"last":0.78000000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.81299999999999994}]},{"osd":3,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82499999999999996,"5min":0.871,"15min":0.871},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.44,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71399999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73999999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.627},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64300000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.438,"15min":0.434},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.497},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.45700000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76000000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.40500000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.014,"15min":1.014},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.79500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.875}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.92600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":426635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.36499999999999999,"15min":0.34399999999999997},"min":{"1min":0.29299999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.35799999999999998},"min":{"1min":0.26600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38300000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.438,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60699999999999998,"15min":0.58599999999999997},"min":{"1min":0.48499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73199999999999998}]},{"osd":4,"last update":"Sat Sep 23 09:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.499,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.505},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45200000000000001}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.48799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.71099999999999997,"15min":0.69999999999999996},"min":{"1min":0.628,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.629}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.61499999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":4,"up_from":33,"seq":141733920809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":426704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.998,"15min":0.998},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.38},"min":{"1min":0.33900000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.751}]},{"osd":3,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.495,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.436}]},{"osd":5,"last update":"Sat Sep 23 09:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83999999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Sep 23 09:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.58599999999999997,"15min":0.56599999999999995},"min":{"1min":0.44400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.502},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.57099999999999995}]},{"osd":8,"last update":"Sat Sep 23 09:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71099999999999997}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.65300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":1017232,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34899999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65600000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71599999999999997,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.377,"15min":0.36099999999999999},"min":{"1min":0.311,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.437},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.46100000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53400000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.45400000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.375,"15min":0.375},"max":{"1min":0.80500000000000005,"5min":1.538,"15min":1.538},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.45400000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71399999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.54800000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.755}]},{"osd":8,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56299999999999994}]},{"osd":9,"last update":"Sat Sep 23 09:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84999999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T09:27:58.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T09:27:58.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:57 smithi098 ceph-mon[102386]: pgmap v254: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2023-09-23T09:27:58.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:57 smithi098 ceph-mon[102386]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:58.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:57 smithi098 ceph-mon[102386]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-23T09:27:58.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:57 smithi098 ceph-mon[102386]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:58.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:57 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/250118943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:58.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:57 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3379279953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:58.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:57 smithi123 ceph-mon[103577]: pgmap v254: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2023-09-23T09:27:58.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:57 smithi123 ceph-mon[103577]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:57 smithi123 ceph-mon[103577]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-23T09:27:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:57 smithi123 ceph-mon[103577]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:27:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:57 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/250118943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:27:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:57 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3379279953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:27:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.198+0000 7fd256278700 1 -- 172.21.15.45:0/2107822662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd250101890 msgr2=0x7fd250108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.198+0000 7fd256278700 1 --2- 172.21.15.45:0/2107822662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd250101890 0x7fd250108140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd24800b0d0 tx=0x7fd24800b3e0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 -- 172.21.15.45:0/2107822662 shutdown_connections 2023-09-23T09:27:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 --2- 172.21.15.45:0/2107822662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd250101890 0x7fd250108140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 --2- 172.21.15.45:0/2107822662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500ffd00 0x7fd250100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 --2- 172.21.15.45:0/2107822662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd250100ef0 0x7fd250101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 -- 172.21.15.45:0/2107822662 >> 172.21.15.45:0/2107822662 conn(0x7fd2500fb250 msgr2=0x7fd2500fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 -- 172.21.15.45:0/2107822662 shutdown_connections 2023-09-23T09:27:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 -- 172.21.15.45:0/2107822662 wait complete. 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 Processor -- start 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.199+0000 7fd256278700 1 -- start start 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2500ffd00 0x7fd250196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd250100ef0 0x7fd2501970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd250101890 0x7fd25019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019e6f0 con 0x7fd2500ffd00 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019e830 con 0x7fd250101890 2023-09-23T09:27:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019e9c0 con 0x7fd250100ef0 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2500ffd00 0x7fd250196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2500ffd00 0x7fd250196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59000/0 (socket says 172.21.15.45:59000) 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 -- 172.21.15.45:0/299127242 learned_addr learned my addr 172.21.15.45:0/299127242 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd254815700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd250101890 0x7fd25019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24f7fe700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd250100ef0 0x7fd2501970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 -- 172.21.15.45:0/299127242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd250100ef0 msgr2=0x7fd2501970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd250100ef0 0x7fd2501970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 -- 172.21.15.45:0/299127242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd250101890 msgr2=0x7fd25019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd250101890 0x7fd25019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 -- 172.21.15.45:0/299127242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd248009d00 con 0x7fd2500ffd00 2023-09-23T09:27:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24ffff700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2500ffd00 0x7fd250196b70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd24000d980 tx=0x7fd24000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:58.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24d7fa700 1 -- 172.21.15.45:0/299127242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240004020 con 0x7fd2500ffd00 2023-09-23T09:27:58.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25019eca0 con 0x7fd2500ffd00 2023-09-23T09:27:58.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24d7fa700 1 -- 172.21.15.45:0/299127242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd240009730 con 0x7fd2500ffd00 2023-09-23T09:27:58.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd24d7fa700 1 -- 172.21.15.45:0/299127242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240003a30 con 0x7fd2500ffd00 2023-09-23T09:27:58.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.200+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25019f1b0 con 0x7fd2500ffd00 2023-09-23T09:27:58.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.201+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25010b520 con 0x7fd2500ffd00 2023-09-23T09:27:58.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.203+0000 7fd24d7fa700 1 -- 172.21.15.45:0/299127242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fd2400098a0 con 0x7fd2500ffd00 2023-09-23T09:27:58.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.203+0000 7fd24d7fa700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd2380a1670 0x7fd2380a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:27:58.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.203+0000 7fd24d7fa700 1 -- 172.21.15.45:0/299127242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2400ccbd0 con 0x7fd2500ffd00 2023-09-23T09:27:58.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.203+0000 7fd24f7fe700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd2380a1670 0x7fd2380a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:27:58.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.204+0000 7fd24f7fe700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd2380a1670 0x7fd2380a3b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd244005fd0 tx=0x7fd24400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:27:58.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.207+0000 7fd24d7fa700 1 -- 172.21.15.45:0/299127242 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd240096210 con 0x7fd2500ffd00 2023-09-23T09:27:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.409+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2500686f0 con 0x7fd2380a1670 2023-09-23T09:27:58.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.421+0000 7fd24d7fa700 1 -- 172.21.15.45:0/299127242 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336007 (secure 0 0 0) 0x7fd2500686f0 con 0x7fd2380a1670 2023-09-23T09:27:58.426 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:27:58.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd2380a1670 msgr2=0x7fd2380a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:58.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd2380a1670 0x7fd2380a3b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd244005fd0 tx=0x7fd24400a560 comp rx=0 tx=0).stop 2023-09-23T09:27:58.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2500ffd00 msgr2=0x7fd250196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2500ffd00 0x7fd250196b70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd24000d980 tx=0x7fd24000dc90 comp rx=0 tx=0).stop 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 shutdown_connections 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd250101890 0x7fd25019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2500ffd00 0x7fd250196b70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd2380a1670 0x7fd2380a3b30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 --2- 172.21.15.45:0/299127242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd250100ef0 0x7fd2501970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 >> 172.21.15.45:0/299127242 conn(0x7fd2500fb250 msgr2=0x7fd250067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 shutdown_connections 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:27:58.425+0000 7fd256278700 1 -- 172.21.15.45:0/299127242 wait complete. 2023-09-23T09:27:58.429 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-09-23T09:27:58.439 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2023-09-23T09:27:58.322935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595890,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":154,"num_read_kb":134,"num_write":296,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":252,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334960,"kb_used_data":10732,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606864,"statfs":{"total":1151940427776,"available":1151597428736,"internally_reserved":0,"allocated":10989568,"data_stored":6902176,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5522,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":115,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.069014"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735132+0000","last_change":"2023-09-23T09:27:50.270599+0000","last_active":"2023-09-23T09:27:51.735132+0000","last_peered":"2023-09-23T09:27:51.735132+0000","last_clean":"2023-09-23T09:27:51.735132+0000","last_became_active":"2023-09-23T09:27:50.269926+0000","last_became_peered":"2023-09-23T09:27:50.269926+0000","last_unstale":"2023-09-23T09:27:51.735132+0000","last_undegraded":"2023-09-23T09:27:51.735132+0000","last_fullsized":"2023-09-23T09:27:51.735132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:12:08.908188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733803+0000","last_change":"2023-09-23T09:27:50.270451+0000","last_active":"2023-09-23T09:27:51.733803+0000","last_peered":"2023-09-23T09:27:51.733803+0000","last_clean":"2023-09-23T09:27:51.733803+0000","last_became_active":"2023-09-23T09:27:50.270055+0000","last_became_peered":"2023-09-23T09:27:50.270055+0000","last_unstale":"2023-09-23T09:27:51.733803+0000","last_undegraded":"2023-09-23T09:27:51.733803+0000","last_fullsized":"2023-09-23T09:27:51.733803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:56:43.388109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734671+0000","last_change":"2023-09-23T09:27:50.278101+0000","last_active":"2023-09-23T09:27:51.734671+0000","last_peered":"2023-09-23T09:27:51.734671+0000","last_clean":"2023-09-23T09:27:51.734671+0000","last_became_active":"2023-09-23T09:27:50.277739+0000","last_became_peered":"2023-09-23T09:27:50.277739+0000","last_unstale":"2023-09-23T09:27:51.734671+0000","last_undegraded":"2023-09-23T09:27:51.734671+0000","last_fullsized":"2023-09-23T09:27:51.734671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:52:31.582623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733930+0000","last_change":"2023-09-23T09:27:50.265962+0000","last_active":"2023-09-23T09:27:51.733930+0000","last_peered":"2023-09-23T09:27:51.733930+0000","last_clean":"2023-09-23T09:27:51.733930+0000","last_became_active":"2023-09-23T09:27:50.265681+0000","last_became_peered":"2023-09-23T09:27:50.265681+0000","last_unstale":"2023-09-23T09:27:51.733930+0000","last_undegraded":"2023-09-23T09:27:51.733930+0000","last_fullsized":"2023-09-23T09:27:51.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:12.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734705+0000","last_change":"2023-09-23T09:27:50.270356+0000","last_active":"2023-09-23T09:27:51.734705+0000","last_peered":"2023-09-23T09:27:51.734705+0000","last_clean":"2023-09-23T09:27:51.734705+0000","last_became_active":"2023-09-23T09:27:50.270124+0000","last_became_peered":"2023-09-23T09:27:50.270124+0000","last_unstale":"2023-09-23T09:27:51.734705+0000","last_undegraded":"2023-09-23T09:27:51.734705+0000","last_fullsized":"2023-09-23T09:27:51.734705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:14:00.508926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398274+0000","last_change":"2023-09-23T09:27:50.276122+0000","last_active":"2023-09-23T09:27:51.398274+0000","last_peered":"2023-09-23T09:27:51.398274+0000","last_clean":"2023-09-23T09:27:51.398274+0000","last_became_active":"2023-09-23T09:27:50.275847+0000","last_became_peered":"2023-09-23T09:27:50.275847+0000","last_unstale":"2023-09-23T09:27:51.398274+0000","last_undegraded":"2023-09-23T09:27:51.398274+0000","last_fullsized":"2023-09-23T09:27:51.398274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:04:03.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734273+0000","last_change":"2023-09-23T09:27:50.279420+0000","last_active":"2023-09-23T09:27:51.734273+0000","last_peered":"2023-09-23T09:27:51.734273+0000","last_clean":"2023-09-23T09:27:51.734273+0000","last_became_active":"2023-09-23T09:27:50.279120+0000","last_became_peered":"2023-09-23T09:27:50.279120+0000","last_unstale":"2023-09-23T09:27:51.734273+0000","last_undegraded":"2023-09-23T09:27:51.734273+0000","last_fullsized":"2023-09-23T09:27:51.734273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:48:03.461868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734300+0000","last_change":"2023-09-23T09:27:50.276364+0000","last_active":"2023-09-23T09:27:51.734300+0000","last_peered":"2023-09-23T09:27:51.734300+0000","last_clean":"2023-09-23T09:27:51.734300+0000","last_became_active":"2023-09-23T09:27:50.276060+0000","last_became_peered":"2023-09-23T09:27:50.276060+0000","last_unstale":"2023-09-23T09:27:51.734300+0000","last_undegraded":"2023-09-23T09:27:51.734300+0000","last_fullsized":"2023-09-23T09:27:51.734300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:26.998609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259923+0000","last_change":"2023-09-23T09:27:50.280302+0000","last_active":"2023-09-23T09:27:51.259923+0000","last_peered":"2023-09-23T09:27:51.259923+0000","last_clean":"2023-09-23T09:27:51.259923+0000","last_became_active":"2023-09-23T09:27:50.279295+0000","last_became_peered":"2023-09-23T09:27:50.279295+0000","last_unstale":"2023-09-23T09:27:51.259923+0000","last_undegraded":"2023-09-23T09:27:51.259923+0000","last_fullsized":"2023-09-23T09:27:51.259923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:15.329483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398526+0000","last_change":"2023-09-23T09:27:50.271876+0000","last_active":"2023-09-23T09:27:51.398526+0000","last_peered":"2023-09-23T09:27:51.398526+0000","last_clean":"2023-09-23T09:27:51.398526+0000","last_became_active":"2023-09-23T09:27:50.271576+0000","last_became_peered":"2023-09-23T09:27:50.271576+0000","last_unstale":"2023-09-23T09:27:51.398526+0000","last_undegraded":"2023-09-23T09:27:51.398526+0000","last_fullsized":"2023-09-23T09:27:51.398526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:17:22.656828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734649+0000","last_change":"2023-09-23T09:27:50.271405+0000","last_active":"2023-09-23T09:27:51.734649+0000","last_peered":"2023-09-23T09:27:51.734649+0000","last_clean":"2023-09-23T09:27:51.734649+0000","last_became_active":"2023-09-23T09:27:50.271087+0000","last_became_peered":"2023-09-23T09:27:50.271087+0000","last_unstale":"2023-09-23T09:27:51.734649+0000","last_undegraded":"2023-09-23T09:27:51.734649+0000","last_fullsized":"2023-09-23T09:27:51.734649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:09.363894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733876+0000","last_change":"2023-09-23T09:27:50.265855+0000","last_active":"2023-09-23T09:27:51.733876+0000","last_peered":"2023-09-23T09:27:51.733876+0000","last_clean":"2023-09-23T09:27:51.733876+0000","last_became_active":"2023-09-23T09:27:50.265101+0000","last_became_peered":"2023-09-23T09:27:50.265101+0000","last_unstale":"2023-09-23T09:27:51.733876+0000","last_undegraded":"2023-09-23T09:27:51.733876+0000","last_fullsized":"2023-09-23T09:27:51.733876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:52:40.456013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734201+0000","last_change":"2023-09-23T09:27:50.274405+0000","last_active":"2023-09-23T09:27:51.734201+0000","last_peered":"2023-09-23T09:27:51.734201+0000","last_clean":"2023-09-23T09:27:51.734201+0000","last_became_active":"2023-09-23T09:27:50.274036+0000","last_became_peered":"2023-09-23T09:27:50.274036+0000","last_unstale":"2023-09-23T09:27:51.734201+0000","last_undegraded":"2023-09-23T09:27:51.734201+0000","last_fullsized":"2023-09-23T09:27:51.734201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:38.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733331+0000","last_change":"2023-09-23T09:27:50.272237+0000","last_active":"2023-09-23T09:27:51.733331+0000","last_peered":"2023-09-23T09:27:51.733331+0000","last_clean":"2023-09-23T09:27:51.733331+0000","last_became_active":"2023-09-23T09:27:50.271897+0000","last_became_peered":"2023-09-23T09:27:50.271897+0000","last_unstale":"2023-09-23T09:27:51.733331+0000","last_undegraded":"2023-09-23T09:27:51.733331+0000","last_fullsized":"2023-09-23T09:27:51.733331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:31:31.115205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733452+0000","last_change":"2023-09-23T09:27:50.268902+0000","last_active":"2023-09-23T09:27:51.733452+0000","last_peered":"2023-09-23T09:27:51.733452+0000","last_clean":"2023-09-23T09:27:51.733452+0000","last_became_active":"2023-09-23T09:27:50.268590+0000","last_became_peered":"2023-09-23T09:27:50.268590+0000","last_unstale":"2023-09-23T09:27:51.733452+0000","last_undegraded":"2023-09-23T09:27:51.733452+0000","last_fullsized":"2023-09-23T09:27:51.733452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:54:39.384835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259359+0000","last_change":"2023-09-23T09:27:50.276934+0000","last_active":"2023-09-23T09:27:51.259359+0000","last_peered":"2023-09-23T09:27:51.259359+0000","last_clean":"2023-09-23T09:27:51.259359+0000","last_became_active":"2023-09-23T09:27:50.276645+0000","last_became_peered":"2023-09-23T09:27:50.276645+0000","last_unstale":"2023-09-23T09:27:51.259359+0000","last_undegraded":"2023-09-23T09:27:51.259359+0000","last_fullsized":"2023-09-23T09:27:51.259359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:07.100550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735229+0000","last_change":"2023-09-23T09:27:50.270781+0000","last_active":"2023-09-23T09:27:51.735229+0000","last_peered":"2023-09-23T09:27:51.735229+0000","last_clean":"2023-09-23T09:27:51.735229+0000","last_became_active":"2023-09-23T09:27:50.270319+0000","last_became_peered":"2023-09-23T09:27:50.270319+0000","last_unstale":"2023-09-23T09:27:51.735229+0000","last_undegraded":"2023-09-23T09:27:51.735229+0000","last_fullsized":"2023-09-23T09:27:51.735229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:42:44.882122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259692+0000","last_change":"2023-09-23T09:27:50.278775+0000","last_active":"2023-09-23T09:27:51.259692+0000","last_peered":"2023-09-23T09:27:51.259692+0000","last_clean":"2023-09-23T09:27:51.259692+0000","last_became_active":"2023-09-23T09:27:50.278375+0000","last_became_peered":"2023-09-23T09:27:50.278375+0000","last_unstale":"2023-09-23T09:27:51.259692+0000","last_undegraded":"2023-09-23T09:27:51.259692+0000","last_fullsized":"2023-09-23T09:27:51.259692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:14:00.076810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264221+0000","last_change":"2023-09-23T09:27:50.270141+0000","last_active":"2023-09-23T09:27:51.264221+0000","last_peered":"2023-09-23T09:27:51.264221+0000","last_clean":"2023-09-23T09:27:51.264221+0000","last_became_active":"2023-09-23T09:27:50.269849+0000","last_became_peered":"2023-09-23T09:27:50.269849+0000","last_unstale":"2023-09-23T09:27:51.264221+0000","last_undegraded":"2023-09-23T09:27:51.264221+0000","last_fullsized":"2023-09-23T09:27:51.264221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:49:41.837886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704055+0000","last_change":"2023-09-23T09:27:49.654458+0000","last_active":"2023-09-23T09:27:54.704055+0000","last_peered":"2023-09-23T09:27:54.704055+0000","last_clean":"2023-09-23T09:27:54.704055+0000","last_became_active":"2023-09-23T09:27:49.654084+0000","last_became_peered":"2023-09-23T09:27:49.654084+0000","last_unstale":"2023-09-23T09:27:54.704055+0000","last_undegraded":"2023-09-23T09:27:54.704055+0000","last_fullsized":"2023-09-23T09:27:54.704055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:47:32.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734956+0000","last_change":"2023-09-23T09:27:50.276558+0000","last_active":"2023-09-23T09:27:51.734956+0000","last_peered":"2023-09-23T09:27:51.734956+0000","last_clean":"2023-09-23T09:27:51.734956+0000","last_became_active":"2023-09-23T09:27:50.276255+0000","last_became_peered":"2023-09-23T09:27:50.276255+0000","last_unstale":"2023-09-23T09:27:51.734956+0000","last_undegraded":"2023-09-23T09:27:51.734956+0000","last_fullsized":"2023-09-23T09:27:51.734956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:13:09.870383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735166+0000","last_change":"2023-09-23T09:27:49.659256+0000","last_active":"2023-09-23T09:27:51.735166+0000","last_peered":"2023-09-23T09:27:51.735166+0000","last_clean":"2023-09-23T09:27:51.735166+0000","last_became_active":"2023-09-23T09:27:49.658793+0000","last_became_peered":"2023-09-23T09:27:49.658793+0000","last_unstale":"2023-09-23T09:27:51.735166+0000","last_undegraded":"2023-09-23T09:27:51.735166+0000","last_fullsized":"2023-09-23T09:27:51.735166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:03.416666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733674+0000","last_change":"2023-09-23T09:27:50.277338+0000","last_active":"2023-09-23T09:27:51.733674+0000","last_peered":"2023-09-23T09:27:51.733674+0000","last_clean":"2023-09-23T09:27:51.733674+0000","last_became_active":"2023-09-23T09:27:50.277011+0000","last_became_peered":"2023-09-23T09:27:50.277011+0000","last_unstale":"2023-09-23T09:27:51.733674+0000","last_undegraded":"2023-09-23T09:27:51.733674+0000","last_fullsized":"2023-09-23T09:27:51.733674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:52:23.028794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702688+0000","last_change":"2023-09-23T09:27:49.660679+0000","last_active":"2023-09-23T09:27:54.702688+0000","last_peered":"2023-09-23T09:27:54.702688+0000","last_clean":"2023-09-23T09:27:54.702688+0000","last_became_active":"2023-09-23T09:27:49.660201+0000","last_became_peered":"2023-09-23T09:27:49.660201+0000","last_unstale":"2023-09-23T09:27:54.702688+0000","last_undegraded":"2023-09-23T09:27:54.702688+0000","last_fullsized":"2023-09-23T09:27:54.702688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:33:56.205322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259494+0000","last_change":"2023-09-23T09:27:50.278373+0000","last_active":"2023-09-23T09:27:51.259494+0000","last_peered":"2023-09-23T09:27:51.259494+0000","last_clean":"2023-09-23T09:27:51.259494+0000","last_became_active":"2023-09-23T09:27:50.277837+0000","last_became_peered":"2023-09-23T09:27:50.277837+0000","last_unstale":"2023-09-23T09:27:51.259494+0000","last_undegraded":"2023-09-23T09:27:51.259494+0000","last_fullsized":"2023-09-23T09:27:51.259494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:24:26.914509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.117099+0000","last_change":"2023-09-23T09:27:49.657578+0000","last_active":"2023-09-23T09:27:51.117099+0000","last_peered":"2023-09-23T09:27:51.117099+0000","last_clean":"2023-09-23T09:27:51.117099+0000","last_became_active":"2023-09-23T09:27:49.657278+0000","last_became_peered":"2023-09-23T09:27:49.657278+0000","last_unstale":"2023-09-23T09:27:51.117099+0000","last_undegraded":"2023-09-23T09:27:51.117099+0000","last_fullsized":"2023-09-23T09:27:51.117099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:53.440997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734804+0000","last_change":"2023-09-23T09:27:50.279784+0000","last_active":"2023-09-23T09:27:51.734804+0000","last_peered":"2023-09-23T09:27:51.734804+0000","last_clean":"2023-09-23T09:27:51.734804+0000","last_became_active":"2023-09-23T09:27:50.279473+0000","last_became_peered":"2023-09-23T09:27:50.279473+0000","last_unstale":"2023-09-23T09:27:51.734804+0000","last_undegraded":"2023-09-23T09:27:51.734804+0000","last_fullsized":"2023-09-23T09:27:51.734804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:30:13.010556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690814+0000","last_change":"2023-09-23T09:27:49.655391+0000","last_active":"2023-09-23T09:27:53.690814+0000","last_peered":"2023-09-23T09:27:53.690814+0000","last_clean":"2023-09-23T09:27:53.690814+0000","last_became_active":"2023-09-23T09:27:49.655067+0000","last_became_peered":"2023-09-23T09:27:49.655067+0000","last_unstale":"2023-09-23T09:27:53.690814+0000","last_undegraded":"2023-09-23T09:27:53.690814+0000","last_fullsized":"2023-09-23T09:27:53.690814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.701471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733903+0000","last_change":"2023-09-23T09:27:50.275688+0000","last_active":"2023-09-23T09:27:51.733903+0000","last_peered":"2023-09-23T09:27:51.733903+0000","last_clean":"2023-09-23T09:27:51.733903+0000","last_became_active":"2023-09-23T09:27:50.275356+0000","last_became_peered":"2023-09-23T09:27:50.275356+0000","last_unstale":"2023-09-23T09:27:51.733903+0000","last_undegraded":"2023-09-23T09:27:51.733903+0000","last_fullsized":"2023-09-23T09:27:51.733903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:43:31.829209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.663153+0000","last_change":"2023-09-23T09:27:49.657066+0000","last_active":"2023-09-23T09:27:50.663153+0000","last_peered":"2023-09-23T09:27:50.663153+0000","last_clean":"2023-09-23T09:27:50.663153+0000","last_became_active":"2023-09-23T09:27:49.656754+0000","last_became_peered":"2023-09-23T09:27:49.656754+0000","last_unstale":"2023-09-23T09:27:50.663153+0000","last_undegraded":"2023-09-23T09:27:50.663153+0000","last_fullsized":"2023-09-23T09:27:50.663153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:52.277474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734458+0000","last_change":"2023-09-23T09:27:50.273705+0000","last_active":"2023-09-23T09:27:51.734458+0000","last_peered":"2023-09-23T09:27:51.734458+0000","last_clean":"2023-09-23T09:27:51.734458+0000","last_became_active":"2023-09-23T09:27:50.272245+0000","last_became_peered":"2023-09-23T09:27:50.272245+0000","last_unstale":"2023-09-23T09:27:51.734458+0000","last_undegraded":"2023-09-23T09:27:51.734458+0000","last_fullsized":"2023-09-23T09:27:51.734458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:45:57.318326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.255097+0000","last_change":"2023-09-23T09:27:49.651958+0000","last_active":"2023-09-23T09:27:50.255097+0000","last_peered":"2023-09-23T09:27:50.255097+0000","last_clean":"2023-09-23T09:27:50.255097+0000","last_became_active":"2023-09-23T09:27:49.651058+0000","last_became_peered":"2023-09-23T09:27:49.651058+0000","last_unstale":"2023-09-23T09:27:50.255097+0000","last_undegraded":"2023-09-23T09:27:50.255097+0000","last_fullsized":"2023-09-23T09:27:50.255097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:26:56.360566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734149+0000","last_change":"2023-09-23T09:27:50.270804+0000","last_active":"2023-09-23T09:27:51.734149+0000","last_peered":"2023-09-23T09:27:51.734149+0000","last_clean":"2023-09-23T09:27:51.734149+0000","last_became_active":"2023-09-23T09:27:50.270044+0000","last_became_peered":"2023-09-23T09:27:50.270044+0000","last_unstale":"2023-09-23T09:27:51.734149+0000","last_undegraded":"2023-09-23T09:27:51.734149+0000","last_fullsized":"2023-09-23T09:27:51.734149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:59:20.821949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734563+0000","last_change":"2023-09-23T09:27:49.657030+0000","last_active":"2023-09-23T09:27:51.734563+0000","last_peered":"2023-09-23T09:27:51.734563+0000","last_clean":"2023-09-23T09:27:51.734563+0000","last_became_active":"2023-09-23T09:27:49.656710+0000","last_became_peered":"2023-09-23T09:27:49.656710+0000","last_unstale":"2023-09-23T09:27:51.734563+0000","last_undegraded":"2023-09-23T09:27:51.734563+0000","last_fullsized":"2023-09-23T09:27:51.734563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:24:59.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259415+0000","last_change":"2023-09-23T09:27:50.271688+0000","last_active":"2023-09-23T09:27:51.259415+0000","last_peered":"2023-09-23T09:27:51.259415+0000","last_clean":"2023-09-23T09:27:51.259415+0000","last_became_active":"2023-09-23T09:27:50.271276+0000","last_became_peered":"2023-09-23T09:27:50.271276+0000","last_unstale":"2023-09-23T09:27:51.259415+0000","last_undegraded":"2023-09-23T09:27:51.259415+0000","last_fullsized":"2023-09-23T09:27:51.259415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:40:48.333279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.692028+0000","last_change":"2023-09-23T09:27:49.655183+0000","last_active":"2023-09-23T09:27:53.692028+0000","last_peered":"2023-09-23T09:27:53.692028+0000","last_clean":"2023-09-23T09:27:53.692028+0000","last_became_active":"2023-09-23T09:27:49.654875+0000","last_became_peered":"2023-09-23T09:27:49.654875+0000","last_unstale":"2023-09-23T09:27:53.692028+0000","last_undegraded":"2023-09-23T09:27:53.692028+0000","last_fullsized":"2023-09-23T09:27:53.692028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:12:49.944732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259843+0000","last_change":"2023-09-23T09:27:50.276347+0000","last_active":"2023-09-23T09:27:51.259843+0000","last_peered":"2023-09-23T09:27:51.259843+0000","last_clean":"2023-09-23T09:27:51.259843+0000","last_became_active":"2023-09-23T09:27:50.276095+0000","last_became_peered":"2023-09-23T09:27:50.276095+0000","last_unstale":"2023-09-23T09:27:51.259843+0000","last_undegraded":"2023-09-23T09:27:51.259843+0000","last_fullsized":"2023-09-23T09:27:51.259843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:42.952808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.692327+0000","last_change":"2023-09-23T09:27:49.657115+0000","last_active":"2023-09-23T09:27:53.692327+0000","last_peered":"2023-09-23T09:27:53.692327+0000","last_clean":"2023-09-23T09:27:53.692327+0000","last_became_active":"2023-09-23T09:27:49.656816+0000","last_became_peered":"2023-09-23T09:27:49.656816+0000","last_unstale":"2023-09-23T09:27:53.692327+0000","last_undegraded":"2023-09-23T09:27:53.692327+0000","last_fullsized":"2023-09-23T09:27:53.692327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:37.205221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733619+0000","last_change":"2023-09-23T09:27:50.277477+0000","last_active":"2023-09-23T09:27:51.733619+0000","last_peered":"2023-09-23T09:27:51.733619+0000","last_clean":"2023-09-23T09:27:51.733619+0000","last_became_active":"2023-09-23T09:27:50.277200+0000","last_became_peered":"2023-09-23T09:27:50.277200+0000","last_unstale":"2023-09-23T09:27:51.733619+0000","last_undegraded":"2023-09-23T09:27:51.733619+0000","last_fullsized":"2023-09-23T09:27:51.733619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:45:03.167471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734861+0000","last_change":"2023-09-23T09:27:49.663678+0000","last_active":"2023-09-23T09:27:51.734861+0000","last_peered":"2023-09-23T09:27:51.734861+0000","last_clean":"2023-09-23T09:27:51.734861+0000","last_became_active":"2023-09-23T09:27:49.663142+0000","last_became_peered":"2023-09-23T09:27:49.663142+0000","last_unstale":"2023-09-23T09:27:51.734861+0000","last_undegraded":"2023-09-23T09:27:51.734861+0000","last_fullsized":"2023-09-23T09:27:51.734861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:14.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259656+0000","last_change":"2023-09-23T09:27:50.275044+0000","last_active":"2023-09-23T09:27:51.259656+0000","last_peered":"2023-09-23T09:27:51.259656+0000","last_clean":"2023-09-23T09:27:51.259656+0000","last_became_active":"2023-09-23T09:27:50.274752+0000","last_became_peered":"2023-09-23T09:27:50.274752+0000","last_unstale":"2023-09-23T09:27:51.259656+0000","last_undegraded":"2023-09-23T09:27:51.259656+0000","last_fullsized":"2023-09-23T09:27:51.259656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:19.914752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.691494+0000","last_change":"2023-09-23T09:27:49.653694+0000","last_active":"2023-09-23T09:27:53.691494+0000","last_peered":"2023-09-23T09:27:53.691494+0000","last_clean":"2023-09-23T09:27:53.691494+0000","last_became_active":"2023-09-23T09:27:49.653355+0000","last_became_peered":"2023-09-23T09:27:49.653355+0000","last_unstale":"2023-09-23T09:27:53.691494+0000","last_undegraded":"2023-09-23T09:27:53.691494+0000","last_fullsized":"2023-09-23T09:27:53.691494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:19:53.282723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.272849+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.272411+0000","last_became_peered":"2023-09-23T09:27:50.272411+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:01:41.972357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702769+0000","last_change":"2023-09-23T09:27:49.652769+0000","last_active":"2023-09-23T09:27:54.702769+0000","last_peered":"2023-09-23T09:27:54.702769+0000","last_clean":"2023-09-23T09:27:54.702769+0000","last_became_active":"2023-09-23T09:27:49.652440+0000","last_became_peered":"2023-09-23T09:27:49.652440+0000","last_unstale":"2023-09-23T09:27:54.702769+0000","last_undegraded":"2023-09-23T09:27:54.702769+0000","last_fullsized":"2023-09-23T09:27:54.702769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:02.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734000+0000","last_change":"2023-09-23T09:27:50.272105+0000","last_active":"2023-09-23T09:27:51.734000+0000","last_peered":"2023-09-23T09:27:51.734000+0000","last_clean":"2023-09-23T09:27:51.734000+0000","last_became_active":"2023-09-23T09:27:50.271766+0000","last_became_peered":"2023-09-23T09:27:50.271766+0000","last_unstale":"2023-09-23T09:27:51.734000+0000","last_undegraded":"2023-09-23T09:27:51.734000+0000","last_fullsized":"2023-09-23T09:27:51.734000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:46:46.012381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690285+0000","last_change":"2023-09-23T09:27:49.663473+0000","last_active":"2023-09-23T09:27:53.690285+0000","last_peered":"2023-09-23T09:27:53.690285+0000","last_clean":"2023-09-23T09:27:53.690285+0000","last_became_active":"2023-09-23T09:27:49.663085+0000","last_became_peered":"2023-09-23T09:27:49.663085+0000","last_unstale":"2023-09-23T09:27:53.690285+0000","last_undegraded":"2023-09-23T09:27:53.690285+0000","last_fullsized":"2023-09-23T09:27:53.690285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:28:05.246720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734657+0000","last_change":"2023-09-23T09:27:50.274172+0000","last_active":"2023-09-23T09:27:51.734657+0000","last_peered":"2023-09-23T09:27:51.734657+0000","last_clean":"2023-09-23T09:27:51.734657+0000","last_became_active":"2023-09-23T09:27:50.273874+0000","last_became_peered":"2023-09-23T09:27:50.273874+0000","last_unstale":"2023-09-23T09:27:51.734657+0000","last_undegraded":"2023-09-23T09:27:51.734657+0000","last_fullsized":"2023-09-23T09:27:51.734657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:04:06.864055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:56.267519+0000","last_change":"2023-09-23T09:27:49.654341+0000","last_active":"2023-09-23T09:27:56.267519+0000","last_peered":"2023-09-23T09:27:56.267519+0000","last_clean":"2023-09-23T09:27:56.267519+0000","last_became_active":"2023-09-23T09:27:49.653936+0000","last_became_peered":"2023-09-23T09:27:49.653936+0000","last_unstale":"2023-09-23T09:27:56.267519+0000","last_undegraded":"2023-09-23T09:27:56.267519+0000","last_fullsized":"2023-09-23T09:27:56.267519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:48.876752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734194+0000","last_change":"2023-09-23T09:27:50.278620+0000","last_active":"2023-09-23T09:27:51.734194+0000","last_peered":"2023-09-23T09:27:51.734194+0000","last_clean":"2023-09-23T09:27:51.734194+0000","last_became_active":"2023-09-23T09:27:50.278289+0000","last_became_peered":"2023-09-23T09:27:50.278289+0000","last_unstale":"2023-09-23T09:27:51.734194+0000","last_undegraded":"2023-09-23T09:27:51.734194+0000","last_fullsized":"2023-09-23T09:27:51.734194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:20:44.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704421+0000","last_change":"2023-09-23T09:27:49.662970+0000","last_active":"2023-09-23T09:27:54.704421+0000","last_peered":"2023-09-23T09:27:54.704421+0000","last_clean":"2023-09-23T09:27:54.704421+0000","last_became_active":"2023-09-23T09:27:49.662664+0000","last_became_peered":"2023-09-23T09:27:49.662664+0000","last_unstale":"2023-09-23T09:27:54.704421+0000","last_undegraded":"2023-09-23T09:27:54.704421+0000","last_fullsized":"2023-09-23T09:27:54.704421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:13.189686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734127+0000","last_change":"2023-09-23T09:27:50.278475+0000","last_active":"2023-09-23T09:27:51.734127+0000","last_peered":"2023-09-23T09:27:51.734127+0000","last_clean":"2023-09-23T09:27:51.734127+0000","last_became_active":"2023-09-23T09:27:50.277802+0000","last_became_peered":"2023-09-23T09:27:50.277802+0000","last_unstale":"2023-09-23T09:27:51.734127+0000","last_undegraded":"2023-09-23T09:27:51.734127+0000","last_fullsized":"2023-09-23T09:27:51.734127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:44.608172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734574+0000","last_change":"2023-09-23T09:27:49.657191+0000","last_active":"2023-09-23T09:27:51.734574+0000","last_peered":"2023-09-23T09:27:51.734574+0000","last_clean":"2023-09-23T09:27:51.734574+0000","last_became_active":"2023-09-23T09:27:49.656863+0000","last_became_peered":"2023-09-23T09:27:49.656863+0000","last_unstale":"2023-09-23T09:27:51.734574+0000","last_undegraded":"2023-09-23T09:27:51.734574+0000","last_fullsized":"2023-09-23T09:27:51.734574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:10.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734984+0000","last_change":"2023-09-23T09:27:50.274609+0000","last_active":"2023-09-23T09:27:51.734984+0000","last_peered":"2023-09-23T09:27:51.734984+0000","last_clean":"2023-09-23T09:27:51.734984+0000","last_became_active":"2023-09-23T09:27:50.274316+0000","last_became_peered":"2023-09-23T09:27:50.274316+0000","last_unstale":"2023-09-23T09:27:51.734984+0000","last_undegraded":"2023-09-23T09:27:51.734984+0000","last_fullsized":"2023-09-23T09:27:51.734984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:10:51.284306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259542+0000","last_change":"2023-09-23T09:27:49.659410+0000","last_active":"2023-09-23T09:27:51.259542+0000","last_peered":"2023-09-23T09:27:51.259542+0000","last_clean":"2023-09-23T09:27:51.259542+0000","last_became_active":"2023-09-23T09:27:49.659087+0000","last_became_peered":"2023-09-23T09:27:49.659087+0000","last_unstale":"2023-09-23T09:27:51.259542+0000","last_undegraded":"2023-09-23T09:27:51.259542+0000","last_fullsized":"2023-09-23T09:27:51.259542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:23:41.677416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398391+0000","last_change":"2023-09-23T09:27:50.268568+0000","last_active":"2023-09-23T09:27:51.398391+0000","last_peered":"2023-09-23T09:27:51.398391+0000","last_clean":"2023-09-23T09:27:51.398391+0000","last_became_active":"2023-09-23T09:27:50.268268+0000","last_became_peered":"2023-09-23T09:27:50.268268+0000","last_unstale":"2023-09-23T09:27:51.398391+0000","last_undegraded":"2023-09-23T09:27:51.398391+0000","last_fullsized":"2023-09-23T09:27:51.398391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:59:22.516794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734411+0000","last_change":"2023-09-23T09:27:49.657160+0000","last_active":"2023-09-23T09:27:51.734411+0000","last_peered":"2023-09-23T09:27:51.734411+0000","last_clean":"2023-09-23T09:27:51.734411+0000","last_became_active":"2023-09-23T09:27:49.656853+0000","last_became_peered":"2023-09-23T09:27:49.656853+0000","last_unstale":"2023-09-23T09:27:51.734411+0000","last_undegraded":"2023-09-23T09:27:51.734411+0000","last_fullsized":"2023-09-23T09:27:51.734411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:17.489250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398429+0000","last_change":"2023-09-23T09:27:50.272495+0000","last_active":"2023-09-23T09:27:51.398429+0000","last_peered":"2023-09-23T09:27:51.398429+0000","last_clean":"2023-09-23T09:27:51.398429+0000","last_became_active":"2023-09-23T09:27:50.272207+0000","last_became_peered":"2023-09-23T09:27:50.272207+0000","last_unstale":"2023-09-23T09:27:51.398429+0000","last_undegraded":"2023-09-23T09:27:51.398429+0000","last_fullsized":"2023-09-23T09:27:51.398429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:24.952253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734722+0000","last_change":"2023-09-23T09:27:50.275551+0000","last_active":"2023-09-23T09:27:51.734722+0000","last_peered":"2023-09-23T09:27:51.734722+0000","last_clean":"2023-09-23T09:27:51.734722+0000","last_became_active":"2023-09-23T09:27:50.275235+0000","last_became_peered":"2023-09-23T09:27:50.275235+0000","last_unstale":"2023-09-23T09:27:51.734722+0000","last_undegraded":"2023-09-23T09:27:51.734722+0000","last_fullsized":"2023-09-23T09:27:51.734722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:41.009566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734928+0000","last_change":"2023-09-23T09:27:49.652889+0000","last_active":"2023-09-23T09:27:51.734928+0000","last_peered":"2023-09-23T09:27:51.734928+0000","last_clean":"2023-09-23T09:27:51.734928+0000","last_became_active":"2023-09-23T09:27:49.652571+0000","last_became_peered":"2023-09-23T09:27:49.652571+0000","last_unstale":"2023-09-23T09:27:51.734928+0000","last_undegraded":"2023-09-23T09:27:51.734928+0000","last_fullsized":"2023-09-23T09:27:51.734928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:22.484052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734814+0000","last_change":"2023-09-23T09:27:50.278419+0000","last_active":"2023-09-23T09:27:51.734814+0000","last_peered":"2023-09-23T09:27:51.734814+0000","last_clean":"2023-09-23T09:27:51.734814+0000","last_became_active":"2023-09-23T09:27:50.278071+0000","last_became_peered":"2023-09-23T09:27:50.278071+0000","last_unstale":"2023-09-23T09:27:51.734814+0000","last_undegraded":"2023-09-23T09:27:51.734814+0000","last_fullsized":"2023-09-23T09:27:51.734814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:28:45.896479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:55.260221+0000","last_change":"2023-09-23T09:27:49.651251+0000","last_active":"2023-09-23T09:27:55.260221+0000","last_peered":"2023-09-23T09:27:55.260221+0000","last_clean":"2023-09-23T09:27:55.260221+0000","last_became_active":"2023-09-23T09:27:49.650933+0000","last_became_peered":"2023-09-23T09:27:49.650933+0000","last_unstale":"2023-09-23T09:27:55.260221+0000","last_undegraded":"2023-09-23T09:27:55.260221+0000","last_fullsized":"2023-09-23T09:27:55.260221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:00:54.100400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.264684+0000","last_change":"2023-09-23T09:26:51.296669+0000","last_active":"2023-09-23T09:27:50.264684+0000","last_peered":"2023-09-23T09:27:50.264684+0000","last_clean":"2023-09-23T09:27:50.264684+0000","last_became_active":"2023-09-23T09:26:51.296229+0000","last_became_peered":"2023-09-23T09:26:51.296229+0000","last_unstale":"2023-09-23T09:27:50.264684+0000","last_undegraded":"2023-09-23T09:27:50.264684+0000","last_fullsized":"2023-09-23T09:27:50.264684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_clean_scrub_stamp":"2023-09-23T09:24:01.204422+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:31:24.888152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734832+0000","last_change":"2023-09-23T09:27:50.267662+0000","last_active":"2023-09-23T09:27:51.734832+0000","last_peered":"2023-09-23T09:27:51.734832+0000","last_clean":"2023-09-23T09:27:51.734832+0000","last_became_active":"2023-09-23T09:27:50.267374+0000","last_became_peered":"2023-09-23T09:27:50.267374+0000","last_unstale":"2023-09-23T09:27:51.734832+0000","last_undegraded":"2023-09-23T09:27:51.734832+0000","last_fullsized":"2023-09-23T09:27:51.734832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:22.797939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.701373+0000","last_change":"2023-09-23T09:27:49.659773+0000","last_active":"2023-09-23T09:27:54.701373+0000","last_peered":"2023-09-23T09:27:54.701373+0000","last_clean":"2023-09-23T09:27:54.701373+0000","last_became_active":"2023-09-23T09:27:49.659526+0000","last_became_peered":"2023-09-23T09:27:49.659526+0000","last_unstale":"2023-09-23T09:27:54.701373+0000","last_undegraded":"2023-09-23T09:27:54.701373+0000","last_fullsized":"2023-09-23T09:27:54.701373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:27.423680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689202+0000","last_change":"2023-09-23T09:27:49.661336+0000","last_active":"2023-09-23T09:27:53.689202+0000","last_peered":"2023-09-23T09:27:53.689202+0000","last_clean":"2023-09-23T09:27:53.689202+0000","last_became_active":"2023-09-23T09:27:49.661038+0000","last_became_peered":"2023-09-23T09:27:49.661038+0000","last_unstale":"2023-09-23T09:27:53.689202+0000","last_undegraded":"2023-09-23T09:27:53.689202+0000","last_fullsized":"2023-09-23T09:27:53.689202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:20.634410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735024+0000","last_change":"2023-09-23T09:27:50.272485+0000","last_active":"2023-09-23T09:27:51.735024+0000","last_peered":"2023-09-23T09:27:51.735024+0000","last_clean":"2023-09-23T09:27:51.735024+0000","last_became_active":"2023-09-23T09:27:50.272164+0000","last_became_peered":"2023-09-23T09:27:50.272164+0000","last_unstale":"2023-09-23T09:27:51.735024+0000","last_undegraded":"2023-09-23T09:27:51.735024+0000","last_fullsized":"2023-09-23T09:27:51.735024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:48:00.596034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702774+0000","last_change":"2023-09-23T09:27:49.657563+0000","last_active":"2023-09-23T09:27:54.702774+0000","last_peered":"2023-09-23T09:27:54.702774+0000","last_clean":"2023-09-23T09:27:54.702774+0000","last_became_active":"2023-09-23T09:27:49.657232+0000","last_became_peered":"2023-09-23T09:27:49.657232+0000","last_unstale":"2023-09-23T09:27:54.702774+0000","last_undegraded":"2023-09-23T09:27:54.702774+0000","last_fullsized":"2023-09-23T09:27:54.702774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:29:10.640013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735007+0000","last_change":"2023-09-23T09:27:50.274959+0000","last_active":"2023-09-23T09:27:51.735007+0000","last_peered":"2023-09-23T09:27:51.735007+0000","last_clean":"2023-09-23T09:27:51.735007+0000","last_became_active":"2023-09-23T09:27:50.274614+0000","last_became_peered":"2023-09-23T09:27:50.274614+0000","last_unstale":"2023-09-23T09:27:51.735007+0000","last_undegraded":"2023-09-23T09:27:51.735007+0000","last_fullsized":"2023-09-23T09:27:51.735007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:00:09.831103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690819+0000","last_change":"2023-09-23T09:27:49.654673+0000","last_active":"2023-09-23T09:27:53.690819+0000","last_peered":"2023-09-23T09:27:53.690819+0000","last_clean":"2023-09-23T09:27:53.690819+0000","last_became_active":"2023-09-23T09:27:49.654356+0000","last_became_peered":"2023-09-23T09:27:49.654356+0000","last_unstale":"2023-09-23T09:27:53.690819+0000","last_undegraded":"2023-09-23T09:27:53.690819+0000","last_fullsized":"2023-09-23T09:27:53.690819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:39:50.628346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.267368+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.266962+0000","last_became_peered":"2023-09-23T09:27:50.266962+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:24.660228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690757+0000","last_change":"2023-09-23T09:27:49.661200+0000","last_active":"2023-09-23T09:27:53.690757+0000","last_peered":"2023-09-23T09:27:53.690757+0000","last_clean":"2023-09-23T09:27:53.690757+0000","last_became_active":"2023-09-23T09:27:49.660902+0000","last_became_peered":"2023-09-23T09:27:49.660902+0000","last_unstale":"2023-09-23T09:27:53.690757+0000","last_undegraded":"2023-09-23T09:27:53.690757+0000","last_fullsized":"2023-09-23T09:27:53.690757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:41.036216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735005+0000","last_change":"2023-09-23T09:27:50.271834+0000","last_active":"2023-09-23T09:27:51.735005+0000","last_peered":"2023-09-23T09:27:51.735005+0000","last_clean":"2023-09-23T09:27:51.735005+0000","last_became_active":"2023-09-23T09:27:50.271519+0000","last_became_peered":"2023-09-23T09:27:50.271519+0000","last_unstale":"2023-09-23T09:27:51.735005+0000","last_undegraded":"2023-09-23T09:27:51.735005+0000","last_fullsized":"2023-09-23T09:27:51.735005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:08.036615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689622+0000","last_change":"2023-09-23T09:27:49.656794+0000","last_active":"2023-09-23T09:27:53.689622+0000","last_peered":"2023-09-23T09:27:53.689622+0000","last_clean":"2023-09-23T09:27:53.689622+0000","last_became_active":"2023-09-23T09:27:49.656480+0000","last_became_peered":"2023-09-23T09:27:49.656480+0000","last_unstale":"2023-09-23T09:27:53.689622+0000","last_undegraded":"2023-09-23T09:27:53.689622+0000","last_fullsized":"2023-09-23T09:27:53.689622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:35:46.948703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734135+0000","last_change":"2023-09-23T09:27:50.271805+0000","last_active":"2023-09-23T09:27:51.734135+0000","last_peered":"2023-09-23T09:27:51.734135+0000","last_clean":"2023-09-23T09:27:51.734135+0000","last_became_active":"2023-09-23T09:27:50.271463+0000","last_became_peered":"2023-09-23T09:27:50.271463+0000","last_unstale":"2023-09-23T09:27:51.734135+0000","last_undegraded":"2023-09-23T09:27:51.734135+0000","last_fullsized":"2023-09-23T09:27:51.734135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:00:03.608504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264108+0000","last_change":"2023-09-23T09:27:49.656988+0000","last_active":"2023-09-23T09:27:51.264108+0000","last_peered":"2023-09-23T09:27:51.264108+0000","last_clean":"2023-09-23T09:27:51.264108+0000","last_became_active":"2023-09-23T09:27:49.656655+0000","last_became_peered":"2023-09-23T09:27:49.656655+0000","last_unstale":"2023-09-23T09:27:51.264108+0000","last_undegraded":"2023-09-23T09:27:51.264108+0000","last_fullsized":"2023-09-23T09:27:51.264108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:07:33.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734679+0000","last_change":"2023-09-23T09:27:50.275021+0000","last_active":"2023-09-23T09:27:51.734679+0000","last_peered":"2023-09-23T09:27:51.734679+0000","last_clean":"2023-09-23T09:27:51.734679+0000","last_became_active":"2023-09-23T09:27:50.274551+0000","last_became_peered":"2023-09-23T09:27:50.274551+0000","last_unstale":"2023-09-23T09:27:51.734679+0000","last_undegraded":"2023-09-23T09:27:51.734679+0000","last_fullsized":"2023-09-23T09:27:51.734679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:50:45.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690216+0000","last_change":"2023-09-23T09:27:49.659603+0000","last_active":"2023-09-23T09:27:53.690216+0000","last_peered":"2023-09-23T09:27:53.690216+0000","last_clean":"2023-09-23T09:27:53.690216+0000","last_became_active":"2023-09-23T09:27:49.659293+0000","last_became_peered":"2023-09-23T09:27:49.659293+0000","last_unstale":"2023-09-23T09:27:53.690216+0000","last_undegraded":"2023-09-23T09:27:53.690216+0000","last_fullsized":"2023-09-23T09:27:53.690216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:31:25.165888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734478+0000","last_change":"2023-09-23T09:27:50.275069+0000","last_active":"2023-09-23T09:27:51.734478+0000","last_peered":"2023-09-23T09:27:51.734478+0000","last_clean":"2023-09-23T09:27:51.734478+0000","last_became_active":"2023-09-23T09:27:50.274731+0000","last_became_peered":"2023-09-23T09:27:50.274731+0000","last_unstale":"2023-09-23T09:27:51.734478+0000","last_undegraded":"2023-09-23T09:27:51.734478+0000","last_fullsized":"2023-09-23T09:27:51.734478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:45:23.794774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.703349+0000","last_change":"2023-09-23T09:27:49.659914+0000","last_active":"2023-09-23T09:27:54.703349+0000","last_peered":"2023-09-23T09:27:54.703349+0000","last_clean":"2023-09-23T09:27:54.703349+0000","last_became_active":"2023-09-23T09:27:49.659604+0000","last_became_peered":"2023-09-23T09:27:49.659604+0000","last_unstale":"2023-09-23T09:27:54.703349+0000","last_undegraded":"2023-09-23T09:27:54.703349+0000","last_fullsized":"2023-09-23T09:27:54.703349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:20.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734661+0000","last_change":"2023-09-23T09:27:50.271309+0000","last_active":"2023-09-23T09:27:51.734661+0000","last_peered":"2023-09-23T09:27:51.734661+0000","last_clean":"2023-09-23T09:27:51.734661+0000","last_became_active":"2023-09-23T09:27:50.270951+0000","last_became_peered":"2023-09-23T09:27:50.270951+0000","last_unstale":"2023-09-23T09:27:51.734661+0000","last_undegraded":"2023-09-23T09:27:51.734661+0000","last_fullsized":"2023-09-23T09:27:51.734661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:18.811428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.664667+0000","last_change":"2023-09-23T09:27:49.663849+0000","last_active":"2023-09-23T09:27:50.664667+0000","last_peered":"2023-09-23T09:27:50.664667+0000","last_clean":"2023-09-23T09:27:50.664667+0000","last_became_active":"2023-09-23T09:27:49.663499+0000","last_became_peered":"2023-09-23T09:27:49.663499+0000","last_unstale":"2023-09-23T09:27:50.664667+0000","last_undegraded":"2023-09-23T09:27:50.664667+0000","last_fullsized":"2023-09-23T09:27:50.664667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:31:08.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734512+0000","last_change":"2023-09-23T09:27:50.275681+0000","last_active":"2023-09-23T09:27:51.734512+0000","last_peered":"2023-09-23T09:27:51.734512+0000","last_clean":"2023-09-23T09:27:51.734512+0000","last_became_active":"2023-09-23T09:27:50.275370+0000","last_became_peered":"2023-09-23T09:27:50.275370+0000","last_unstale":"2023-09-23T09:27:51.734512+0000","last_undegraded":"2023-09-23T09:27:51.734512+0000","last_fullsized":"2023-09-23T09:27:51.734512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:53:01.710377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.665421+0000","last_change":"2023-09-23T09:27:49.652127+0000","last_active":"2023-09-23T09:27:50.665421+0000","last_peered":"2023-09-23T09:27:50.665421+0000","last_clean":"2023-09-23T09:27:50.665421+0000","last_became_active":"2023-09-23T09:27:49.651792+0000","last_became_peered":"2023-09-23T09:27:49.651792+0000","last_unstale":"2023-09-23T09:27:50.665421+0000","last_undegraded":"2023-09-23T09:27:50.665421+0000","last_fullsized":"2023-09-23T09:27:50.665421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:23:39.299203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734734+0000","last_change":"2023-09-23T09:27:50.271063+0000","last_active":"2023-09-23T09:27:51.734734+0000","last_peered":"2023-09-23T09:27:51.734734+0000","last_clean":"2023-09-23T09:27:51.734734+0000","last_became_active":"2023-09-23T09:27:50.270487+0000","last_became_peered":"2023-09-23T09:27:50.270487+0000","last_unstale":"2023-09-23T09:27:51.734734+0000","last_undegraded":"2023-09-23T09:27:51.734734+0000","last_fullsized":"2023-09-23T09:27:51.734734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:11:05.621855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734264+0000","last_change":"2023-09-23T09:27:50.275900+0000","last_active":"2023-09-23T09:27:51.734264+0000","last_peered":"2023-09-23T09:27:51.734264+0000","last_clean":"2023-09-23T09:27:51.734264+0000","last_became_active":"2023-09-23T09:27:50.275565+0000","last_became_peered":"2023-09-23T09:27:50.275565+0000","last_unstale":"2023-09-23T09:27:51.734264+0000","last_undegraded":"2023-09-23T09:27:51.734264+0000","last_fullsized":"2023-09-23T09:27:51.734264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:57:37.004604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264041+0000","last_change":"2023-09-23T09:27:50.275911+0000","last_active":"2023-09-23T09:27:51.264041+0000","last_peered":"2023-09-23T09:27:51.264041+0000","last_clean":"2023-09-23T09:27:51.264041+0000","last_became_active":"2023-09-23T09:27:50.275626+0000","last_became_peered":"2023-09-23T09:27:50.275626+0000","last_unstale":"2023-09-23T09:27:51.264041+0000","last_undegraded":"2023-09-23T09:27:51.264041+0000","last_fullsized":"2023-09-23T09:27:51.264041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:46.790012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264037+0000","last_change":"2023-09-23T09:27:50.269483+0000","last_active":"2023-09-23T09:27:51.264037+0000","last_peered":"2023-09-23T09:27:51.264037+0000","last_clean":"2023-09-23T09:27:51.264037+0000","last_became_active":"2023-09-23T09:27:50.269199+0000","last_became_peered":"2023-09-23T09:27:50.269199+0000","last_unstale":"2023-09-23T09:27:51.264037+0000","last_undegraded":"2023-09-23T09:27:51.264037+0000","last_fullsized":"2023-09-23T09:27:51.264037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:58:34.304439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733479+0000","last_change":"2023-09-23T09:27:50.267917+0000","last_active":"2023-09-23T09:27:51.733479+0000","last_peered":"2023-09-23T09:27:51.733479+0000","last_clean":"2023-09-23T09:27:51.733479+0000","last_became_active":"2023-09-23T09:27:50.267036+0000","last_became_peered":"2023-09-23T09:27:50.267036+0000","last_unstale":"2023-09-23T09:27:51.733479+0000","last_undegraded":"2023-09-23T09:27:51.733479+0000","last_fullsized":"2023-09-23T09:27:51.733479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:42.866260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735091+0000","last_change":"2023-09-23T09:27:50.269934+0000","last_active":"2023-09-23T09:27:51.735091+0000","last_peered":"2023-09-23T09:27:51.735091+0000","last_clean":"2023-09-23T09:27:51.735091+0000","last_became_active":"2023-09-23T09:27:50.269060+0000","last_became_peered":"2023-09-23T09:27:50.269060+0000","last_unstale":"2023-09-23T09:27:51.735091+0000","last_undegraded":"2023-09-23T09:27:51.735091+0000","last_fullsized":"2023-09-23T09:27:51.735091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:47:31.191479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733829+0000","last_change":"2023-09-23T09:27:50.277885+0000","last_active":"2023-09-23T09:27:51.733829+0000","last_peered":"2023-09-23T09:27:51.733829+0000","last_clean":"2023-09-23T09:27:51.733829+0000","last_became_active":"2023-09-23T09:27:50.277576+0000","last_became_peered":"2023-09-23T09:27:50.277576+0000","last_unstale":"2023-09-23T09:27:51.733829+0000","last_undegraded":"2023-09-23T09:27:51.733829+0000","last_fullsized":"2023-09-23T09:27:51.733829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:03:43.338185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264326+0000","last_change":"2023-09-23T09:27:50.278225+0000","last_active":"2023-09-23T09:27:51.264326+0000","last_peered":"2023-09-23T09:27:51.264326+0000","last_clean":"2023-09-23T09:27:51.264326+0000","last_became_active":"2023-09-23T09:27:50.277891+0000","last_became_peered":"2023-09-23T09:27:50.277891+0000","last_unstale":"2023-09-23T09:27:51.264326+0000","last_undegraded":"2023-09-23T09:27:51.264326+0000","last_fullsized":"2023-09-23T09:27:51.264326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:27:08.961439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734854+0000","last_change":"2023-09-23T09:27:50.273445+0000","last_active":"2023-09-23T09:27:51.734854+0000","last_peered":"2023-09-23T09:27:51.734854+0000","last_clean":"2023-09-23T09:27:51.734854+0000","last_became_active":"2023-09-23T09:27:50.273124+0000","last_became_peered":"2023-09-23T09:27:50.273124+0000","last_unstale":"2023-09-23T09:27:51.734854+0000","last_undegraded":"2023-09-23T09:27:51.734854+0000","last_fullsized":"2023-09-23T09:27:51.734854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:46.030212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734647+0000","last_change":"2023-09-23T09:27:50.276164+0000","last_active":"2023-09-23T09:27:51.734647+0000","last_peered":"2023-09-23T09:27:51.734647+0000","last_clean":"2023-09-23T09:27:51.734647+0000","last_became_active":"2023-09-23T09:27:50.275860+0000","last_became_peered":"2023-09-23T09:27:50.275860+0000","last_unstale":"2023-09-23T09:27:51.734647+0000","last_undegraded":"2023-09-23T09:27:51.734647+0000","last_fullsized":"2023-09-23T09:27:51.734647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:53:50.521872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734518+0000","last_change":"2023-09-23T09:27:50.275221+0000","last_active":"2023-09-23T09:27:51.734518+0000","last_peered":"2023-09-23T09:27:51.734518+0000","last_clean":"2023-09-23T09:27:51.734518+0000","last_became_active":"2023-09-23T09:27:50.274920+0000","last_became_peered":"2023-09-23T09:27:50.274920+0000","last_unstale":"2023-09-23T09:27:51.734518+0000","last_undegraded":"2023-09-23T09:27:51.734518+0000","last_fullsized":"2023-09-23T09:27:51.734518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:45:35.023433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259534+0000","last_change":"2023-09-23T09:27:50.277776+0000","last_active":"2023-09-23T09:27:51.259534+0000","last_peered":"2023-09-23T09:27:51.259534+0000","last_clean":"2023-09-23T09:27:51.259534+0000","last_became_active":"2023-09-23T09:27:50.277291+0000","last_became_peered":"2023-09-23T09:27:50.277291+0000","last_unstale":"2023-09-23T09:27:51.259534+0000","last_undegraded":"2023-09-23T09:27:51.259534+0000","last_fullsized":"2023-09-23T09:27:51.259534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:23:46.984450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734180+0000","last_change":"2023-09-23T09:27:50.271219+0000","last_active":"2023-09-23T09:27:51.734180+0000","last_peered":"2023-09-23T09:27:51.734180+0000","last_clean":"2023-09-23T09:27:51.734180+0000","last_became_active":"2023-09-23T09:27:50.270747+0000","last_became_peered":"2023-09-23T09:27:50.270747+0000","last_unstale":"2023-09-23T09:27:51.734180+0000","last_undegraded":"2023-09-23T09:27:51.734180+0000","last_fullsized":"2023-09-23T09:27:51.734180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:19.237260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733913+0000","last_change":"2023-09-23T09:27:50.263794+0000","last_active":"2023-09-23T09:27:51.733913+0000","last_peered":"2023-09-23T09:27:51.733913+0000","last_clean":"2023-09-23T09:27:51.733913+0000","last_became_active":"2023-09-23T09:27:50.263459+0000","last_became_peered":"2023-09-23T09:27:50.263459+0000","last_unstale":"2023-09-23T09:27:51.733913+0000","last_undegraded":"2023-09-23T09:27:51.733913+0000","last_fullsized":"2023-09-23T09:27:51.733913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:05:37.618959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250071,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5522,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":115,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":32315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":426682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":1019618,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57599999999999996}]},{"osd":9,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":427500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.55400000000000005}]},{"osd":2,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.28599999999999998}]},{"osd":4,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.36899999999999999}]},{"osd":7,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.47799999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.629}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68200000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":427420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72399999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.78000000000000003}]},{"osd":3,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.749}]},{"osd":4,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.90400000000000003}]},{"osd":6,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.871},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.80400000000000005}]},{"osd":7,"last update":"Sat Sep 23 09:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.006}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.89100000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":1017981,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41199999999999998,"15min":0.40400000000000003},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.55400000000000005}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59299999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.311,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.745},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80900000000000005}]},{"osd":8,"last update":"Sat Sep 23 09:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69699999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":426682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.41199999999999998},"min":{"1min":0.33300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.30099999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.499,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.51200000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63200000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.57099999999999995},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98399999999999999,"5min":1.006,"15min":1.006},"last":0.65000000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.36299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.79200000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.747,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":1.077,"15min":1.077},"last":0.46600000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.61299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.56699999999999995,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73999999999999999}]},{"osd":7,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.61899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.77100000000000002}]},{"osd":8,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75800000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":428294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.37,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.74299999999999999}]},{"osd":2,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83199999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.29499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39900000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.42199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91900000000000004}]},{"osd":6,"last update":"Sat Sep 23 09:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89400000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81499999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.624},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85799999999999998}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72999999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":426772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 09:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.55100000000000005,"15min":0.51300000000000001},"min":{"1min":0.46400000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.31},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.439,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.32800000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":1.054,"15min":1.054},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.53100000000000003,"15min":0.504},"min":{"1min":0.376,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Sep 23 09:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.26400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81100000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.40200000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.57099999999999995,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.442}]},{"osd":5,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.52600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.308,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.55000000000000004,"15min":0.52000000000000002},"min":{"1min":0.40200000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41799999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.718,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.718},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.64100000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.47899999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.57199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":429644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.505,"15min":0.50700000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.754,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.5,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.97199999999999998,"5min":1.655,"15min":1.655},"last":0.78000000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.81299999999999994}]},{"osd":3,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82499999999999996,"5min":0.871,"15min":0.871},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.44,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71399999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73999999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.627},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64300000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.438,"15min":0.434},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.497},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.45700000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76000000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.40500000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.014,"15min":1.014},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.79500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.875}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.92600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":426635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.36499999999999999,"15min":0.34399999999999997},"min":{"1min":0.29299999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.35799999999999998},"min":{"1min":0.26600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38300000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.438,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60699999999999998,"15min":0.58599999999999997},"min":{"1min":0.48499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73199999999999998}]},{"osd":4,"last update":"Sat Sep 23 09:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.499,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.505},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45200000000000001}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.48799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.71099999999999997,"15min":0.69999999999999996},"min":{"1min":0.628,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.629}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.61499999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":4,"up_from":33,"seq":141733920810,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":427716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.998,"15min":0.998},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.38},"min":{"1min":0.33900000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40200000000000002}]},{"osd":2,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73699999999999999}]},{"osd":3,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.495,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45600000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83999999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.754},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67700000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.58599999999999997,"15min":0.56599999999999995},"min":{"1min":0.44400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.624},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82199999999999995}]},{"osd":8,"last update":"Sat Sep 23 09:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.83899999999999997}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.80300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":1017232,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34899999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65600000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71599999999999997,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.377,"15min":0.36099999999999999},"min":{"1min":0.311,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.437},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.46100000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53400000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.45400000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.375,"15min":0.375},"max":{"1min":0.80500000000000005,"5min":1.538,"15min":1.538},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.45400000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71399999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.54800000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.755}]},{"osd":8,"last update":"Sat Sep 23 09:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56299999999999994}]},{"osd":9,"last update":"Sat Sep 23 09:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84999999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T09:27:58.447 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T09:27:58.447 INFO:tasks.cephadm:Setting up client nodes... 2023-09-23T09:27:58.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-23T09:27:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:27:59.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:58 smithi098 ceph-mon[102386]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:27:59.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:58 smithi045 ceph-mon[98627]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:27:59.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:58 smithi123 ceph-mon[103577]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:27:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:59 smithi045 ceph-mon[98627]: pgmap v255: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2023-09-23T09:27:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:27:59 smithi045 ceph-mon[98627]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:00.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:59 smithi098 ceph-mon[102386]: pgmap v255: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2023-09-23T09:28:00.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:27:59 smithi098 ceph-mon[102386]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:00.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:59 smithi123 ceph-mon[103577]: pgmap v255: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2023-09-23T09:28:00.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:27:59 smithi123 ceph-mon[103577]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:00.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 -- 172.21.15.45:0/1295886540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59201042a0 msgr2=0x7f5920106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:00.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 --2- 172.21.15.45:0/1295886540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59201042a0 0x7f5920106690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5914009990 tx=0x7f5914009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 -- 172.21.15.45:0/1295886540 shutdown_connections 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 --2- 172.21.15.45:0/1295886540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5920106bd0 0x7f5920109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 --2- 172.21.15.45:0/1295886540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5920101970 0x7f5920103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 --2- 172.21.15.45:0/1295886540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59201042a0 0x7f5920106690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 -- 172.21.15.45:0/1295886540 >> 172.21.15.45:0/1295886540 conn(0x7f59200fb360 msgr2=0x7f59200fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 -- 172.21.15.45:0/1295886540 shutdown_connections 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.271+0000 7f59263ab700 1 -- 172.21.15.45:0/1295886540 wait complete. 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 Processor -- start 2023-09-23T09:28:00.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 -- start start 2023-09-23T09:28:00.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5920101970 0x7f5920196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:00.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59201042a0 0x7f5920197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5920106bd0 0x7f592019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592019e6d0 con 0x7f5920106bd0 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592019e810 con 0x7f5920101970 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f59263ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592019e9a0 con 0x7f59201042a0 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f5924948700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5920106bd0 0x7f592019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f591ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5920101970 0x7f5920196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f5924948700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5920106bd0 0x7f592019e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59016/0 (socket says 172.21.15.45:59016) 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f5924948700 1 -- 172.21.15.45:0/2305744980 learned_addr learned my addr 172.21.15.45:0/2305744980 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.272+0000 7f591f7fe700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59201042a0 0x7f5920197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:00.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f5924948700 1 -- 172.21.15.45:0/2305744980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59201042a0 msgr2=0x7f5920197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:00.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f5924948700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59201042a0 0x7f5920197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f5924948700 1 -- 172.21.15.45:0/2305744980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5920101970 msgr2=0x7f5920196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:00.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f5924948700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5920101970 0x7f5920196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f5924948700 1 -- 172.21.15.45:0/2305744980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140095f0 con 0x7f5920106bd0 2023-09-23T09:28:00.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f5924948700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5920106bd0 0x7f592019e100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f591800ba10 tx=0x7f591800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:00.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f591d7fa700 1 -- 172.21.15.45:0/2305744980 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591800d8e0 con 0x7f5920106bd0 2023-09-23T09:28:00.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592019ec80 con 0x7f5920106bd0 2023-09-23T09:28:00.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f591d7fa700 1 -- 172.21.15.45:0/2305744980 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5918009880 con 0x7f5920106bd0 2023-09-23T09:28:00.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f591d7fa700 1 -- 172.21.15.45:0/2305744980 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59180143d0 con 0x7f5920106bd0 2023-09-23T09:28:00.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.273+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592019f130 con 0x7f5920106bd0 2023-09-23T09:28:00.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.274+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59200fcee0 con 0x7f5920106bd0 2023-09-23T09:28:00.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.275+0000 7f591d7fa700 1 -- 172.21.15.45:0/2305744980 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f5918014550 con 0x7f5920106bd0 2023-09-23T09:28:00.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.276+0000 7f591d7fa700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f59080a1630 0x7f59080a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:00.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.276+0000 7f591ffff700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f59080a1630 0x7f59080a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:00.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.276+0000 7f591d7fa700 1 -- 172.21.15.45:0/2305744980 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5918011070 con 0x7f5920106bd0 2023-09-23T09:28:00.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.277+0000 7f591ffff700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f59080a1630 0x7f59080a3af0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f591000ac50 tx=0x7f591000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.281+0000 7f591d7fa700 1 -- 172.21.15.45:0/2305744980 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f591804eec0 con 0x7f5920106bd0 2023-09-23T09:28:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.569+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f592010c770 con 0x7f5920106bd0 2023-09-23T09:28:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.571+0000 7f591d7fa700 1 -- 172.21.15.45:0/2305744980 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5918095e10 con 0x7f5920106bd0 2023-09-23T09:28:00.573 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-09-23T09:28:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCgrw5l+2UCIhAATkQaqqGPdS1yf2DblLk4yQ== 2023-09-23T09:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.574+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f59080a1630 msgr2=0x7f59080a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.574+0000 7f59263ab700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f59080a1630 0x7f59080a3af0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f591000ac50 tx=0x7f591000a380 comp rx=0 tx=0).stop 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.574+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5920106bd0 msgr2=0x7f592019e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.574+0000 7f59263ab700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5920106bd0 0x7f592019e100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f591800ba10 tx=0x7f591800bd20 comp rx=0 tx=0).stop 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 shutdown_connections 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5920106bd0 0x7f592019e100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5920101970 0x7f5920196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f59080a1630 0x7f59080a3af0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 --2- 172.21.15.45:0/2305744980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59201042a0 0x7f5920197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 >> 172.21.15.45:0/2305744980 conn(0x7f59200fb360 msgr2=0x7f59201025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 shutdown_connections 2023-09-23T09:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:00.575+0000 7f59263ab700 1 -- 172.21.15.45:0/2305744980 wait complete. 2023-09-23T09:28:00.984 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:28:00.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-23T09:28:00.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-23T09:28:01.027 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-23T09:28:01.028 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-23T09:28:01.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph mgr dump --format=json 2023-09-23T09:28:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:01 smithi098 ceph-mon[102386]: pgmap v256: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2023-09-23T09:28:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:01 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2305744980' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-23T09:28:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:01 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2305744980' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-23T09:28:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:01 smithi045 ceph-mon[98627]: pgmap v256: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2023-09-23T09:28:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:01 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2305744980' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-23T09:28:01.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:01 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2305744980' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-23T09:28:01.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:01 smithi123 ceph-mon[103577]: pgmap v256: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2023-09-23T09:28:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:01 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2305744980' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-23T09:28:01.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:01 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2305744980' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-23T09:28:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.715+0000 7f4597293700 1 -- 172.21.15.45:0/658277467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590105e10 msgr2=0x7f45901061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 --2- 172.21.15.45:0/658277467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590105e10 0x7f45901061f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f458c009990 tx=0x7f458c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 -- 172.21.15.45:0/658277467 shutdown_connections 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 --2- 172.21.15.45:0/658277467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4590100720 0x7f459010cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 --2- 172.21.15.45:0/658277467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4590105e10 0x7f45901061f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 --2- 172.21.15.45:0/658277467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45900ffd60 0x7f45901001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 -- 172.21.15.45:0/658277467 >> 172.21.15.45:0/658277467 conn(0x7f4590076e20 msgr2=0x7f4590077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 -- 172.21.15.45:0/658277467 shutdown_connections 2023-09-23T09:28:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.716+0000 7f4597293700 1 -- 172.21.15.45:0/658277467 wait complete. 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 Processor -- start 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 -- start start 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45900ffd60 0x7f4590195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4590100720 0x7f45901962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4590105e10 0x7f459019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019ad40 con 0x7f45900ffd60 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019aeb0 con 0x7f4590100720 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4597293700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019b070 con 0x7f4590105e10 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45900ffd60 0x7f4590195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45900ffd60 0x7f4590195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59026/0 (socket says 172.21.15.45:59026) 2023-09-23T09:28:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 -- 172.21.15.45:0/3749114434 learned_addr learned my addr 172.21.15.45:0/3749114434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459482e700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4590100720 0x7f45901962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 -- 172.21.15.45:0/3749114434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4590105e10 msgr2=0x7f459019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4595830700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4590105e10 0x7f459019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4590105e10 0x7f459019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 -- 172.21.15.45:0/3749114434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4590100720 msgr2=0x7f45901962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4590100720 0x7f45901962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f459502f700 1 -- 172.21.15.45:0/3749114434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0095f0 con 0x7f45900ffd60 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.717+0000 7f4595830700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4590105e10 0x7f459019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.718+0000 7f459502f700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45900ffd60 0x7f4590195d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f458c000c00 tx=0x7f458c01a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.718+0000 7f45867fc700 1 -- 172.21.15.45:0/3749114434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c01f070 con 0x7f45900ffd60 2023-09-23T09:28:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.718+0000 7f45867fc700 1 -- 172.21.15.45:0/3749114434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458c01ac80 con 0x7f45900ffd60 2023-09-23T09:28:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.718+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459019b2f0 con 0x7f45900ffd60 2023-09-23T09:28:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.718+0000 7f45867fc700 1 -- 172.21.15.45:0/3749114434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c025710 con 0x7f45900ffd60 2023-09-23T09:28:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.718+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590068ab0 con 0x7f45900ffd60 2023-09-23T09:28:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.719+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459004eb20 con 0x7f45900ffd60 2023-09-23T09:28:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.721+0000 7f45867fc700 1 -- 172.21.15.45:0/3749114434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f458c023070 con 0x7f45900ffd60 2023-09-23T09:28:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.722+0000 7f45867fc700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a16b0 0x7f457c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.722+0000 7f459482e700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a16b0 0x7f457c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.722+0000 7f45867fc700 1 -- 172.21.15.45:0/3749114434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f458c0d0ad0 con 0x7f45900ffd60 2023-09-23T09:28:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.722+0000 7f459482e700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a16b0 0x7f457c0a3b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45901973a0 tx=0x7f4580005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.725+0000 7f45867fc700 1 -- 172.21.15.45:0/3749114434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f458c09a0a0 con 0x7f45900ffd60 2023-09-23T09:28:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.985+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f459010fc70 con 0x7f45900ffd60 2023-09-23T09:28:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.990+0000 7f45867fc700 1 -- 172.21.15.45:0/3749114434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260591 (secure 0 0 0) 0x7f458c09d6c0 con 0x7f45900ffd60 2023-09-23T09:28:02.995 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.992+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a16b0 msgr2=0x7f457c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.992+0000 7f4597293700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a16b0 0x7f457c0a3b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45901973a0 tx=0x7f4580005cf0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.992+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45900ffd60 msgr2=0x7f4590195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.992+0000 7f4597293700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45900ffd60 0x7f4590195d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f458c000c00 tx=0x7f458c01a9e0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 shutdown_connections 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f457c0a16b0 0x7f457c0a3b70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4590105e10 0x7f459019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45900ffd60 0x7f4590195d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 --2- 172.21.15.45:0/3749114434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4590100720 0x7f45901962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 >> 172.21.15.45:0/3749114434 conn(0x7f4590076e20 msgr2=0x7f45900fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 shutdown_connections 2023-09-23T09:28:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:02.993+0000 7f4597293700 1 -- 172.21.15.45:0/3749114434 wait complete. 2023-09-23T09:28:03.379 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":944196506},{"type":"v1","addr":"172.21.15.45:6801","nonce":944196506}]},"active_addr":"172.21.15.45:6801/944196506","active_change":"2023-09-23T09:21:54.248259+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2324483632}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":65945615}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":230875070}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1970790569}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3269510756}]}]} 2023-09-23T09:28:03.386 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-23T09:28:03.386 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-23T09:28:03.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd dump --format=json 2023-09-23T09:28:03.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:03 smithi098 ceph-mon[102386]: pgmap v257: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-23T09:28:03.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:03 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3749114434' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-23T09:28:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:03 smithi045 ceph-mon[98627]: pgmap v257: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-23T09:28:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:03 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3749114434' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-23T09:28:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:03 smithi123 ceph-mon[103577]: pgmap v257: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-23T09:28:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:03 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3749114434' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-23T09:28:03.950 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.118+0000 7f02b1ecd700 1 -- 172.21.15.45:0/86746766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac106fb0 msgr2=0x7f02ac107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.118+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/86746766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac106fb0 0x7f02ac107390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f029c009990 tx=0x7f029c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.118+0000 7f02b1ecd700 1 -- 172.21.15.45:0/86746766 shutdown_connections 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.118+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/86746766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac069300 0x7f02ac10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.118+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/86746766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac106fb0 0x7f02ac107390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.118+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/86746766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02ac068940 0x7f02ac068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.118+0000 7f02b1ecd700 1 -- 172.21.15.45:0/86746766 >> 172.21.15.45:0/86746766 conn(0x7f02ac076e20 msgr2=0x7f02ac077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.119+0000 7f02b1ecd700 1 -- 172.21.15.45:0/86746766 shutdown_connections 2023-09-23T09:28:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.119+0000 7f02b1ecd700 1 -- 172.21.15.45:0/86746766 wait complete. 2023-09-23T09:28:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.119+0000 7f02b1ecd700 1 Processor -- start 2023-09-23T09:28:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02b1ecd700 1 -- start start 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02b1ecd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02ac068940 0x7f02ac109f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02b1ecd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac069300 0x7f02ac10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02b1ecd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac106fb0 0x7f02ac10a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02b1ecd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1a9720 con 0x7f02ac069300 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02b1ecd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1a9890 con 0x7f02ac068940 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02b1ecd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1a9a50 con 0x7f02ac106fb0 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac069300 0x7f02ac10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac069300 0x7f02ac10a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59040/0 (socket says 172.21.15.45:59040) 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 -- 172.21.15.45:0/536737707 learned_addr learned my addr 172.21.15.45:0/536737707 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02abfff700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac106fb0 0x7f02ac10a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02ac068940 0x7f02ac109f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 -- 172.21.15.45:0/536737707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac106fb0 msgr2=0x7f02ac10a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac106fb0 0x7f02ac10a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 -- 172.21.15.45:0/536737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02ac068940 msgr2=0x7f02ac109f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02ac068940 0x7f02ac109f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 -- 172.21.15.45:0/536737707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0095f0 con 0x7f02ac069300 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02ac068940 0x7f02ac109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.120+0000 7f02aaffd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac069300 0x7f02ac10a4b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f029400c920 tx=0x7f029400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.121+0000 7f02a8ff9700 1 -- 172.21.15.45:0/536737707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294007b00 con 0x7f02ac069300 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.121+0000 7f02a8ff9700 1 -- 172.21.15.45:0/536737707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f029400ce70 con 0x7f02ac069300 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.121+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac1a9d30 con 0x7f02ac069300 2023-09-23T09:28:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.121+0000 7f02a8ff9700 1 -- 172.21.15.45:0/536737707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940054e0 con 0x7f02ac069300 2023-09-23T09:28:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.121+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac1aa220 con 0x7f02ac069300 2023-09-23T09:28:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.122+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac04eb20 con 0x7f02ac069300 2023-09-23T09:28:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.123+0000 7f02a8ff9700 1 -- 172.21.15.45:0/536737707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f0294004750 con 0x7f02ac069300 2023-09-23T09:28:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.123+0000 7f02a8ff9700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02980a16b0 0x7f02980a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.123+0000 7f02a8ff9700 1 -- 172.21.15.45:0/536737707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f02940cca00 con 0x7f02ac069300 2023-09-23T09:28:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.123+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02980a16b0 0x7f02980a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.124+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02980a16b0 0x7f02980a3b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f029c00b5c0 tx=0x7f029c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.128+0000 7f02a8ff9700 1 -- 172.21.15.45:0/536737707 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0294096040 con 0x7f02ac069300 2023-09-23T09:28:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.330+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02ac101130 con 0x7f02ac069300 2023-09-23T09:28:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.332+0000 7f02a8ff9700 1 -- 172.21.15.45:0/536737707 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7f029401d090 con 0x7f02ac069300 2023-09-23T09:28:05.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:05.334 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:51.253687+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:28:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02980a16b0 msgr2=0x7f02980a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02980a16b0 0x7f02980a3b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f029c00b5c0 tx=0x7f029c005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac069300 msgr2=0x7f02ac10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac069300 0x7f02ac10a4b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f029400c920 tx=0x7f029400cc30 comp rx=0 tx=0).stop 2023-09-23T09:28:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 shutdown_connections 2023-09-23T09:28:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02980a16b0 0x7f02980a3b70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac106fb0 0x7f02ac10a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02ac068940 0x7f02ac109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 --2- 172.21.15.45:0/536737707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac069300 0x7f02ac10a4b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 >> 172.21.15.45:0/536737707 conn(0x7f02ac076e20 msgr2=0x7f02ac104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 shutdown_connections 2023-09-23T09:28:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:05.334+0000 7f02b1ecd700 1 -- 172.21.15.45:0/536737707 wait complete. 2023-09-23T09:28:05.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:05 smithi045 ceph-mon[98627]: pgmap v258: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2023-09-23T09:28:05.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:05 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/536737707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:05.697 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-23T09:28:05.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd dump --format=json 2023-09-23T09:28:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:05 smithi098 ceph-mon[102386]: pgmap v258: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2023-09-23T09:28:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:05 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/536737707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:05 smithi123 ceph-mon[103577]: pgmap v258: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2023-09-23T09:28:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:05 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/536737707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:07.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 -- 172.21.15.45:0/1882424745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13300ffd60 msgr2=0x7f13301001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:07.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 --2- 172.21.15.45:0/1882424745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13300ffd60 0x7f13301001e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1324009990 tx=0x7f1324009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 -- 172.21.15.45:0/1882424745 shutdown_connections 2023-09-23T09:28:07.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 --2- 172.21.15.45:0/1882424745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1330100720 0x7f133010cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 --2- 172.21.15.45:0/1882424745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330105e10 0x7f13301061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 --2- 172.21.15.45:0/1882424745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13300ffd60 0x7f13301001e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 -- 172.21.15.45:0/1882424745 >> 172.21.15.45:0/1882424745 conn(0x7f1330076e20 msgr2=0x7f1330077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:07.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 -- 172.21.15.45:0/1882424745 shutdown_connections 2023-09-23T09:28:07.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.681+0000 7f1335058700 1 -- 172.21.15.45:0/1882424745 wait complete. 2023-09-23T09:28:07.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 Processor -- start 2023-09-23T09:28:07.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 -- start start 2023-09-23T09:28:07.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13300ffd60 0x7f13301991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:07.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1330100720 0x7f1330199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:07.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330105e10 0x7f133019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133019e1b0 con 0x7f1330100720 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133019e320 con 0x7f13300ffd60 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f1335058700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133019e4e0 con 0x7f1330105e10 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1330100720 0x7f1330199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1330100720 0x7f1330199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59052/0 (socket says 172.21.15.45:59052) 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 -- 172.21.15.45:0/1330470129 learned_addr learned my addr 172.21.15.45:0/1330470129 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132f59e700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330105e10 0x7f133019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132ed9d700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13300ffd60 0x7f13301991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 -- 172.21.15.45:0/1330470129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330105e10 msgr2=0x7f133019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330105e10 0x7f133019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 -- 172.21.15.45:0/1330470129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13300ffd60 msgr2=0x7f13301991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13300ffd60 0x7f13301991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.682+0000 7f132e59c700 1 -- 172.21.15.45:0/1330470129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240095f0 con 0x7f1330100720 2023-09-23T09:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.683+0000 7f132e59c700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1330100720 0x7f1330199730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1324006010 tx=0x7f132400f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.683+0000 7f1317fff700 1 -- 172.21.15.45:0/1330470129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132401d070 con 0x7f1330100720 2023-09-23T09:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.683+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133019e760 con 0x7f1330100720 2023-09-23T09:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.683+0000 7f1317fff700 1 -- 172.21.15.45:0/1330470129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1324022470 con 0x7f1330100720 2023-09-23T09:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.683+0000 7f1317fff700 1 -- 172.21.15.45:0/1330470129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324017650 con 0x7f1330100720 2023-09-23T09:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.683+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13301a6020 con 0x7f1330100720 2023-09-23T09:28:07.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.684+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133004eb20 con 0x7f1330100720 2023-09-23T09:28:07.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.685+0000 7f1317fff700 1 -- 172.21.15.45:0/1330470129 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f13240225e0 con 0x7f1330100720 2023-09-23T09:28:07.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.685+0000 7f1317fff700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13180a1630 0x7f13180a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:07.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.686+0000 7f1317fff700 1 -- 172.21.15.45:0/1330470129 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f13240ce3b0 con 0x7f1330100720 2023-09-23T09:28:07.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.686+0000 7f132ed9d700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13180a1630 0x7f13180a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:07.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.686+0000 7f132ed9d700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13180a1630 0x7f13180a3af0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1320009a10 tx=0x7f1320009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:07.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.690+0000 7f1317fff700 1 -- 172.21.15.45:0/1330470129 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1324097ba0 con 0x7f1330100720 2023-09-23T09:28:07.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:07 smithi098 ceph-mon[102386]: pgmap v259: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2023-09-23T09:28:07.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:07 smithi045 ceph-mon[98627]: pgmap v259: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2023-09-23T09:28:07.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:07 smithi123 ceph-mon[103577]: pgmap v259: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2023-09-23T09:28:07.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.892+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1330110e70 con 0x7f1330100720 2023-09-23T09:28:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.894+0000 7f1317fff700 1 -- 172.21.15.45:0/1330470129 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7f1324027090 con 0x7f1330100720 2023-09-23T09:28:07.896 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:07.896 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:51.253687+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13180a1630 msgr2=0x7f13180a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13180a1630 0x7f13180a3af0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1320009a10 tx=0x7f1320009190 comp rx=0 tx=0).stop 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1330100720 msgr2=0x7f1330199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1330100720 0x7f1330199730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1324006010 tx=0x7f132400f700 comp rx=0 tx=0).stop 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 shutdown_connections 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13180a1630 0x7f13180a3af0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330105e10 0x7f133019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13300ffd60 0x7f13301991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 --2- 172.21.15.45:0/1330470129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1330100720 0x7f1330199730 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 >> 172.21.15.45:0/1330470129 conn(0x7f1330076e20 msgr2=0x7f133010aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 shutdown_connections 2023-09-23T09:28:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:07.896+0000 7f1335058700 1 -- 172.21.15.45:0/1330470129 wait complete. 2023-09-23T09:28:08.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-09-23T09:28:08.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-09-23T09:28:08.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-09-23T09:28:08.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-09-23T09:28:08.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-09-23T09:28:08.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-09-23T09:28:08.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-09-23T09:28:08.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-09-23T09:28:08.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-09-23T09:28:08.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-09-23T09:28:08.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-09-23T09:28:08.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-09-23T09:28:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:08 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1330470129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:08.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:08 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1330470129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:08.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:08 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1330470129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:09.775 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:09.775 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:09 smithi098 ceph-mon[102386]: pgmap v260: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2023-09-23T09:28:09.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:09 smithi045 ceph-mon[98627]: pgmap v260: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2023-09-23T09:28:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:09 smithi123 ceph-mon[103577]: pgmap v260: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2023-09-23T09:28:09.933 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:11.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:11 smithi098 ceph-mon[102386]: pgmap v261: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-09-23T09:28:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:11 smithi045 ceph-mon[98627]: pgmap v261: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-09-23T09:28:11.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:11 smithi123 ceph-mon[103577]: pgmap v261: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-09-23T09:28:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:12.365 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:12.365 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 -- 172.21.15.45:0/3625522396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 msgr2=0x7f0ac01001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 --2- 172.21.15.45:0/3625522396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 0x7f0ac01001e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009990 tx=0x7f0ab4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 -- 172.21.15.45:0/3625522396 shutdown_connections 2023-09-23T09:28:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 --2- 172.21.15.45:0/3625522396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac0100720 0x7f0ac010cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 --2- 172.21.15.45:0/3625522396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac0105e10 0x7f0ac01061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 --2- 172.21.15.45:0/3625522396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 0x7f0ac01001e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 -- 172.21.15.45:0/3625522396 >> 172.21.15.45:0/3625522396 conn(0x7f0ac0076e20 msgr2=0x7f0ac0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 -- 172.21.15.45:0/3625522396 shutdown_connections 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.560+0000 7f0ac67fe700 1 -- 172.21.15.45:0/3625522396 wait complete. 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.561+0000 7f0ac67fe700 1 Processor -- start 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.561+0000 7f0ac67fe700 1 -- start start 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.561+0000 7f0ac67fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 0x7f0ac0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.561+0000 7f0ac67fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac0100720 0x7f0ac0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.561+0000 7f0ac67fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac0105e10 0x7f0ac01a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.561+0000 7f0ac67fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0198bc0 con 0x7f0ac00ffd60 2023-09-23T09:28:12.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0ac67fe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac01a3300 con 0x7f0ac0100720 2023-09-23T09:28:12.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0ac67fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac01a34c0 con 0x7f0ac0105e10 2023-09-23T09:28:12.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 0x7f0ac0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:12.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 0x7f0ac0197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59062/0 (socket says 172.21.15.45:59062) 2023-09-23T09:28:12.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 -- 172.21.15.45:0/284471917 learned_addr learned my addr 172.21.15.45:0/284471917 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:12.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0ac4d9b700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac0105e10 0x7f0ac01a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:12.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac0100720 0x7f0ac0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 -- 172.21.15.45:0/284471917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac0105e10 msgr2=0x7f0ac01a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac0105e10 0x7f0ac01a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 -- 172.21.15.45:0/284471917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac0100720 msgr2=0x7f0ac0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac0100720 0x7f0ac0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 -- 172.21.15.45:0/284471917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40095f0 con 0x7f0ac00ffd60 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abffff700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 0x7f0ac0197d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b530 tx=0x7f0ab000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0011830 con 0x7f0ac00ffd60 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac01a37a0 con 0x7f0ac00ffd60 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ab0011e50 con 0x7f0ac00ffd60 2023-09-23T09:28:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000f2e0 con 0x7f0ac00ffd60 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.562+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac01a3cf0 con 0x7f0ac00ffd60 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.563+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ac004eb20 con 0x7f0ac00ffd60 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.564+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f0ab0010580 con 0x7f0ac00ffd60 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0abd7fa700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0aa80a1630 0x7f0aa80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0aa80a1630 0x7f0aa80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ab00cd080 con 0x7f0ac00ffd60 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0abd7fa700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] conn(0x7f0aa80aae10 0x7f0aa80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 --> [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0aa80ad8e0 con 0x7f0aa80aae10 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab00cd3f0 con 0x7f0ac00ffd60 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0ac4d9b700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] conn(0x7f0aa80aae10 0x7f0aa80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.565+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0aa80a1630 0x7f0aa80a3af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b5c0 tx=0x7f0ab4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.566+0000 7f0ac4d9b700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] conn(0x7f0aa80aae10 0x7f0aa80ad230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.566+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== osd.9 v2:172.21.15.45:6826/2888091166 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0aa80ad8e0 con 0x7f0aa80aae10 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.586+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 --> [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ac010ffe0 con 0x7f0aa80aae10 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.586+0000 7f0abd7fa700 1 -- 172.21.15.45:0/284471917 <== osd.9 v2:172.21.15.45:6826/2888091166 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0ab8007d10 con 0x7f0aa80aae10 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] conn(0x7f0aa80aae10 msgr2=0x7f0aa80ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] conn(0x7f0aa80aae10 0x7f0aa80ad230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0aa80a1630 msgr2=0x7f0aa80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0aa80a1630 0x7f0aa80a3af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b5c0 tx=0x7f0ab4005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 msgr2=0x7f0ac0197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 --2- 172.21.15.45:0/284471917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ffd60 0x7f0ac0197d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b530 tx=0x7f0ab000b8f0 comp rx=0 tx=0).stop 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac4d9b700 1 -- 172.21.15.45:0/284471917 reap_dead start 2023-09-23T09:28:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 shutdown_connections 2023-09-23T09:28:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 >> 172.21.15.45:0/284471917 conn(0x7f0ac0076e20 msgr2=0x7f0ac00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 shutdown_connections 2023-09-23T09:28:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:12.587+0000 7f0ac67fe700 1 -- 172.21.15.45:0/284471917 wait complete. 2023-09-23T09:28:13.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:13 smithi045 ceph-mon[98627]: pgmap v262: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2023-09-23T09:28:13.790 INFO:teuthology.orchestra.run.smithi045.stdout:283467841560 2023-09-23T09:28:13.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-09-23T09:28:13.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:13 smithi098 ceph-mon[102386]: pgmap v262: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2023-09-23T09:28:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:13 smithi123 ceph-mon[103577]: pgmap v262: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2023-09-23T09:28:14.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 -- 172.21.15.45:0/3742902711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b41010a0 msgr2=0x7f22b4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 --2- 172.21.15.45:0/3742902711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b41010a0 0x7f22b4101480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009990 tx=0x7f22a4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 -- 172.21.15.45:0/3742902711 shutdown_connections 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 --2- 172.21.15.45:0/3742902711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22b4106180 0x7f22b410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 --2- 172.21.15.45:0/3742902711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b41010a0 0x7f22b4101480 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 --2- 172.21.15.45:0/3742902711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22b4101a50 0x7f22b4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 -- 172.21.15.45:0/3742902711 >> 172.21.15.45:0/3742902711 conn(0x7f22b40fc910 msgr2=0x7f22b40fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 -- 172.21.15.45:0/3742902711 shutdown_connections 2023-09-23T09:28:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.413+0000 7f22ba657700 1 -- 172.21.15.45:0/3742902711 wait complete. 2023-09-23T09:28:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.414+0000 7f22ba657700 1 Processor -- start 2023-09-23T09:28:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.414+0000 7f22ba657700 1 -- start start 2023-09-23T09:28:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22ba657700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22b41010a0 0x7f22b4196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22ba657700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22b4101a50 0x7f22b41974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22ba657700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b4106180 0x7f22b419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22ba657700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b419bf60 con 0x7f22b4106180 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22ba657700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b419c0d0 con 0x7f22b41010a0 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22ba657700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b419c290 con 0x7f22b4101a50 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b4106180 0x7f22b419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b4106180 0x7f22b419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59096/0 (socket says 172.21.15.45:59096) 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 -- 172.21.15.45:0/1718272934 learned_addr learned my addr 172.21.15.45:0/1718272934 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b3fff700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22b41010a0 0x7f22b4196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b37fe700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22b4101a50 0x7f22b41974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 -- 172.21.15.45:0/1718272934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22b4101a50 msgr2=0x7f22b41974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22b4101a50 0x7f22b41974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 -- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22b41010a0 msgr2=0x7f22b4196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22b41010a0 0x7f22b4196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.415+0000 7f22b8bf4700 1 -- 172.21.15.45:0/1718272934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a40095f0 con 0x7f22b4106180 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.416+0000 7f22b8bf4700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b4106180 0x7f22b419b950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00ada0 tx=0x7f22ac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.416+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00f070 con 0x7f22b4106180 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.416+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ac007b30 con 0x7f22b4106180 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.416+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b419c570 con 0x7f22b4106180 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.416+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0053e0 con 0x7f22b4106180 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.416+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b41a3dd0 con 0x7f22b4106180 2023-09-23T09:28:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.417+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22b41110a0 con 0x7f22b4106180 2023-09-23T09:28:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.418+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f22ac00c980 con 0x7f22b4106180 2023-09-23T09:28:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.418+0000 7f22b17fa700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f229c0a1630 0x7f229c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b3fff700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f229c0a1630 0x7f229c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f22ac0cce50 con 0x7f22b4106180 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b17fa700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] conn(0x7f229c0aae10 0x7f229c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 --> [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f229c0ad8e0 con 0x7f229c0aae10 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22ac0cd2f0 con 0x7f22b4106180 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b3fff700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f229c0a1630 0x7f229c0a3af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b5c0 tx=0x7f22a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b37fe700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] conn(0x7f229c0aae10 0x7f229c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.419+0000 7f22b37fe700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] conn(0x7f229c0aae10 0x7f229c0ad230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.420+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== osd.1 v2:172.21.15.98:6800/2863958609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f229c0ad8e0 con 0x7f229c0aae10 2023-09-23T09:28:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.437+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 --> [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f22b404eb20 con 0x7f229c0aae10 2023-09-23T09:28:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22b17fa700 1 -- 172.21.15.45:0/1718272934 <== osd.1 v2:172.21.15.98:6800/2863958609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f22b404eb20 con 0x7f229c0aae10 2023-09-23T09:28:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] conn(0x7f229c0aae10 msgr2=0x7f229c0ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] conn(0x7f229c0aae10 0x7f229c0ad230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f229c0a1630 msgr2=0x7f229c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f229c0a1630 0x7f229c0a3af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b5c0 tx=0x7f22a4005fb0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b4106180 msgr2=0x7f22b419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 --2- 172.21.15.45:0/1718272934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22b4106180 0x7f22b419b950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00ada0 tx=0x7f22ac00c330 comp rx=0 tx=0).stop 2023-09-23T09:28:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22b8bf4700 1 -- 172.21.15.45:0/1718272934 reap_dead start 2023-09-23T09:28:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 shutdown_connections 2023-09-23T09:28:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 >> 172.21.15.45:0/1718272934 conn(0x7f22b40fc910 msgr2=0x7f22b40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 shutdown_connections 2023-09-23T09:28:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.438+0000 7f22ba657700 1 -- 172.21.15.45:0/1718272934 wait complete. 2023-09-23T09:28:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.820+0000 7f61c76ec700 1 -- 172.21.15.45:0/504820132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c00fffd0 msgr2=0x7f61c01003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.820+0000 7f61c76ec700 1 --2- 172.21.15.45:0/504820132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c00fffd0 0x7f61c01003b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009990 tx=0x7f61bc009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.820+0000 7f61c76ec700 1 -- 172.21.15.45:0/504820132 shutdown_connections 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.820+0000 7f61c76ec700 1 --2- 172.21.15.45:0/504820132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c01050a0 0x7f61c010cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.820+0000 7f61c76ec700 1 --2- 172.21.15.45:0/504820132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c00fffd0 0x7f61c01003b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.820+0000 7f61c76ec700 1 --2- 172.21.15.45:0/504820132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c0100980 0x7f61c01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.820+0000 7f61c76ec700 1 -- 172.21.15.45:0/504820132 >> 172.21.15.45:0/504820132 conn(0x7f61c0076ea0 msgr2=0x7f61c00772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.821+0000 7f61c76ec700 1 -- 172.21.15.45:0/504820132 shutdown_connections 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.821+0000 7f61c76ec700 1 -- 172.21.15.45:0/504820132 wait complete. 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.821+0000 7f61c76ec700 1 Processor -- start 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.821+0000 7f61c76ec700 1 -- start start 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.821+0000 7f61c76ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00fffd0 0x7f61c0196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c76ec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c0100980 0x7f61c01973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c76ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c01050a0 0x7f61c019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c76ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c019be20 con 0x7f61c01050a0 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c76ec700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c019bf90 con 0x7f61c0100980 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c76ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c019c150 con 0x7f61c00fffd0 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c01050a0 0x7f61c019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c01050a0 0x7f61c019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59082/0 (socket says 172.21.15.45:59082) 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 -- 172.21.15.45:0/2054642081 learned_addr learned my addr 172.21.15.45:0/2054642081 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5488700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00fffd0 0x7f61c0196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c4c87700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c0100980 0x7f61c01973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 -- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00fffd0 msgr2=0x7f61c0196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00fffd0 0x7f61c0196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 -- 172.21.15.45:0/2054642081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c0100980 msgr2=0x7f61c01973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c0100980 0x7f61c01973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 -- 172.21.15.45:0/2054642081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc0095f0 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c5c89700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c01050a0 0x7f61c019b810 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007d70 tx=0x7f61b800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800ac90 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b800adf0 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80115e0 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.822+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c019c430 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.823+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c01a3c90 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.824+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61c004eb20 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.825+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f61b8012420 con 0x7f61c01050a0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61b67fc700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f61ac0a16b0 0x7f61ac0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61c5488700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f61ac0a16b0 0x7f61ac0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f61b80cd450 con 0x7f61c01050a0 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61b67fc700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] conn(0x7f61ac0aae90 0x7f61ac0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 --> [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61ac0ad960 con 0x7f61ac0aae90 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b80cd8f0 con 0x7f61c01050a0 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61c5488700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f61ac0a16b0 0x7f61ac0a3b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00b5c0 tx=0x7f61bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.826+0000 7f61c4c87700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] conn(0x7f61ac0aae90 0x7f61ac0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.827+0000 7f61c4c87700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] conn(0x7f61ac0aae90 0x7f61ac0ad2b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.827+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== osd.8 v2:172.21.15.123:6816/101690512 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f61ac0ad960 con 0x7f61ac0aae90 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.855+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 --> [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61c010ffb0 con 0x7f61ac0aae90 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61b67fc700 1 -- 172.21.15.45:0/2054642081 <== osd.8 v2:172.21.15.123:6816/101690512 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f61b000bd30 con 0x7f61ac0aae90 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] conn(0x7f61ac0aae90 msgr2=0x7f61ac0ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] conn(0x7f61ac0aae90 0x7f61ac0ad2b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f61ac0a16b0 msgr2=0x7f61ac0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f61ac0a16b0 0x7f61ac0a3b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00b5c0 tx=0x7f61bc005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c01050a0 msgr2=0x7f61c019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 --2- 172.21.15.45:0/2054642081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c01050a0 0x7f61c019b810 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007d70 tx=0x7f61b800a340 comp rx=0 tx=0).stop 2023-09-23T09:28:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c5c89700 1 -- 172.21.15.45:0/2054642081 reap_dead start 2023-09-23T09:28:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 shutdown_connections 2023-09-23T09:28:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 >> 172.21.15.45:0/2054642081 conn(0x7f61c0076ea0 msgr2=0x7f61c0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 shutdown_connections 2023-09-23T09:28:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:13.856+0000 7f61c76ec700 1 -- 172.21.15.45:0/2054642081 wait complete. 2023-09-23T09:28:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 -- 172.21.15.45:0/4011991582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ffcd0 msgr2=0x7f917010dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 --2- 172.21.15.45:0/4011991582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ffcd0 0x7f917010dd00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f916800b0a0 tx=0x7f916800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 -- 172.21.15.45:0/4011991582 shutdown_connections 2023-09-23T09:28:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 --2- 172.21.15.45:0/4011991582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ffcd0 0x7f917010dd00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 --2- 172.21.15.45:0/4011991582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9170105e60 0x7f9170106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 --2- 172.21.15.45:0/4011991582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700ff310 0x7f91700ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 -- 172.21.15.45:0/4011991582 >> 172.21.15.45:0/4011991582 conn(0x7f9170076290 msgr2=0x7f91700fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 -- 172.21.15.45:0/4011991582 shutdown_connections 2023-09-23T09:28:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.565+0000 7f917672b700 1 -- 172.21.15.45:0/4011991582 wait complete. 2023-09-23T09:28:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 Processor -- start 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 -- start start 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ff310 0x7f9170197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700ffcd0 0x7f91701975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9170105e60 0x7f917019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917019c040 con 0x7f91700ff310 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917019c1b0 con 0x7f9170105e60 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.566+0000 7f917672b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917019c370 con 0x7f91700ffcd0 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ff310 0x7f9170197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ff310 0x7f9170197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59112/0 (socket says 172.21.15.45:59112) 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916ffff700 1 -- 172.21.15.45:0/1442322644 learned_addr learned my addr 172.21.15.45:0/1442322644 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916f7fe700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700ffcd0 0x7f91701975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f9174cc8700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9170105e60 0x7f917019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916f7fe700 1 -- 172.21.15.45:0/1442322644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9170105e60 msgr2=0x7f917019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916f7fe700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9170105e60 0x7f917019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916f7fe700 1 -- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ff310 msgr2=0x7f9170197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916f7fe700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91700ff310 0x7f9170197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916f7fe700 1 -- 172.21.15.45:0/1442322644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9168009d00 con 0x7f91700ffcd0 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.567+0000 7f916f7fe700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700ffcd0 0x7f91701975c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f916400d750 tx=0x7f916400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.568+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164004020 con 0x7f91700ffcd0 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.568+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9164004830 con 0x7f91700ffcd0 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.568+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164005550 con 0x7f91700ffcd0 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.568+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917019c650 con 0x7f91700ffcd0 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.568+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91701005f0 con 0x7f91700ffcd0 2023-09-23T09:28:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.569+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9170111120 con 0x7f91700ffcd0 2023-09-23T09:28:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.571+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f9164004ca0 con 0x7f91700ffcd0 2023-09-23T09:28:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.571+0000 7f916d7fa700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91580a16b0 0x7f91580a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.571+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91640ccb30 con 0x7f91700ffcd0 2023-09-23T09:28:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.572+0000 7f916ffff700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91580a16b0 0x7f91580a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.572+0000 7f916d7fa700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] conn(0x7f91580aae90 0x7f91580ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.572+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 --> [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91580ad960 con 0x7f91580aae90 2023-09-23T09:28:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.572+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91640ccfd0 con 0x7f91700ffcd0 2023-09-23T09:28:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.572+0000 7f9174cc8700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] conn(0x7f91580aae90 0x7f91580ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.572+0000 7f916ffff700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91580a16b0 0x7f91580a3b70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9160005950 tx=0x7f916000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.573+0000 7f9174cc8700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] conn(0x7f91580aae90 0x7f91580ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.573+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== osd.6 v2:172.21.15.45:6818/668839971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f91580ad960 con 0x7f91580aae90 2023-09-23T09:28:14.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.590+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 --> [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f917004eb20 con 0x7f91580aae90 2023-09-23T09:28:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.590+0000 7f916d7fa700 1 -- 172.21.15.45:0/1442322644 <== osd.6 v2:172.21.15.45:6818/668839971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f917004eb20 con 0x7f91580aae90 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] conn(0x7f91580aae90 msgr2=0x7f91580ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] conn(0x7f91580aae90 0x7f91580ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91580a16b0 msgr2=0x7f91580a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91580a16b0 0x7f91580a3b70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9160005950 tx=0x7f916000b500 comp rx=0 tx=0).stop 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700ffcd0 msgr2=0x7f91701975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 --2- 172.21.15.45:0/1442322644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700ffcd0 0x7f91701975c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f916400d750 tx=0x7f916400da60 comp rx=0 tx=0).stop 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f9174cc8700 1 -- 172.21.15.45:0/1442322644 reap_dead start 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 shutdown_connections 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 >> 172.21.15.45:0/1442322644 conn(0x7f9170076290 msgr2=0x7f91700fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 shutdown_connections 2023-09-23T09:28:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:14.591+0000 7f917672b700 1 -- 172.21.15.45:0/1442322644 wait complete. 2023-09-23T09:28:15.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:15 smithi045 ceph-mon[98627]: pgmap v263: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-09-23T09:28:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:15 smithi098 ceph-mon[102386]: pgmap v263: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-09-23T09:28:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:15 smithi123 ceph-mon[103577]: pgmap v263: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.168+0000 7f027328a700 1 -- 172.21.15.45:0/2239902690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c1010a0 msgr2=0x7f026c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.168+0000 7f027328a700 1 --2- 172.21.15.45:0/2239902690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c1010a0 0x7f026c101480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.170+0000 7f027328a700 1 -- 172.21.15.45:0/2239902690 shutdown_connections 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.170+0000 7f027328a700 1 --2- 172.21.15.45:0/2239902690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f026c106180 0x7f026c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.170+0000 7f027328a700 1 --2- 172.21.15.45:0/2239902690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c1010a0 0x7f026c101480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.170+0000 7f027328a700 1 --2- 172.21.15.45:0/2239902690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026c101a50 0x7f026c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.170+0000 7f027328a700 1 -- 172.21.15.45:0/2239902690 >> 172.21.15.45:0/2239902690 conn(0x7f026c0fc910 msgr2=0x7f026c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.170+0000 7f027328a700 1 -- 172.21.15.45:0/2239902690 shutdown_connections 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.170+0000 7f027328a700 1 -- 172.21.15.45:0/2239902690 wait complete. 2023-09-23T09:28:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 Processor -- start 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 -- start start 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026c101a50 0x7f026c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f026c106180 0x7f026c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c19baf0 0x7f026c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c19c4e0 con 0x7f026c19baf0 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c19c650 con 0x7f026c101a50 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f027328a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c19c810 con 0x7f026c106180 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f0271827700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c19baf0 0x7f026c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f0271827700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c19baf0 0x7f026c19bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59126/0 (socket says 172.21.15.45:59126) 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f0271827700 1 -- 172.21.15.45:0/969187483 learned_addr learned my addr 172.21.15.45:0/969187483 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.171+0000 7f0271026700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026c101a50 0x7f026c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0270825700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f026c106180 0x7f026c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0271827700 1 -- 172.21.15.45:0/969187483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f026c106180 msgr2=0x7f026c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0271827700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f026c106180 0x7f026c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0271827700 1 -- 172.21.15.45:0/969187483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026c101a50 msgr2=0x7f026c197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0271827700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f026c101a50 0x7f026c197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0271827700 1 -- 172.21.15.45:0/969187483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680095f0 con 0x7f026c19baf0 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0270825700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f026c106180 0x7f026c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f0271827700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c19baf0 0x7f026c19bf70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0264007ee0 tx=0x7f026400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264017070 con 0x7f026c19baf0 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c1a3da0 con 0x7f026c19baf0 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026400f040 con 0x7f026c19baf0 2023-09-23T09:28:16.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264005740 con 0x7f026c19baf0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.172+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c1a42a0 con 0x7f026c19baf0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.173+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f026c111090 con 0x7f026c19baf0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.175+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f026400dc60 con 0x7f026c19baf0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.175+0000 7f02627fc700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02580a1630 0x7f02580a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f02640ccd20 con 0x7f026c19baf0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f0271026700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02580a1630 0x7f02580a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f02627fc700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] conn(0x7f02580aae10 0x7f02580ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 --> [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02580ad8e0 con 0x7f02580aae10 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02640cd1c0 con 0x7f026c19baf0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f0270825700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] conn(0x7f02580aae10 0x7f02580ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f0271026700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02580a1630 0x7f02580a3af0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.176+0000 7f0270825700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] conn(0x7f02580aae10 0x7f02580ad230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.177+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== osd.0 v2:172.21.15.45:6802/1500595998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f02580ad8e0 con 0x7f02580aae10 2023-09-23T09:28:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 --> [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f026c04eb20 con 0x7f02580aae10 2023-09-23T09:28:16.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f02627fc700 1 -- 172.21.15.45:0/969187483 <== osd.0 v2:172.21.15.45:6802/1500595998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f026c04eb20 con 0x7f02580aae10 2023-09-23T09:28:16.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] conn(0x7f02580aae10 msgr2=0x7f02580ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f027328a700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] conn(0x7f02580aae10 0x7f02580ad230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02580a1630 msgr2=0x7f02580a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f027328a700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f02580a1630 0x7f02580a3af0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c19baf0 msgr2=0x7f026c19bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f027328a700 1 --2- 172.21.15.45:0/969187483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c19baf0 0x7f026c19bf70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0264007ee0 tx=0x7f026400d3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.196+0000 7f0271827700 1 -- 172.21.15.45:0/969187483 reap_dead start 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.197+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 shutdown_connections 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.197+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 >> 172.21.15.45:0/969187483 conn(0x7f026c0fc910 msgr2=0x7f026c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.197+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 shutdown_connections 2023-09-23T09:28:16.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.197+0000 7f027328a700 1 -- 172.21.15.45:0/969187483 wait complete. 2023-09-23T09:28:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 -- 172.21.15.45:0/1536439800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 msgr2=0x7ffa50068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 --2- 172.21.15.45:0/1536439800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 0x7ffa50068dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009990 tx=0x7ffa40009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 -- 172.21.15.45:0/1536439800 shutdown_connections 2023-09-23T09:28:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 --2- 172.21.15.45:0/1536439800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50069300 0x7ffa5010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 --2- 172.21.15.45:0/1536439800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa50106fb0 0x7ffa50107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 --2- 172.21.15.45:0/1536439800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 0x7ffa50068dc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 -- 172.21.15.45:0/1536439800 >> 172.21.15.45:0/1536439800 conn(0x7ffa50076e20 msgr2=0x7ffa50077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 -- 172.21.15.45:0/1536439800 shutdown_connections 2023-09-23T09:28:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.336+0000 7ffa58088700 1 -- 172.21.15.45:0/1536439800 wait complete. 2023-09-23T09:28:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 Processor -- start 2023-09-23T09:28:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 -- start start 2023-09-23T09:28:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 0x7ffa50196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50069300 0x7ffa501973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa50106fb0 0x7ffa5019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019be30 con 0x7ffa50068940 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019bfa0 con 0x7ffa50106fb0 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa58088700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019c160 con 0x7ffa50069300 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa55e24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 0x7ffa50196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa55e24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 0x7ffa50196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59154/0 (socket says 172.21.15.45:59154) 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa55e24700 1 -- 172.21.15.45:0/3341955269 learned_addr learned my addr 172.21.15.45:0/3341955269 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.337+0000 7ffa55623700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50069300 0x7ffa501973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa56625700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa50106fb0 0x7ffa5019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa55e24700 1 -- 172.21.15.45:0/3341955269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50069300 msgr2=0x7ffa501973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa55e24700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50069300 0x7ffa501973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa55e24700 1 -- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa50106fb0 msgr2=0x7ffa5019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa55e24700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa50106fb0 0x7ffa5019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa55e24700 1 -- 172.21.15.45:0/3341955269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa50068940 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa56625700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa50106fb0 0x7ffa5019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa55e24700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 0x7ffa50196e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00c920 tx=0x7ffa4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c007b00 con 0x7ffa50068940 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5019c440 con 0x7ffa50068940 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4c00ce70 con 0x7ffa50068940 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c005460 con 0x7ffa50068940 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.338+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa501a3ca0 con 0x7ffa50068940 2023-09-23T09:28:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.339+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa5004eb20 con 0x7ffa50068940 2023-09-23T09:28:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.341+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7ffa4c004750 con 0x7ffa50068940 2023-09-23T09:28:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.341+0000 7ffa46ffd700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffa3c0a1630 0x7ffa3c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.342+0000 7ffa55623700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffa3c0a1630 0x7ffa3c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.342+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffa4c0ccf60 con 0x7ffa50068940 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.342+0000 7ffa46ffd700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] conn(0x7ffa3c0aae10 0x7ffa3c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.342+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 --> [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa3c0ad8e0 con 0x7ffa3c0aae10 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.342+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa4c0cd400 con 0x7ffa50068940 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.342+0000 7ffa55623700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffa3c0a1630 0x7ffa3c0a3af0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa50198490 tx=0x7ffa40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.342+0000 7ffa56625700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] conn(0x7ffa3c0aae10 0x7ffa3c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.343+0000 7ffa56625700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] conn(0x7ffa3c0aae10 0x7ffa3c0ad230 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.344+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== osd.4 v2:172.21.15.98:6808/4058003666 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ffa3c0ad8e0 con 0x7ffa3c0aae10 2023-09-23T09:28:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.360+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 --> [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa50111080 con 0x7ffa3c0aae10 2023-09-23T09:28:16.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.361+0000 7ffa46ffd700 1 -- 172.21.15.45:0/3341955269 <== osd.4 v2:172.21.15.98:6808/4058003666 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffa48009980 con 0x7ffa3c0aae10 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.361+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] conn(0x7ffa3c0aae10 msgr2=0x7ffa3c0ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.361+0000 7ffa58088700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] conn(0x7ffa3c0aae10 0x7ffa3c0ad230 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffa3c0a1630 msgr2=0x7ffa3c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffa3c0a1630 0x7ffa3c0a3af0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa50198490 tx=0x7ffa40005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 msgr2=0x7ffa50196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 --2- 172.21.15.45:0/3341955269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50068940 0x7ffa50196e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00c920 tx=0x7ffa4c00cc30 comp rx=0 tx=0).stop 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa56625700 1 -- 172.21.15.45:0/3341955269 reap_dead start 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 shutdown_connections 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 >> 172.21.15.45:0/3341955269 conn(0x7ffa50076e20 msgr2=0x7ffa500fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 shutdown_connections 2023-09-23T09:28:16.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.362+0000 7ffa58088700 1 -- 172.21.15.45:0/3341955269 wait complete. 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 -- 172.21.15.45:0/2609550682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069a90 msgr2=0x7f007010d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 --2- 172.21.15.45:0/2609550682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069a90 0x7f007010d590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0064009990 tx=0x7f0064009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 -- 172.21.15.45:0/2609550682 shutdown_connections 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 --2- 172.21.15.45:0/2609550682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f007010dad0 0x7f007010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 --2- 172.21.15.45:0/2609550682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0070069170 0x7f0070069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 --2- 172.21.15.45:0/2609550682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069a90 0x7f007010d590 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 -- 172.21.15.45:0/2609550682 >> 172.21.15.45:0/2609550682 conn(0x7f0070077c10 msgr2=0x7f0070078020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:16.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 -- 172.21.15.45:0/2609550682 shutdown_connections 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 -- 172.21.15.45:0/2609550682 wait complete. 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.251+0000 7f0075301700 1 Processor -- start 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f0075301700 1 -- start start 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f0075301700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069170 0x7f0070199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f0075301700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0070069a90 0x7f0070199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f0075301700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f007010dad0 0x7f007019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f0075301700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007019dfc0 con 0x7f0070069170 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f0075301700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007019e130 con 0x7f007010dad0 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f0075301700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007019e2f0 con 0x7f0070069a90 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069170 0x7f0070199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069170 0x7f0070199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59140/0 (socket says 172.21.15.45:59140) 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 -- 172.21.15.45:0/1170851373 learned_addr learned my addr 172.21.15.45:0/1170851373 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006f7fe700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f007010dad0 0x7f007019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006e7fc700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0070069a90 0x7f0070199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 -- 172.21.15.45:0/1170851373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0070069a90 msgr2=0x7f0070199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0070069a90 0x7f0070199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 -- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f007010dad0 msgr2=0x7f007019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f007010dad0 0x7f007019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 -- 172.21.15.45:0/1170851373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00640095f0 con 0x7f0070069170 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.252+0000 7f006effd700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069170 0x7f0070199000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f006000d950 tx=0x7f006000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.253+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060009940 con 0x7f0070069170 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.253+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0060010460 con 0x7f0070069170 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.253+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000f620 con 0x7f0070069170 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.253+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007019e5d0 con 0x7f0070069170 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.253+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00701a5e30 con 0x7f0070069170 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.254+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00701131d0 con 0x7f0070069170 2023-09-23T09:28:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.255+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f00600108d0 con 0x7f0070069170 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.255+0000 7f0057fff700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f00580a1630 0x7f00580a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.256+0000 7f006e7fc700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f00580a1630 0x7f00580a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.256+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00600cce30 con 0x7f0070069170 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.256+0000 7f0057fff700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] conn(0x7f00580aae10 0x7f00580ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.256+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 --> [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00580ad8e0 con 0x7f00580aae10 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.256+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00600cf410 con 0x7f0070069170 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.256+0000 7f006e7fc700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f00580a1630 0x7f00580a3af0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0064006010 tx=0x7f006400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.256+0000 7f006f7fe700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] conn(0x7f00580aae10 0x7f00580ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.257+0000 7f006f7fe700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] conn(0x7f00580aae10 0x7f00580ad230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.257+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== osd.7 v2:172.21.15.98:6816/3609791505 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f00580ad8e0 con 0x7f00580aae10 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.274+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 --> [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f007004eb20 con 0x7f00580aae10 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0057fff700 1 -- 172.21.15.45:0/1170851373 <== osd.7 v2:172.21.15.98:6816/3609791505 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f007004eb20 con 0x7f00580aae10 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] conn(0x7f00580aae10 msgr2=0x7f00580ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] conn(0x7f00580aae10 0x7f00580ad230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f00580a1630 msgr2=0x7f00580a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f00580a1630 0x7f00580a3af0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0064006010 tx=0x7f006400b540 comp rx=0 tx=0).stop 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069170 msgr2=0x7f0070199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 --2- 172.21.15.45:0/1170851373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070069170 0x7f0070199000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f006000d950 tx=0x7f006000dc60 comp rx=0 tx=0).stop 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f006f7fe700 1 -- 172.21.15.45:0/1170851373 reap_dead start 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 shutdown_connections 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.275+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 >> 172.21.15.45:0/1170851373 conn(0x7f0070077c10 msgr2=0x7f00701075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.276+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 shutdown_connections 2023-09-23T09:28:16.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.276+0000 7f0075301700 1 -- 172.21.15.45:0/1170851373 wait complete. 2023-09-23T09:28:17.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:17 smithi045 ceph-mon[98627]: pgmap v264: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:17.653 INFO:teuthology.orchestra.run.smithi045.stdout:141733920814 2023-09-23T09:28:17.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-09-23T09:28:17.680 INFO:teuthology.orchestra.run.smithi045.stdout:55834574906 2023-09-23T09:28:17.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-09-23T09:28:17.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:17 smithi098 ceph-mon[102386]: pgmap v264: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 -- 172.21.15.45:0/1053227215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0106fb0 msgr2=0x7f6db0107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 --2- 172.21.15.45:0/1053227215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0106fb0 0x7f6db0107390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009990 tx=0x7f6da4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 -- 172.21.15.45:0/1053227215 shutdown_connections 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 --2- 172.21.15.45:0/1053227215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6db0069300 0x7f6db010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 --2- 172.21.15.45:0/1053227215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0106fb0 0x7f6db0107390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 --2- 172.21.15.45:0/1053227215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db0068940 0x7f6db0068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 -- 172.21.15.45:0/1053227215 >> 172.21.15.45:0/1053227215 conn(0x7f6db0076e20 msgr2=0x7f6db0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 -- 172.21.15.45:0/1053227215 shutdown_connections 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 -- 172.21.15.45:0/1053227215 wait complete. 2023-09-23T09:28:17.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 Processor -- start 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.422+0000 7f6db6c67700 1 -- start start 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db6c67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0068940 0x7f6db00ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db6c67700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6db0069300 0x7f6db0100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db6c67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db0106fb0 0x7f6db0100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db6c67700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01a52f0 con 0x7f6db0068940 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db6c67700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01a5460 con 0x7f6db0069300 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db6c67700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01a5620 con 0x7f6db0106fb0 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0068940 0x7f6db00ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0068940 0x7f6db00ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59170/0 (socket says 172.21.15.45:59170) 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 -- 172.21.15.45:0/2865182641 learned_addr learned my addr 172.21.15.45:0/2865182641 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6daffff700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6db0069300 0x7f6db0100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 -- 172.21.15.45:0/2865182641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db0106fb0 msgr2=0x7f6db0100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:17.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db0106fb0 0x7f6db0100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 -- 172.21.15.45:0/2865182641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6db0069300 msgr2=0x7f6db0100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6db0069300 0x7f6db0100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 -- 172.21.15.45:0/2865182641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da40095f0 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6daffff700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6db0069300 0x7f6db0100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db4a03700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0068940 0x7f6db00ffdd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6da4000c00 tx=0x7f6da400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da401d070 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db01a58a0 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6da400fc20 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.423+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4017650 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.424+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db01a5d40 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.424+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6db0111020 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.426+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f6da4021410 con 0x7f6db0068940 2023-09-23T09:28:17.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.426+0000 7f6dadffb700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6da00a1630 0x7f6da00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.426+0000 7f6daffff700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6da00a1630 0x7f6da00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.426+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6da40cedf0 con 0x7f6db0068940 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.426+0000 7f6dadffb700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] conn(0x7f6da00aae10 0x7f6da00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.426+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 --> [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6da00ad8e0 con 0x7f6da00aae10 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.426+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da40cf290 con 0x7f6db0068940 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.427+0000 7f6daffff700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6da00a1630 0x7f6da00a3af0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6db01015c0 tx=0x7f6d9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.427+0000 7f6db5204700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] conn(0x7f6da00aae10 0x7f6da00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.427+0000 7f6db5204700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] conn(0x7f6da00aae10 0x7f6da00ad230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.428+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== osd.2 v2:172.21.15.123:6800/976297925 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6da00ad8e0 con 0x7f6da00aae10 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.444+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 --> [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6db004eb20 con 0x7f6da00aae10 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.445+0000 7f6dadffb700 1 -- 172.21.15.45:0/2865182641 <== osd.2 v2:172.21.15.123:6800/976297925 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6db004eb20 con 0x7f6da00aae10 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.445+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 >> [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] conn(0x7f6da00aae10 msgr2=0x7f6da00ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.445+0000 7f6db6c67700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] conn(0x7f6da00aae10 0x7f6da00ad230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.445+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6da00a1630 msgr2=0x7f6da00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.445+0000 7f6db6c67700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6da00a1630 0x7f6da00a3af0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6db01015c0 tx=0x7f6d9c009260 comp rx=0 tx=0).stop 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.446+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0068940 msgr2=0x7f6db00ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.446+0000 7f6db6c67700 1 --2- 172.21.15.45:0/2865182641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6db0068940 0x7f6db00ffdd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6da4000c00 tx=0x7f6da400f510 comp rx=0 tx=0).stop 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.446+0000 7f6db5204700 1 -- 172.21.15.45:0/2865182641 reap_dead start 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.446+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 shutdown_connections 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.446+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 >> 172.21.15.45:0/2865182641 conn(0x7f6db0076e20 msgr2=0x7f6db00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.446+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 shutdown_connections 2023-09-23T09:28:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:16.446+0000 7f6db6c67700 1 -- 172.21.15.45:0/2865182641 wait complete. 2023-09-23T09:28:17.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:17 smithi123 ceph-mon[103577]: pgmap v264: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:18.101 INFO:teuthology.orchestra.run.smithi045.stdout:249108103196 2023-09-23T09:28:18.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-09-23T09:28:18.273 INFO:teuthology.orchestra.run.smithi045.stdout:193273528358 2023-09-23T09:28:18.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-09-23T09:28:18.621 INFO:teuthology.orchestra.run.smithi045.stdout:34359738431 2023-09-23T09:28:18.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-09-23T09:28:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 -- 172.21.15.45:0/3738339276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea881031f0 msgr2=0x7fea8810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 --2- 172.21.15.45:0/3738339276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea881031f0 0x7fea8810de10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea8000b0d0 tx=0x7fea8000b3e0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 -- 172.21.15.45:0/3738339276 shutdown_connections 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 --2- 172.21.15.45:0/3738339276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea881031f0 0x7fea8810de10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 --2- 172.21.15.45:0/3738339276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea88069530 0x7fea88069910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 --2- 172.21.15.45:0/3738339276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea88069e50 0x7fea88102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 -- 172.21.15.45:0/3738339276 >> 172.21.15.45:0/3738339276 conn(0x7fea88076e10 msgr2=0x7fea88077220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 -- 172.21.15.45:0/3738339276 shutdown_connections 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.695+0000 7fea8f9d6700 1 -- 172.21.15.45:0/3738339276 wait complete. 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 Processor -- start 2023-09-23T09:28:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 -- start start 2023-09-23T09:28:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea88069530 0x7fea880ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea88069e50 0x7fea88100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881031f0 0x7fea88100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a5540 con 0x7fea88069530 2023-09-23T09:28:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a56b0 con 0x7fea881031f0 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8f9d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a5870 con 0x7fea88069e50 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8d772700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea88069530 0x7fea880ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8d772700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea88069530 0x7fea880ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59186/0 (socket says 172.21.15.45:59186) 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8d772700 1 -- 172.21.15.45:0/691040707 learned_addr learned my addr 172.21.15.45:0/691040707 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8df73700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881031f0 0x7fea88100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.696+0000 7fea8cf71700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea88069e50 0x7fea88100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8d772700 1 -- 172.21.15.45:0/691040707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea88069e50 msgr2=0x7fea88100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8d772700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea88069e50 0x7fea88100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8d772700 1 -- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881031f0 msgr2=0x7fea88100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8d772700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881031f0 0x7fea88100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8d772700 1 -- 172.21.15.45:0/691040707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80009d00 con 0x7fea88069530 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8d772700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea88069530 0x7fea880ffdd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c9b0 tx=0x7fea8400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84007b00 con 0x7fea88069530 2023-09-23T09:28:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea881a5b50 con 0x7fea88069530 2023-09-23T09:28:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea8400ce50 con 0x7fea88069530 2023-09-23T09:28:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84018660 con 0x7fea88069530 2023-09-23T09:28:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.697+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea881a60a0 con 0x7fea88069530 2023-09-23T09:28:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.699+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea881111a0 con 0x7fea88069530 2023-09-23T09:28:18.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.700+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fea8400f450 con 0x7fea88069530 2023-09-23T09:28:18.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea7e7fc700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fea740a1630 0x7fea740a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea8cf71700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fea740a1630 0x7fea740a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fea840ccdf0 con 0x7fea88069530 2023-09-23T09:28:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea7e7fc700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] conn(0x7fea740aae10 0x7fea740ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 --> [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea740ad8e0 con 0x7fea740aae10 2023-09-23T09:28:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea840cf500 con 0x7fea88069530 2023-09-23T09:28:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea8cf71700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fea740a1630 0x7fea740a3af0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea88101520 tx=0x7fea780073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.701+0000 7fea8df73700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] conn(0x7fea740aae10 0x7fea740ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.702+0000 7fea8df73700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] conn(0x7fea740aae10 0x7fea740ad230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.703+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== osd.10 v2:172.21.15.98:6824/2174091835 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fea740ad8e0 con 0x7fea740aae10 2023-09-23T09:28:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.720+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 --> [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea8804eb20 con 0x7fea740aae10 2023-09-23T09:28:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea7e7fc700 1 -- 172.21.15.45:0/691040707 <== osd.10 v2:172.21.15.98:6824/2174091835 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fea8804eb20 con 0x7fea740aae10 2023-09-23T09:28:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] conn(0x7fea740aae10 msgr2=0x7fea740ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] conn(0x7fea740aae10 0x7fea740ad230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fea740a1630 msgr2=0x7fea740a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fea740a1630 0x7fea740a3af0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea88101520 tx=0x7fea780073d0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea88069530 msgr2=0x7fea880ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 --2- 172.21.15.45:0/691040707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea88069530 0x7fea880ffdd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c9b0 tx=0x7fea8400ccc0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8df73700 1 -- 172.21.15.45:0/691040707 reap_dead start 2023-09-23T09:28:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 shutdown_connections 2023-09-23T09:28:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.721+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 >> 172.21.15.45:0/691040707 conn(0x7fea88076e10 msgr2=0x7fea88105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.722+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 shutdown_connections 2023-09-23T09:28:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.722+0000 7fea8f9d6700 1 -- 172.21.15.45:0/691040707 wait complete. 2023-09-23T09:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1108734092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98101980 msgr2=0x7f4c98101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 --2- 172.21.15.45:0/1108734092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98101980 0x7f4c98101e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0099c0 tx=0x7f4c8c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1108734092 shutdown_connections 2023-09-23T09:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 --2- 172.21.15.45:0/1108734092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98102340 0x7f4c9810cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 --2- 172.21.15.45:0/1108734092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c981000a0 0x7f4c98100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 --2- 172.21.15.45:0/1108734092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98101980 0x7f4c98101e00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1108734092 >> 172.21.15.45:0/1108734092 conn(0x7f4c98076e10 msgr2=0x7f4c98077220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1108734092 shutdown_connections 2023-09-23T09:28:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.756+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1108734092 wait complete. 2023-09-23T09:28:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 Processor -- start 2023-09-23T09:28:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 -- start start 2023-09-23T09:28:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c981000a0 0x7f4c98197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c98101980 0x7f4c98197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98102340 0x7f4c9819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819bfd0 con 0x7f4c98102340 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819c140 con 0x7f4c98101980 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c9d75f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819c300 con 0x7f4c981000a0 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c977fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98102340 0x7f4c9819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c977fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98102340 0x7f4c9819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59206/0 (socket says 172.21.15.45:59206) 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c977fe700 1 -- 172.21.15.45:0/1127515582 learned_addr learned my addr 172.21.15.45:0/1127515582 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c967fc700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c98101980 0x7f4c98197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.757+0000 7f4c96ffd700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c981000a0 0x7f4c98197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c977fe700 1 -- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c981000a0 msgr2=0x7f4c98197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c977fe700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c981000a0 0x7f4c98197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c977fe700 1 -- 172.21.15.45:0/1127515582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c98101980 msgr2=0x7f4c98197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c977fe700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c98101980 0x7f4c98197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c977fe700 1 -- 172.21.15.45:0/1127515582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0095f0 con 0x7f4c98102340 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c977fe700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98102340 0x7f4c9819b9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000ef30 tx=0x7f4c9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90018070 con 0x7f4c98102340 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c9000f040 con 0x7f4c98102340 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9819c580 con 0x7f4c98102340 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9000cd00 con 0x7f4c98102340 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.758+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c981a3e40 con 0x7f4c98102340 2023-09-23T09:28:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.759+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c9804eb20 con 0x7f4c98102340 2023-09-23T09:28:18.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.760+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f4c90003a00 con 0x7f4c98102340 2023-09-23T09:28:18.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.761+0000 7f4c7ffff700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4c800a1630 0x7f4c800a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.761+0000 7f4c96ffd700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4c800a1630 0x7f4c800a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.762+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4c900cca00 con 0x7f4c98102340 2023-09-23T09:28:18.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.762+0000 7f4c7ffff700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] conn(0x7f4c800aae10 0x7f4c800ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:18.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.762+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 --> [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c800ad8e0 con 0x7f4c800aae10 2023-09-23T09:28:18.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.762+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c900d12c0 con 0x7f4c98102340 2023-09-23T09:28:18.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.762+0000 7f4c96ffd700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4c800a1630 0x7f4c800a3af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009c60 tx=0x7f4c88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:18.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.762+0000 7f4c967fc700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] conn(0x7f4c800aae10 0x7f4c800ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:18.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.763+0000 7f4c967fc700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] conn(0x7f4c800aae10 0x7f4c800ad230 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:18.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.763+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== osd.11 v2:172.21.15.123:6824/523003313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4c800ad8e0 con 0x7f4c800aae10 2023-09-23T09:28:18.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.780+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 --> [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c98110000 con 0x7f4c800aae10 2023-09-23T09:28:18.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c7ffff700 1 -- 172.21.15.45:0/1127515582 <== osd.11 v2:172.21.15.123:6824/523003313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4c98110000 con 0x7f4c800aae10 2023-09-23T09:28:18.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] conn(0x7f4c800aae10 msgr2=0x7f4c800ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] conn(0x7f4c800aae10 0x7f4c800ad230 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4c800a1630 msgr2=0x7f4c800a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4c800a1630 0x7f4c800a3af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009c60 tx=0x7f4c88009260 comp rx=0 tx=0).stop 2023-09-23T09:28:18.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98102340 msgr2=0x7f4c9819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:18.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 --2- 172.21.15.45:0/1127515582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c98102340 0x7f4c9819b9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000ef30 tx=0x7f4c9000c3a0 comp rx=0 tx=0).stop 2023-09-23T09:28:18.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c977fe700 1 -- 172.21.15.45:0/1127515582 reap_dead start 2023-09-23T09:28:18.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 shutdown_connections 2023-09-23T09:28:18.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.781+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 >> 172.21.15.45:0/1127515582 conn(0x7f4c98076e10 msgr2=0x7f4c980691b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:18.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.782+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 shutdown_connections 2023-09-23T09:28:18.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:18.782+0000 7f4c9d75f700 1 -- 172.21.15.45:0/1127515582 wait complete. 2023-09-23T09:28:18.867 INFO:teuthology.orchestra.run.smithi045.stdout:214748364833 2023-09-23T09:28:18.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-09-23T09:28:19.264 INFO:teuthology.orchestra.run.smithi045.stdout:77309411383 2023-09-23T09:28:19.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-09-23T09:28:19.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 -- 172.21.15.45:0/2195374495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc106fb0 msgr2=0x7f5cfc107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 --2- 172.21.15.45:0/2195374495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc106fb0 0x7f5cfc107390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009990 tx=0x7f5cec009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 -- 172.21.15.45:0/2195374495 shutdown_connections 2023-09-23T09:28:19.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 --2- 172.21.15.45:0/2195374495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cfc069300 0x7f5cfc10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 --2- 172.21.15.45:0/2195374495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc106fb0 0x7f5cfc107390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 --2- 172.21.15.45:0/2195374495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cfc068940 0x7f5cfc068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 -- 172.21.15.45:0/2195374495 >> 172.21.15.45:0/2195374495 conn(0x7f5cfc076e20 msgr2=0x7f5cfc077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:19.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 -- 172.21.15.45:0/2195374495 shutdown_connections 2023-09-23T09:28:19.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.477+0000 7f5d0167c700 1 -- 172.21.15.45:0/2195374495 wait complete. 2023-09-23T09:28:19.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 Processor -- start 2023-09-23T09:28:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 -- start start 2023-09-23T09:28:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cfc068940 0x7f5cfc196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc069300 0x7f5cfc1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cfc106fb0 0x7f5cfc19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc19be70 con 0x7f5cfc069300 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc19bfe0 con 0x7f5cfc106fb0 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.478+0000 7f5d0167c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc19c1a0 con 0x7f5cfc068940 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc069300 0x7f5cfc1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc069300 0x7f5cfc1973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59220/0 (socket says 172.21.15.45:59220) 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 -- 172.21.15.45:0/2409525454 learned_addr learned my addr 172.21.15.45:0/2409525454 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfaffd700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cfc068940 0x7f5cfc196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfb7fe700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cfc106fb0 0x7f5cfc19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 -- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cfc068940 msgr2=0x7f5cfc196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cfc068940 0x7f5cfc196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 -- 172.21.15.45:0/2409525454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cfc106fb0 msgr2=0x7f5cfc19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cfc106fb0 0x7f5cfc19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 -- 172.21.15.45:0/2409525454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0095f0 con 0x7f5cfc069300 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfaffd700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cfc068940 0x7f5cfc196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5cfa7fc700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc069300 0x7f5cfc1973f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e9a0 tx=0x7f5cf000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0009790 con 0x7f5cfc069300 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cf000cbd0 con 0x7f5cfc069300 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc19c480 con 0x7f5cfc069300 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0018680 con 0x7f5cfc069300 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.479+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc1a3ce0 con 0x7f5cfc069300 2023-09-23T09:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.481+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5cfc111040 con 0x7f5cfc069300 2023-09-23T09:28:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.482+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f5cf000cd40 con 0x7f5cfc069300 2023-09-23T09:28:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.482+0000 7f5ce3fff700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ce40a16b0 0x7f5ce40a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.483+0000 7f5cfaffd700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ce40a16b0 0x7f5ce40a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.483+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5cf0014070 con 0x7f5cfc069300 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.483+0000 7f5ce3fff700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] conn(0x7f5ce40aae90 0x7f5ce40ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.483+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 --> [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ce40ad960 con 0x7f5ce40aae90 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.483+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf00cf630 con 0x7f5cfc069300 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.483+0000 7f5cfaffd700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ce40a16b0 0x7f5ce40a3b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00b2e0 tx=0x7f5cec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.483+0000 7f5cfb7fe700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] conn(0x7f5ce40aae90 0x7f5ce40ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.484+0000 7f5cfb7fe700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] conn(0x7f5ce40aae90 0x7f5ce40ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.485+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== osd.5 v2:172.21.15.123:6808/3175137003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5ce40ad960 con 0x7f5ce40aae90 2023-09-23T09:28:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.506+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 --> [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5cfc04eb20 con 0x7f5ce40aae90 2023-09-23T09:28:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5ce3fff700 1 -- 172.21.15.45:0/2409525454 <== osd.5 v2:172.21.15.123:6808/3175137003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5cfc04eb20 con 0x7f5ce40aae90 2023-09-23T09:28:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] conn(0x7f5ce40aae90 msgr2=0x7f5ce40ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5d0167c700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] conn(0x7f5ce40aae90 0x7f5ce40ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ce40a16b0 msgr2=0x7f5ce40a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5d0167c700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ce40a16b0 0x7f5ce40a3b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00b2e0 tx=0x7f5cec005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc069300 msgr2=0x7f5cfc1973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5d0167c700 1 --2- 172.21.15.45:0/2409525454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc069300 0x7f5cfc1973f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e9a0 tx=0x7f5cf000ecb0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f5cfb7fe700 1 -- 172.21.15.45:0/2409525454 reap_dead start 2023-09-23T09:28:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.508+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 shutdown_connections 2023-09-23T09:28:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.508+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 >> 172.21.15.45:0/2409525454 conn(0x7f5cfc076e20 msgr2=0x7f5cfc104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.508+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 shutdown_connections 2023-09-23T09:28:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.508+0000 7f5d0167c700 1 -- 172.21.15.45:0/2409525454 wait complete. 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 -- 172.21.15.45:0/2718161513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 msgr2=0x7f09bc102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 --2- 172.21.15.45:0/2718161513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 0x7f09bc102ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0099c0 tx=0x7f09ac009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 -- 172.21.15.45:0/2718161513 shutdown_connections 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 --2- 172.21.15.45:0/2718161513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09bc1031f0 0x7f09bc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 --2- 172.21.15.45:0/2718161513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09bc069530 0x7f09bc069910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 --2- 172.21.15.45:0/2718161513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 0x7f09bc102ac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 -- 172.21.15.45:0/2718161513 >> 172.21.15.45:0/2718161513 conn(0x7f09bc076e10 msgr2=0x7f09bc077220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 -- 172.21.15.45:0/2718161513 shutdown_connections 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.500+0000 7f09c3ec8700 1 -- 172.21.15.45:0/2718161513 wait complete. 2023-09-23T09:28:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 Processor -- start 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 -- start start 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09bc069530 0x7f09bc0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 0x7f09bc100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09bc1031f0 0x7f09bc100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1a55c0 con 0x7f09bc069e50 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1a5730 con 0x7f09bc1031f0 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c3ec8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1a58f0 con 0x7f09bc069530 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 0x7f09bc100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 0x7f09bc100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59236/0 (socket says 172.21.15.45:59236) 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1c64700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09bc069530 0x7f09bc0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 -- 172.21.15.45:0/3214451136 learned_addr learned my addr 172.21.15.45:0/3214451136 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c2465700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09bc1031f0 0x7f09bc100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 -- 172.21.15.45:0/3214451136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09bc069530 msgr2=0x7f09bc0ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09bc069530 0x7f09bc0ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 -- 172.21.15.45:0/3214451136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09bc1031f0 msgr2=0x7f09bc100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09bc1031f0 0x7f09bc100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1463700 1 -- 172.21.15.45:0/3214451136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0095f0 con 0x7f09bc069e50 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c1c64700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09bc069530 0x7f09bc0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.501+0000 7f09c2465700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09bc1031f0 0x7f09bc100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.502+0000 7f09c1463700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 0x7f09bc100310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009990 tx=0x7f09ac00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.502+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac01d070 con 0x7f09bc069e50 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.502+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc1a5b70 con 0x7f09bc069e50 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.502+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ac0054e0 con 0x7f09bc069e50 2023-09-23T09:28:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.502+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac004a70 con 0x7f09bc069e50 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.502+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc1a5f80 con 0x7f09bc069e50 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.503+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09bc111130 con 0x7f09bc069e50 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.504+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f09ac00fb80 con 0x7f09bc069e50 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.505+0000 7f09b2ffd700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09a80a16b0 0x7f09a80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.505+0000 7f09c1c64700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09a80a16b0 0x7f09a80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.505+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09ac0cd980 con 0x7f09bc069e50 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.505+0000 7f09b2ffd700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] conn(0x7f09a80aae90 0x7f09a80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.505+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 --> [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a80ad960 con 0x7f09a80aae90 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.505+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09ac0d22c0 con 0x7f09bc069e50 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.505+0000 7f09c2465700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] conn(0x7f09a80aae90 0x7f09a80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.506+0000 7f09c1c64700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09a80a16b0 0x7f09a80a3b70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f09b8011490 tx=0x7f09b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.506+0000 7f09c2465700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] conn(0x7f09a80aae90 0x7f09a80ad2b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.507+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== osd.3 v2:172.21.15.45:6810/3800235078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f09a80ad960 con 0x7f09a80aae90 2023-09-23T09:28:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.527+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 --> [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09bc04eb20 con 0x7f09a80aae90 2023-09-23T09:28:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09b2ffd700 1 -- 172.21.15.45:0/3214451136 <== osd.3 v2:172.21.15.45:6810/3800235078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f09bc04eb20 con 0x7f09a80aae90 2023-09-23T09:28:19.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] conn(0x7f09a80aae90 msgr2=0x7f09a80ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09c3ec8700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] conn(0x7f09a80aae90 0x7f09a80ad2b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09a80a16b0 msgr2=0x7f09a80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09c3ec8700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f09a80a16b0 0x7f09a80a3b70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f09b8011490 tx=0x7f09b8009260 comp rx=0 tx=0).stop 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 msgr2=0x7f09bc100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09c3ec8700 1 --2- 172.21.15.45:0/3214451136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09bc069e50 0x7f09bc100310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009990 tx=0x7f09ac00f690 comp rx=0 tx=0).stop 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.528+0000 7f09c2465700 1 -- 172.21.15.45:0/3214451136 reap_dead start 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.529+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 shutdown_connections 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.529+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 >> 172.21.15.45:0/3214451136 conn(0x7f09bc076e10 msgr2=0x7f09bc0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.529+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 shutdown_connections 2023-09-23T09:28:19.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:19.529+0000 7f09c3ec8700 1 -- 172.21.15.45:0/3214451136 wait complete. 2023-09-23T09:28:19.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:19 smithi045 ceph-mon[98627]: pgmap v265: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:19.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:19 smithi098 ceph-mon[102386]: pgmap v265: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:19 smithi123 ceph-mon[103577]: pgmap v265: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:20.208 INFO:teuthology.orchestra.run.smithi045.stdout:335007449105 2023-09-23T09:28:20.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-09-23T09:28:20.232 INFO:teuthology.orchestra.run.smithi045.stdout:304942678037 2023-09-23T09:28:20.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-09-23T09:28:20.281 INFO:teuthology.orchestra.run.smithi045.stdout:107374182450 2023-09-23T09:28:20.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-09-23T09:28:20.553 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:20.579 INFO:teuthology.orchestra.run.smithi045.stdout:163208757291 2023-09-23T09:28:20.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-09-23T09:28:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:21.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:21 smithi045 ceph-mon[98627]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-23T09:28:21.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:21 smithi098 ceph-mon[102386]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-23T09:28:21.568 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:21.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:21 smithi123 ceph-mon[103577]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-23T09:28:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.918+0000 7fc2881b1700 1 -- 172.21.15.45:0/1947163867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2801044f0 msgr2=0x7fc2801048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.918+0000 7fc2881b1700 1 --2- 172.21.15.45:0/1947163867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2801044f0 0x7fc2801048d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc27000aaa0 tx=0x7fc27000adb0 comp rx=0 tx=0).stop 2023-09-23T09:28:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.919+0000 7fc2881b1700 1 -- 172.21.15.45:0/1947163867 shutdown_connections 2023-09-23T09:28:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.919+0000 7fc2881b1700 1 --2- 172.21.15.45:0/1947163867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280100100 0x7fc28010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.919+0000 7fc2881b1700 1 --2- 172.21.15.45:0/1947163867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2801044f0 0x7fc2801048d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.919+0000 7fc2881b1700 1 --2- 172.21.15.45:0/1947163867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2800ff740 0x7fc2800ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.919+0000 7fc2881b1700 1 -- 172.21.15.45:0/1947163867 >> 172.21.15.45:0/1947163867 conn(0x7fc280076260 msgr2=0x7fc280076670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:22.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.919+0000 7fc2881b1700 1 -- 172.21.15.45:0/1947163867 shutdown_connections 2023-09-23T09:28:22.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.919+0000 7fc2881b1700 1 -- 172.21.15.45:0/1947163867 wait complete. 2023-09-23T09:28:22.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 Processor -- start 2023-09-23T09:28:22.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 -- start start 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2800ff740 0x7fc280197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc280100100 0x7fc280197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2801044f0 0x7fc28019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28019c000 con 0x7fc280100100 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28019c170 con 0x7fc2800ff740 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc2881b1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28019c330 con 0x7fc2801044f0 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc28574c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc280100100 0x7fc280197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc28574c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc280100100 0x7fc280197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59250/0 (socket says 172.21.15.45:59250) 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc28574c700 1 -- 172.21.15.45:0/3392704586 learned_addr learned my addr 172.21.15.45:0/3392704586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc285f4d700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2800ff740 0x7fc280197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.920+0000 7fc28674e700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2801044f0 0x7fc28019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc28574c700 1 -- 172.21.15.45:0/3392704586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2801044f0 msgr2=0x7fc28019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc28574c700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2801044f0 0x7fc28019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc28574c700 1 -- 172.21.15.45:0/3392704586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2800ff740 msgr2=0x7fc280197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc28574c700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2800ff740 0x7fc280197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc28574c700 1 -- 172.21.15.45:0/3392704586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27000a700 con 0x7fc280100100 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc28674e700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2801044f0 0x7fc28019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc285f4d700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2800ff740 0x7fc280197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc28574c700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc280100100 0x7fc280197580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc27400d950 tx=0x7fc27400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc27effd700 1 -- 172.21.15.45:0/3392704586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400f890 con 0x7fc280100100 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc27effd700 1 -- 172.21.15.45:0/3392704586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27400b6e0 con 0x7fc280100100 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28019c610 con 0x7fc280100100 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc27effd700 1 -- 172.21.15.45:0/3392704586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400e3b0 con 0x7fc280100100 2023-09-23T09:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.921+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc280194d00 con 0x7fc280100100 2023-09-23T09:28:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.922+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28004eb20 con 0x7fc280100100 2023-09-23T09:28:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.923+0000 7fc27effd700 1 -- 172.21.15.45:0/3392704586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fc2740105d0 con 0x7fc280100100 2023-09-23T09:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.924+0000 7fc27effd700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2680a16b0 0x7fc2680a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.924+0000 7fc285f4d700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2680a16b0 0x7fc2680a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.925+0000 7fc27effd700 1 -- 172.21.15.45:0/3392704586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc2740ccf30 con 0x7fc280100100 2023-09-23T09:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.925+0000 7fc285f4d700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2680a16b0 0x7fc2680a3b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c6d0 tx=0x7fc27001f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:22.929+0000 7fc27effd700 1 -- 172.21.15.45:0/3392704586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc274096570 con 0x7fc280100100 2023-09-23T09:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.158+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fc2801983a0 con 0x7fc280100100 2023-09-23T09:28:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.158+0000 7fc27effd700 1 -- 172.21.15.45:0/3392704586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc274020020 con 0x7fc280100100 2023-09-23T09:28:23.160 INFO:teuthology.orchestra.run.smithi045.stdout:249108103198 2023-09-23T09:28:23.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2680a16b0 msgr2=0x7fc2680a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:23.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2680a16b0 0x7fc2680a3b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c6d0 tx=0x7fc27001f040 comp rx=0 tx=0).stop 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc280100100 msgr2=0x7fc280197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc280100100 0x7fc280197580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc27400d950 tx=0x7fc27400dc60 comp rx=0 tx=0).stop 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 shutdown_connections 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2680a16b0 0x7fc2680a3b70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2801044f0 0x7fc28019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2800ff740 0x7fc280197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 --2- 172.21.15.45:0/3392704586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc280100100 0x7fc280197580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 >> 172.21.15.45:0/3392704586 conn(0x7fc280076260 msgr2=0x7fc2800fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 shutdown_connections 2023-09-23T09:28:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:23.161+0000 7fc2881b1700 1 -- 172.21.15.45:0/3392704586 wait complete. 2023-09-23T09:28:23.374 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:23.443 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:23 smithi045 ceph-mon[98627]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:23.443 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:23 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3392704586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-23T09:28:23.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:23 smithi098 ceph-mon[102386]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:23.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:23 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3392704586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-23T09:28:23.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:23 smithi123 ceph-mon[103577]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-23T09:28:23.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:23 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3392704586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-23T09:28:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:24.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103196 got 249108103198 for osd.8 2023-09-23T09:28:24.351 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/677464114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec102340 msgr2=0x7f2eec10cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/677464114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec102340 0x7f2eec10cd20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400b0d0 tx=0x7f2ee400b3e0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/677464114 shutdown_connections 2023-09-23T09:28:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/677464114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec102340 0x7f2eec10cd20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/677464114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec1000a0 0x7f2eec100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/677464114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2eec101980 0x7f2eec101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/677464114 >> 172.21.15.45:0/677464114 conn(0x7f2eec076e10 msgr2=0x7f2eec077220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/677464114 shutdown_connections 2023-09-23T09:28:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.391+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/677464114 wait complete. 2023-09-23T09:28:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.392+0000 7f2ef0bf1700 1 Processor -- start 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.392+0000 7f2ef0bf1700 1 -- start start 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ef0bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec1000a0 0x7f2eec102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ef0bf1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec101980 0x7f2eec1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ef0bf1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2eec102340 0x7f2eec103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ef0bf1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec1a5560 con 0x7f2eec101980 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ef0bf1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec1a56d0 con 0x7f2eec102340 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ef0bf1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec1a5890 con 0x7f2eec1000a0 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec101980 0x7f2eec1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec101980 0x7f2eec1030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59276/0 (socket says 172.21.15.45:59276) 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 -- 172.21.15.45:0/3341631037 learned_addr learned my addr 172.21.15.45:0/3341631037 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2eea59c700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec1000a0 0x7f2eec102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2eead9d700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2eec102340 0x7f2eec103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 -- 172.21.15.45:0/3341631037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec1000a0 msgr2=0x7f2eec102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec1000a0 0x7f2eec102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 -- 172.21.15.45:0/3341631037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2eec102340 msgr2=0x7f2eec103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2eec102340 0x7f2eec103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 -- 172.21.15.45:0/3341631037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4009d00 con 0x7f2eec101980 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2eead9d700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2eec102340 0x7f2eec103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.393+0000 7f2ee9d9b700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec101980 0x7f2eec1030a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9a0 tx=0x7f2ee000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.394+0000 7f2edb7fe700 1 -- 172.21.15.45:0/3341631037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0009790 con 0x7f2eec101980 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.394+0000 7f2edb7fe700 1 -- 172.21.15.45:0/3341631037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee000cbd0 con 0x7f2eec101980 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.394+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec1a5b70 con 0x7f2eec101980 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.394+0000 7f2edb7fe700 1 -- 172.21.15.45:0/3341631037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0018680 con 0x7f2eec101980 2023-09-23T09:28:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.394+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec1a6070 con 0x7f2eec101980 2023-09-23T09:28:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.395+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eec04eb20 con 0x7f2eec101980 2023-09-23T09:28:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.396+0000 7f2edb7fe700 1 -- 172.21.15.45:0/3341631037 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f2ee000cd40 con 0x7f2eec101980 2023-09-23T09:28:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.397+0000 7f2edb7fe700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ed40a1630 0x7f2ed40a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.397+0000 7f2eea59c700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ed40a1630 0x7f2ed40a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.397+0000 7f2edb7fe700 1 -- 172.21.15.45:0/3341631037 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2ee0014070 con 0x7f2eec101980 2023-09-23T09:28:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.398+0000 7f2eea59c700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ed40a1630 0x7f2ed40a3af0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009c60 tx=0x7f2edc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:25.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.402+0000 7f2edb7fe700 1 -- 172.21.15.45:0/3341631037 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2ee0096a30 con 0x7f2eec101980 2023-09-23T09:28:25.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.648+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f2eec10fe80 con 0x7f2eec101980 2023-09-23T09:28:25.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.648+0000 7f2edb7fe700 1 -- 172.21.15.45:0/3341631037 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ee009a050 con 0x7f2eec101980 2023-09-23T09:28:25.650 INFO:teuthology.orchestra.run.smithi045.stdout:283467841563 2023-09-23T09:28:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:25 smithi045 ceph-mon[98627]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-23T09:28:25.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ed40a1630 msgr2=0x7f2ed40a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ed40a1630 0x7f2ed40a3af0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009c60 tx=0x7f2edc009260 comp rx=0 tx=0).stop 2023-09-23T09:28:25.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec101980 msgr2=0x7f2eec1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec101980 0x7f2eec1030a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9a0 tx=0x7f2ee000ecb0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 shutdown_connections 2023-09-23T09:28:25.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2eec102340 0x7f2eec103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ed40a1630 0x7f2ed40a3af0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec1000a0 0x7f2eec102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 --2- 172.21.15.45:0/3341631037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec101980 0x7f2eec1030a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 >> 172.21.15.45:0/3341631037 conn(0x7f2eec076e10 msgr2=0x7f2eec0691f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:25.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 shutdown_connections 2023-09-23T09:28:25.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.651+0000 7f2ef0bf1700 1 -- 172.21.15.45:0/3341631037 wait complete. 2023-09-23T09:28:25.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:25 smithi098 ceph-mon[102386]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-23T09:28:25.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:25 smithi123 ceph-mon[103577]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 -- 172.21.15.45:0/730617663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 msgr2=0x7faf48101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 --2- 172.21.15.45:0/730617663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48101480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faf38009990 tx=0x7faf38009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 -- 172.21.15.45:0/730617663 shutdown_connections 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 --2- 172.21.15.45:0/730617663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf48106180 0x7faf4810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 --2- 172.21.15.45:0/730617663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48101480 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 --2- 172.21.15.45:0/730617663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101a50 0x7faf48105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 -- 172.21.15.45:0/730617663 >> 172.21.15.45:0/730617663 conn(0x7faf480fc910 msgr2=0x7faf480fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 -- 172.21.15.45:0/730617663 shutdown_connections 2023-09-23T09:28:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.151+0000 7faf4cae4700 1 -- 172.21.15.45:0/730617663 wait complete. 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 Processor -- start 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 -- start start 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101a50 0x7faf48073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf48106180 0x7faf48073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481a53a0 con 0x7faf481010a0 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481a5510 con 0x7faf48106180 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4cae4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481a56d0 con 0x7faf48101a50 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59260/0 (socket says 172.21.15.45:59260) 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.152+0000 7faf4659c700 1 -- 172.21.15.45:0/780094814 learned_addr learned my addr 172.21.15.45:0/780094814 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf45d9b700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101a50 0x7faf48073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4659c700 1 -- 172.21.15.45:0/780094814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101a50 msgr2=0x7faf48073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4659c700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101a50 0x7faf48073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4659c700 1 -- 172.21.15.45:0/780094814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf48106180 msgr2=0x7faf48073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf46d9d700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf48106180 0x7faf48073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4659c700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf48106180 0x7faf48073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4659c700 1 -- 172.21.15.45:0/780094814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380095f0 con 0x7faf481010a0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4659c700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48073120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf38006010 tx=0x7faf3800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf377fe700 1 -- 172.21.15.45:0/780094814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3801d070 con 0x7faf481010a0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf481a5950 con 0x7faf481010a0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf377fe700 1 -- 172.21.15.45:0/780094814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf3800fca0 con 0x7faf481010a0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf377fe700 1 -- 172.21.15.45:0/780094814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380176e0 con 0x7faf481010a0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.153+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf481a5dc0 con 0x7faf481010a0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.154+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4804eb20 con 0x7faf481010a0 2023-09-23T09:28:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.155+0000 7faf377fe700 1 -- 172.21.15.45:0/780094814 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7faf38017840 con 0x7faf481010a0 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.156+0000 7faf377fe700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7faf300a16b0 0x7faf300a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.156+0000 7faf377fe700 1 -- 172.21.15.45:0/780094814 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf380ce480 con 0x7faf481010a0 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.156+0000 7faf45d9b700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7faf300a16b0 0x7faf300a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stdout:141733920816 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.156+0000 7faf45d9b700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7faf300a16b0 0x7faf300a3b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faf3c005fd0 tx=0x7faf3c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.159+0000 7faf377fe700 1 -- 172.21.15.45:0/780094814 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7faf38097b50 con 0x7faf481010a0 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.383+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faf48074650 con 0x7faf481010a0 2023-09-23T09:28:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.384+0000 7faf377fe700 1 -- 172.21.15.45:0/780094814 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf3809b170 con 0x7faf481010a0 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7faf300a16b0 msgr2=0x7faf300a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7faf300a16b0 0x7faf300a3b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faf3c005fd0 tx=0x7faf3c00a560 comp rx=0 tx=0).stop 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 msgr2=0x7faf48073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48073120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf38006010 tx=0x7faf3800f7b0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 shutdown_connections 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf48106180 0x7faf48073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf481010a0 0x7faf48073120 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7faf300a16b0 0x7faf300a3b70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 --2- 172.21.15.45:0/780094814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101a50 0x7faf48073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 >> 172.21.15.45:0/780094814 conn(0x7faf480fc910 msgr2=0x7faf480fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 shutdown_connections 2023-09-23T09:28:25.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.386+0000 7faf4cae4700 1 -- 172.21.15.45:0/780094814 wait complete. 2023-09-23T09:28:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.461+0000 7f2af01cb700 1 -- 172.21.15.45:0/1422969374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae81010b0 msgr2=0x7f2ae8101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.461+0000 7f2af01cb700 1 --2- 172.21.15.45:0/1422969374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae81010b0 0x7f2ae8101530 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80099c0 tx=0x7f2ad8009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.461+0000 7f2af01cb700 1 -- 172.21.15.45:0/1422969374 shutdown_connections 2023-09-23T09:28:26.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.461+0000 7f2af01cb700 1 --2- 172.21.15.45:0/1422969374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ae8101a70 0x7f2ae810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.461+0000 7f2af01cb700 1 --2- 172.21.15.45:0/1422969374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae8107030 0x7f2ae8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.461+0000 7f2af01cb700 1 --2- 172.21.15.45:0/1422969374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae81010b0 0x7f2ae8101530 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80099c0 tx=0x7f2ad8009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.461+0000 7f2af01cb700 1 -- 172.21.15.45:0/1422969374 >> 172.21.15.45:0/1422969374 conn(0x7f2ae8078e40 msgr2=0x7f2ae8079250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:26.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.462+0000 7f2af01cb700 1 -- 172.21.15.45:0/1422969374 shutdown_connections 2023-09-23T09:28:26.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.462+0000 7f2af01cb700 1 -- 172.21.15.45:0/1422969374 wait complete. 2023-09-23T09:28:26.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 Processor -- start 2023-09-23T09:28:26.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 -- start start 2023-09-23T09:28:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae8101a70 0x7f2ae8073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae8107030 0x7f2ae8073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ae8073ba0 0x7f2ae81a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae81a5a00 con 0x7f2ae8107030 2023-09-23T09:28:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae81a5b70 con 0x7f2ae8073ba0 2023-09-23T09:28:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2af01cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae81a5d30 con 0x7f2ae8101a70 2023-09-23T09:28:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2aed766700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae8107030 0x7f2ae8073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2aed766700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae8107030 0x7f2ae8073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59302/0 (socket says 172.21.15.45:59302) 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2aed766700 1 -- 172.21.15.45:0/3729019951 learned_addr learned my addr 172.21.15.45:0/3729019951 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2aedf67700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae8101a70 0x7f2ae8073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.463+0000 7f2aee768700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ae8073ba0 0x7f2ae81a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2aed766700 1 -- 172.21.15.45:0/3729019951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae8101a70 msgr2=0x7f2ae8073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2aed766700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae8101a70 0x7f2ae8073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2aed766700 1 -- 172.21.15.45:0/3729019951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ae8073ba0 msgr2=0x7f2ae81a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2aed766700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ae8073ba0 0x7f2ae81a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2aed766700 1 -- 172.21.15.45:0/3729019951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80095f0 con 0x7f2ae8107030 2023-09-23T09:28:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2aed766700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae8107030 0x7f2ae8073660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009990 tx=0x7f2ad8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2adeffd700 1 -- 172.21.15.45:0/3729019951 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad801d070 con 0x7f2ae8107030 2023-09-23T09:28:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae81a5f50 con 0x7f2ae8107030 2023-09-23T09:28:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2adeffd700 1 -- 172.21.15.45:0/3729019951 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ad80059a0 con 0x7f2ae8107030 2023-09-23T09:28:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2adeffd700 1 -- 172.21.15.45:0/3729019951 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800fcf0 con 0x7f2ae8107030 2023-09-23T09:28:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.464+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae81a6360 con 0x7f2ae8107030 2023-09-23T09:28:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.465+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae804eb20 con 0x7f2ae8107030 2023-09-23T09:28:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.467+0000 7f2adeffd700 1 -- 172.21.15.45:0/3729019951 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f2ad800f460 con 0x7f2ae8107030 2023-09-23T09:28:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.467+0000 7f2adeffd700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ad40a1630 0x7f2ad40a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.468+0000 7f2aedf67700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ad40a1630 0x7f2ad40a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.468+0000 7f2adeffd700 1 -- 172.21.15.45:0/3729019951 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2ad80cddc0 con 0x7f2ae8107030 2023-09-23T09:28:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.468+0000 7f2aedf67700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ad40a1630 0x7f2ad40a3af0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4005fd0 tx=0x7f2ae400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.475+0000 7f2adeffd700 1 -- 172.21.15.45:0/3729019951 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2ad8097490 con 0x7f2ae8107030 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 -- 172.21.15.45:0/3031324241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 msgr2=0x7f1410107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 --2- 172.21.15.45:0/3031324241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f1410107390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f140c009990 tx=0x7f140c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 -- 172.21.15.45:0/3031324241 shutdown_connections 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 --2- 172.21.15.45:0/3031324241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1410069300 0x7f141010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 --2- 172.21.15.45:0/3031324241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f1410107390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 --2- 172.21.15.45:0/3031324241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1410068940 0x7f1410068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 -- 172.21.15.45:0/3031324241 >> 172.21.15.45:0/3031324241 conn(0x7f1410076e20 msgr2=0x7f1410077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 -- 172.21.15.45:0/3031324241 shutdown_connections 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.873+0000 7f1417538700 1 -- 172.21.15.45:0/3031324241 wait complete. 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 Processor -- start 2023-09-23T09:28:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 -- start start 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1410068940 0x7f1410196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1410069300 0x7f1410197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f141019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141019bde0 con 0x7f1410106fb0 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141019bf50 con 0x7f1410069300 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1417538700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141019c110 con 0x7f1410068940 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1415ad5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f141019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1415ad5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f141019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59290/0 (socket says 172.21.15.45:59290) 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f1415ad5700 1 -- 172.21.15.45:0/115609509 learned_addr learned my addr 172.21.15.45:0/115609509 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.874+0000 7f14152d4700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1410068940 0x7f1410196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1414ad3700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1410069300 0x7f1410197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1415ad5700 1 -- 172.21.15.45:0/115609509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1410068940 msgr2=0x7f1410196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1415ad5700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1410068940 0x7f1410196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1415ad5700 1 -- 172.21.15.45:0/115609509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1410069300 msgr2=0x7f1410197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1415ad5700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1410069300 0x7f1410197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1415ad5700 1 -- 172.21.15.45:0/115609509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c0095f0 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1414ad3700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1410069300 0x7f1410197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1415ad5700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f141019b7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1408007f10 tx=0x7f140800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f14067fc700 1 -- 172.21.15.45:0/115609509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408017070 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f14067fc700 1 -- 172.21.15.45:0/115609509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f140800f040 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141019c3f0 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f14067fc700 1 -- 172.21.15.45:0/115609509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408005740 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.875+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14101a3c50 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.876+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141004eb20 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.877+0000 7f14067fc700 1 -- 172.21.15.45:0/115609509 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f140800dc60 con 0x7f1410106fb0 2023-09-23T09:28:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.878+0000 7f14067fc700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13fc0a1630 0x7f13fc0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.878+0000 7f14152d4700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13fc0a1630 0x7f13fc0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.878+0000 7f14067fc700 1 -- 172.21.15.45:0/115609509 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f14080ccd70 con 0x7f1410106fb0 2023-09-23T09:28:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.878+0000 7f14152d4700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13fc0a1630 0x7f13fc0a3af0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f140c00b5c0 tx=0x7f140c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:25.882+0000 7f14067fc700 1 -- 172.21.15.45:0/115609509 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f14080962b0 con 0x7f1410106fb0 2023-09-23T09:28:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.100+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1410110e80 con 0x7f1410106fb0 2023-09-23T09:28:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.100+0000 7f14067fc700 1 -- 172.21.15.45:0/115609509 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14080998d0 con 0x7f1410106fb0 2023-09-23T09:28:26.493 INFO:teuthology.orchestra.run.smithi045.stdout:55834574908 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13fc0a1630 msgr2=0x7f13fc0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13fc0a1630 0x7f13fc0a3af0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f140c00b5c0 tx=0x7f140c005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 msgr2=0x7f141019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f141019b7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1408007f10 tx=0x7f140800d3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 shutdown_connections 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1410106fb0 0x7f141019b7d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f13fc0a1630 0x7f13fc0a3af0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1410068940 0x7f1410196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 --2- 172.21.15.45:0/115609509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1410069300 0x7f1410197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 >> 172.21.15.45:0/115609509 conn(0x7f1410076e20 msgr2=0x7f14100fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 shutdown_connections 2023-09-23T09:28:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.102+0000 7f1417538700 1 -- 172.21.15.45:0/115609509 wait complete. 2023-09-23T09:28:26.495 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841560 got 283467841563 for osd.9 2023-09-23T09:28:26.495 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 -- 172.21.15.45:0/2406422976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0069300 msgr2=0x7f7df010dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2406422976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0069300 0x7f7df010dc60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b0a0 tx=0x7f7de800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 -- 172.21.15.45:0/2406422976 shutdown_connections 2023-09-23T09:28:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2406422976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0069300 0x7f7df010dc60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2406422976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df0106fb0 0x7f7df0107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2406422976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df0068940 0x7f7df0068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 -- 172.21.15.45:0/2406422976 >> 172.21.15.45:0/2406422976 conn(0x7f7df0076e20 msgr2=0x7f7df0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.651+0000 7f7df5c92700 1 -- 172.21.15.45:0/2406422976 shutdown_connections 2023-09-23T09:28:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.652+0000 7f7df5c92700 1 -- 172.21.15.45:0/2406422976 wait complete. 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.652+0000 7f7df5c92700 1 Processor -- start 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.652+0000 7f7df5c92700 1 -- start start 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7df5c92700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df0068940 0x7f7df00ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7df5c92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df0069300 0x7f7df0100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7df5c92700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0106fb0 0x7f7df0100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7df5c92700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df01a5480 con 0x7f7df0106fb0 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7df5c92700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df01a55f0 con 0x7f7df0068940 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7df5c92700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df01a57b0 con 0x7f7df0069300 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0106fb0 0x7f7df0100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0106fb0 0x7f7df0100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59316/0 (socket says 172.21.15.45:59316) 2023-09-23T09:28:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 -- 172.21.15.45:0/2661419733 learned_addr learned my addr 172.21.15.45:0/2661419733 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deeffd700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df0069300 0x7f7df0100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7def7fe700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df0068940 0x7f7df00ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 -- 172.21.15.45:0/2661419733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df0069300 msgr2=0x7f7df0100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df0069300 0x7f7df0100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 -- 172.21.15.45:0/2661419733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df0068940 msgr2=0x7f7df00ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df0068940 0x7f7df00ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 -- 172.21.15.45:0/2661419733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de8009d00 con 0x7f7df0106fb0 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7deffff700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0106fb0 0x7f7df0100890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b780 tx=0x7f7de8007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7decff9700 1 -- 172.21.15.45:0/2661419733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de800e040 con 0x7f7df0106fb0 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df01a5a30 con 0x7f7df0106fb0 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7decff9700 1 -- 172.21.15.45:0/2661419733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7de801d470 con 0x7f7df0106fb0 2023-09-23T09:28:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.653+0000 7f7decff9700 1 -- 172.21.15.45:0/2661419733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de80052a0 con 0x7f7df0106fb0 2023-09-23T09:28:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.654+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df01a5ef0 con 0x7f7df0106fb0 2023-09-23T09:28:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.655+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df004eb20 con 0x7f7df0106fb0 2023-09-23T09:28:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.656+0000 7f7decff9700 1 -- 172.21.15.45:0/2661419733 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f7de801d5e0 con 0x7f7df0106fb0 2023-09-23T09:28:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.656+0000 7f7decff9700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7dd80a1630 0x7f7dd80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.656+0000 7f7def7fe700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7dd80a1630 0x7f7dd80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.656+0000 7f7decff9700 1 -- 172.21.15.45:0/2661419733 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7de80ce240 con 0x7f7df0106fb0 2023-09-23T09:28:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.657+0000 7f7def7fe700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7dd80a1630 0x7f7dd80a3af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007b60 tx=0x7f7de00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:26.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.661+0000 7f7decff9700 1 -- 172.21.15.45:0/2661419733 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7de8097910 con 0x7f7df0106fb0 2023-09-23T09:28:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.701+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f2ae8110e40 con 0x7f2ae8107030 2023-09-23T09:28:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.702+0000 7f2adeffd700 1 -- 172.21.15.45:0/3729019951 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f2ad809aab0 con 0x7f2ae8107030 2023-09-23T09:28:26.703 INFO:teuthology.orchestra.run.smithi045.stdout:335007449107 2023-09-23T09:28:26.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ad40a1630 msgr2=0x7f2ad40a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ad40a1630 0x7f2ad40a3af0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4005fd0 tx=0x7f2ae400af20 comp rx=0 tx=0).stop 2023-09-23T09:28:26.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae8107030 msgr2=0x7f2ae8073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae8107030 0x7f2ae8073660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009990 tx=0x7f2ad8005e80 comp rx=0 tx=0).stop 2023-09-23T09:28:26.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 shutdown_connections 2023-09-23T09:28:26.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ae8073ba0 0x7f2ae81a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f2ad40a1630 0x7f2ad40a3af0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae8101a70 0x7f2ae8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 --2- 172.21.15.45:0/3729019951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae8107030 0x7f2ae8073660 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 >> 172.21.15.45:0/3729019951 conn(0x7f2ae8078e40 msgr2=0x7f2ae80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 shutdown_connections 2023-09-23T09:28:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.704+0000 7f2af01cb700 1 -- 172.21.15.45:0/3729019951 wait complete. 2023-09-23T09:28:26.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:26 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/780094814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-23T09:28:26.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:26 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3341631037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-23T09:28:26.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:26 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/115609509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-23T09:28:26.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:26 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/780094814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-23T09:28:26.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:26 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3341631037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-23T09:28:26.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:26 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/115609509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-23T09:28:26.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:26 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/780094814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-23T09:28:26.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:26 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3341631037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-23T09:28:26.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:26 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/115609509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-23T09:28:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.880+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7df0110e80 con 0x7f7df0106fb0 2023-09-23T09:28:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.880+0000 7f7decff9700 1 -- 172.21.15.45:0/2661419733 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7de809af30 con 0x7f7df0106fb0 2023-09-23T09:28:26.882 INFO:teuthology.orchestra.run.smithi045.stdout:107374182452 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7dd80a1630 msgr2=0x7f7dd80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7dd80a1630 0x7f7dd80a3af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007b60 tx=0x7f7de00073d0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0106fb0 msgr2=0x7f7df0100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0106fb0 0x7f7df0100890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b780 tx=0x7f7de8007990 comp rx=0 tx=0).stop 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 shutdown_connections 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df0106fb0 0x7f7df0100890 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df0068940 0x7f7df00ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f7dd80a1630 0x7f7dd80a3af0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 --2- 172.21.15.45:0/2661419733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df0069300 0x7f7df0100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 >> 172.21.15.45:0/2661419733 conn(0x7f7df0076e20 msgr2=0x7f7df00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 shutdown_connections 2023-09-23T09:28:26.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:26.882+0000 7f7df5c92700 1 -- 172.21.15.45:0/2661419733 wait complete. 2023-09-23T09:28:27.253 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574906 got 55834574908 for osd.1 2023-09-23T09:28:27.254 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:27.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:27 smithi045 ceph-mon[98627]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-09-23T09:28:27.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:27 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3729019951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-23T09:28:27.438 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:27 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2661419733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-23T09:28:27.514 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182450 got 107374182452 for osd.3 2023-09-23T09:28:27.514 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:27 smithi098 ceph-mon[102386]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-09-23T09:28:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:27 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3729019951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-23T09:28:27.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:27 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2661419733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-23T09:28:27.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:27 smithi123 ceph-mon[103577]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-09-23T09:28:27.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:27 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3729019951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-23T09:28:27.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:27 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2661419733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-23T09:28:27.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920814 got 141733920816 for osd.4 2023-09-23T09:28:27.964 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:28.117 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449105 got 335007449107 for osd.11 2023-09-23T09:28:28.117 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.704+0000 7f5efbc65700 1 -- 172.21.15.45:0/2990504405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef4101a70 msgr2=0x7f5ef410dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.704+0000 7f5efbc65700 1 --2- 172.21.15.45:0/2990504405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef4101a70 0x7f5ef410dcf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009f90 tx=0x7f5eec00b330 comp rx=0 tx=0).stop 2023-09-23T09:28:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.704+0000 7f5efbc65700 1 -- 172.21.15.45:0/2990504405 shutdown_connections 2023-09-23T09:28:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.704+0000 7f5efbc65700 1 --2- 172.21.15.45:0/2990504405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef4101a70 0x7f5ef410dcf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.704+0000 7f5efbc65700 1 --2- 172.21.15.45:0/2990504405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ef4106f60 0x7f5ef4107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.704+0000 7f5efbc65700 1 --2- 172.21.15.45:0/2990504405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef41010b0 0x7f5ef4101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.704+0000 7f5efbc65700 1 -- 172.21.15.45:0/2990504405 >> 172.21.15.45:0/2990504405 conn(0x7f5ef4078e10 msgr2=0x7f5ef4079220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.705+0000 7f5efbc65700 1 -- 172.21.15.45:0/2990504405 shutdown_connections 2023-09-23T09:28:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.705+0000 7f5efbc65700 1 -- 172.21.15.45:0/2990504405 wait complete. 2023-09-23T09:28:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.705+0000 7f5efbc65700 1 Processor -- start 2023-09-23T09:28:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.705+0000 7f5efbc65700 1 -- start start 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5efbc65700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef41010b0 0x7f5ef4196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5efbc65700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ef4101a70 0x7f5ef41974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5efbc65700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef4106f60 0x7f5ef419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5efbc65700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef419bf70 con 0x7f5ef41010b0 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5efbc65700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef419c0e0 con 0x7f5ef4101a70 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5efbc65700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef419c2a0 con 0x7f5ef4106f60 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef41010b0 0x7f5ef4196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef41010b0 0x7f5ef4196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59326/0 (socket says 172.21.15.45:59326) 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 -- 172.21.15.45:0/903765406 learned_addr learned my addr 172.21.15.45:0/903765406 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9200700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ef4101a70 0x7f5ef41974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5efa202700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef4106f60 0x7f5ef419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 -- 172.21.15.45:0/903765406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef4106f60 msgr2=0x7f5ef419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef4106f60 0x7f5ef419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 -- 172.21.15.45:0/903765406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ef4101a70 msgr2=0x7f5ef41974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ef4101a70 0x7f5ef41974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 -- 172.21.15.45:0/903765406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec009c40 con 0x7f5ef41010b0 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.706+0000 7f5ef9a01700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef41010b0 0x7f5ef4196fb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000e9a0 tx=0x7f5ef000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.707+0000 7f5eeaffd700 1 -- 172.21.15.45:0/903765406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0009790 con 0x7f5ef41010b0 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.707+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef419c580 con 0x7f5ef41010b0 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.707+0000 7f5eeaffd700 1 -- 172.21.15.45:0/903765406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ef000cbd0 con 0x7f5ef41010b0 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.707+0000 7f5eeaffd700 1 -- 172.21.15.45:0/903765406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000eee0 con 0x7f5ef41010b0 2023-09-23T09:28:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.707+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef41a3de0 con 0x7f5ef41010b0 2023-09-23T09:28:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.708+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef404eb20 con 0x7f5ef41010b0 2023-09-23T09:28:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.709+0000 7f5eeaffd700 1 -- 172.21.15.45:0/903765406 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f5ef000cd40 con 0x7f5ef41010b0 2023-09-23T09:28:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.709+0000 7f5eeaffd700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ee00a1630 0x7f5ee00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.710+0000 7f5eeaffd700 1 -- 172.21.15.45:0/903765406 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ef0014070 con 0x7f5ef41010b0 2023-09-23T09:28:28.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.710+0000 7f5ef9200700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ee00a1630 0x7f5ee00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:28.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.710+0000 7f5ef9200700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ee00a1630 0x7f5ee00a3af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ef41985d0 tx=0x7f5ee4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:28.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.713+0000 7f5eeaffd700 1 -- 172.21.15.45:0/903765406 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5ef0096c00 con 0x7f5ef41010b0 2023-09-23T09:28:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 -- 172.21.15.45:0/2109099436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f4106fb0 msgr2=0x7f63f4107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2109099436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f4106fb0 0x7f63f4107390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009990 tx=0x7f63f0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 -- 172.21.15.45:0/2109099436 shutdown_connections 2023-09-23T09:28:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2109099436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f4069300 0x7f63f410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2109099436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f4106fb0 0x7f63f4107390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2109099436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4068940 0x7f63f4068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 -- 172.21.15.45:0/2109099436 >> 172.21.15.45:0/2109099436 conn(0x7f63f4076e20 msgr2=0x7f63f4077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 -- 172.21.15.45:0/2109099436 shutdown_connections 2023-09-23T09:28:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.806+0000 7f63fba9e700 1 -- 172.21.15.45:0/2109099436 wait complete. 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 Processor -- start 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 -- start start 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f4068940 0x7f63f4196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f4069300 0x7f63f4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4106fb0 0x7f63f419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f419beb0 con 0x7f63f4068940 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f419c020 con 0x7f63f4069300 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.807+0000 7f63fba9e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f419c1e0 con 0x7f63f4106fb0 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4106fb0 0x7f63f419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4106fb0 0x7f63f419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:33218/0 (socket says 172.21.15.45:33218) 2023-09-23T09:28:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 -- 172.21.15.45:0/2891610201 learned_addr learned my addr 172.21.15.45:0/2891610201 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63f9039700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f4069300 0x7f63f4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 -- 172.21.15.45:0/2891610201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f4069300 msgr2=0x7f63f4197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f4069300 0x7f63f4197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 -- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f4068940 msgr2=0x7f63f4196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f4068940 0x7f63f4196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 -- 172.21.15.45:0/2891610201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00095f0 con 0x7f63f4106fb0 2023-09-23T09:28:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63fa03b700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4106fb0 0x7f63f419b8a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00efc0 tx=0x7f63ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63eaffd700 1 -- 172.21.15.45:0/2891610201 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec018070 con 0x7f63f4106fb0 2023-09-23T09:28:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.808+0000 7f63eaffd700 1 -- 172.21.15.45:0/2891610201 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63ec00f040 con 0x7f63f4106fb0 2023-09-23T09:28:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.809+0000 7f63eaffd700 1 -- 172.21.15.45:0/2891610201 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec00ccc0 con 0x7f63f4106fb0 2023-09-23T09:28:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.809+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f419c460 con 0x7f63f4106fb0 2023-09-23T09:28:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.809+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f406aa10 con 0x7f63f4106fb0 2023-09-23T09:28:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.810+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f404eb20 con 0x7f63f4106fb0 2023-09-23T09:28:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.811+0000 7f63eaffd700 1 -- 172.21.15.45:0/2891610201 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f63ec003a00 con 0x7f63f4106fb0 2023-09-23T09:28:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.812+0000 7f63eaffd700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63e00a1630 0x7f63e00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.812+0000 7f63f983a700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63e00a1630 0x7f63e00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.812+0000 7f63eaffd700 1 -- 172.21.15.45:0/2891610201 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63ec0ccb50 con 0x7f63f4106fb0 2023-09-23T09:28:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.812+0000 7f63f983a700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63e00a1630 0x7f63e00a3af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63f00095c0 tx=0x7f63f0009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.815+0000 7f63eaffd700 1 -- 172.21.15.45:0/2891610201 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f63ec096120 con 0x7f63f4106fb0 2023-09-23T09:28:28.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.924+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5ef4198310 con 0x7f5ef41010b0 2023-09-23T09:28:28.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.924+0000 7f5eeaffd700 1 -- 172.21.15.45:0/903765406 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ef009a220 con 0x7f5ef41010b0 2023-09-23T09:28:28.926 INFO:teuthology.orchestra.run.smithi045.stdout:193273528360 2023-09-23T09:28:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ee00a1630 msgr2=0x7f5ee00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ee00a1630 0x7f5ee00a3af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ef41985d0 tx=0x7f5ee4009310 comp rx=0 tx=0).stop 2023-09-23T09:28:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef41010b0 msgr2=0x7f5ef4196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef41010b0 0x7f5ef4196fb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000e9a0 tx=0x7f5ef000ed60 comp rx=0 tx=0).stop 2023-09-23T09:28:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 shutdown_connections 2023-09-23T09:28:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5ee00a1630 0x7f5ee00a3af0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef4106f60 0x7f5ef419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ef41010b0 0x7f5ef4196fb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 --2- 172.21.15.45:0/903765406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ef4101a70 0x7f5ef41974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 >> 172.21.15.45:0/903765406 conn(0x7f5ef4078e10 msgr2=0x7f5ef4107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 shutdown_connections 2023-09-23T09:28:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:28.926+0000 7f5efbc65700 1 -- 172.21.15.45:0/903765406 wait complete. 2023-09-23T09:28:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.031+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f63f4110de0 con 0x7f63f4106fb0 2023-09-23T09:28:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.032+0000 7f63eaffd700 1 -- 172.21.15.45:0/2891610201 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63ec099740 con 0x7f63f4106fb0 2023-09-23T09:28:29.033 INFO:teuthology.orchestra.run.smithi045.stdout:34359738433 2023-09-23T09:28:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63e00a1630 msgr2=0x7f63e00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63e00a1630 0x7f63e00a3af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63f00095c0 tx=0x7f63f0009530 comp rx=0 tx=0).stop 2023-09-23T09:28:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4106fb0 msgr2=0x7f63f419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4106fb0 0x7f63f419b8a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00efc0 tx=0x7f63ec00c3a0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 shutdown_connections 2023-09-23T09:28:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63e00a1630 0x7f63e00a3af0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63f4106fb0 0x7f63f419b8a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f4068940 0x7f63f4196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 --2- 172.21.15.45:0/2891610201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f4069300 0x7f63f4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 >> 172.21.15.45:0/2891610201 conn(0x7f63f4076e20 msgr2=0x7f63f40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 shutdown_connections 2023-09-23T09:28:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.034+0000 7f63fba9e700 1 -- 172.21.15.45:0/2891610201 wait complete. 2023-09-23T09:28:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 -- 172.21.15.45:0/2791123963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c068940 msgr2=0x7fcb5c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 --2- 172.21.15.45:0/2791123963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c068940 0x7fcb5c068dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009990 tx=0x7fcb50009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 -- 172.21.15.45:0/2791123963 shutdown_connections 2023-09-23T09:28:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 --2- 172.21.15.45:0/2791123963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb5c069300 0x7fcb5c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 --2- 172.21.15.45:0/2791123963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb5c106fb0 0x7fcb5c107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 --2- 172.21.15.45:0/2791123963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c068940 0x7fcb5c068dc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 -- 172.21.15.45:0/2791123963 >> 172.21.15.45:0/2791123963 conn(0x7fcb5c076e20 msgr2=0x7fcb5c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 -- 172.21.15.45:0/2791123963 shutdown_connections 2023-09-23T09:28:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.143+0000 7fcb632a4700 1 -- 172.21.15.45:0/2791123963 wait complete. 2023-09-23T09:28:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 Processor -- start 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 -- start start 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb5c068940 0x7fcb5c196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c069300 0x7fcb5c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb5c106fb0 0x7fcb5c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c19be30 con 0x7fcb5c069300 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c19bfa0 con 0x7fcb5c106fb0 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb632a4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c19c160 con 0x7fcb5c068940 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb6083f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c069300 0x7fcb5c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb6083f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c069300 0x7fcb5c1973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59356/0 (socket says 172.21.15.45:59356) 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb6083f700 1 -- 172.21.15.45:0/1798909167 learned_addr learned my addr 172.21.15.45:0/1798909167 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.144+0000 7fcb61040700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb5c068940 0x7fcb5c196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb61841700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb5c106fb0 0x7fcb5c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb6083f700 1 -- 172.21.15.45:0/1798909167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb5c068940 msgr2=0x7fcb5c196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb6083f700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb5c068940 0x7fcb5c196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb6083f700 1 -- 172.21.15.45:0/1798909167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb5c106fb0 msgr2=0x7fcb5c19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb6083f700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb5c106fb0 0x7fcb5c19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb6083f700 1 -- 172.21.15.45:0/1798909167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500095f0 con 0x7fcb5c069300 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb6083f700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c069300 0x7fcb5c1973b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb4e7fc700 1 -- 172.21.15.45:0/1798909167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001d070 con 0x7fcb5c069300 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c19c3e0 con 0x7fcb5c069300 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb4e7fc700 1 -- 172.21.15.45:0/1798909167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb50021410 con 0x7fcb5c069300 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb4e7fc700 1 -- 172.21.15.45:0/1798909167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50017650 con 0x7fcb5c069300 2023-09-23T09:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.145+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c0ffed0 con 0x7fcb5c069300 2023-09-23T09:28:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.146+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c04eb20 con 0x7fcb5c069300 2023-09-23T09:28:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.148+0000 7fcb4e7fc700 1 -- 172.21.15.45:0/1798909167 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fcb500217f0 con 0x7fcb5c069300 2023-09-23T09:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.148+0000 7fcb4e7fc700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb480a1630 0x7fcb480a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.148+0000 7fcb61040700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb480a1630 0x7fcb480a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.148+0000 7fcb4e7fc700 1 -- 172.21.15.45:0/1798909167 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcb500ce970 con 0x7fcb5c069300 2023-09-23T09:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.149+0000 7fcb61040700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb480a1630 0x7fcb480a3af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007bc0 tx=0x7fcb580073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:29.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.152+0000 7fcb4e7fc700 1 -- 172.21.15.45:0/1798909167 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fcb50098040 con 0x7fcb5c069300 2023-09-23T09:28:29.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.361+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcb5c110d80 con 0x7fcb5c069300 2023-09-23T09:28:29.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.361+0000 7fcb4e7fc700 1 -- 172.21.15.45:0/1798909167 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb5009b660 con 0x7fcb5c069300 2023-09-23T09:28:29.363 INFO:teuthology.orchestra.run.smithi045.stdout:214748364835 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb480a1630 msgr2=0x7fcb480a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb480a1630 0x7fcb480a3af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007bc0 tx=0x7fcb580073d0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c069300 msgr2=0x7fcb5c1973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c069300 0x7fcb5c1973b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb500047b0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 shutdown_connections 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb5c106fb0 0x7fcb5c19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcb480a1630 0x7fcb480a3af0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb5c068940 0x7fcb5c196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 --2- 172.21.15.45:0/1798909167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb5c069300 0x7fcb5c1973b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 >> 172.21.15.45:0/1798909167 conn(0x7fcb5c076e20 msgr2=0x7fcb5c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:29.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 shutdown_connections 2023-09-23T09:28:29.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.364+0000 7fcb632a4700 1 -- 172.21.15.45:0/1798909167 wait complete. 2023-09-23T09:28:29.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:29 smithi045 ceph-mon[98627]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-09-23T09:28:29.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:29 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/903765406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-23T09:28:29.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:29 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2891610201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-23T09:28:29.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:29 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1798909167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-23T09:28:29.517 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528358 got 193273528360 for osd.6 2023-09-23T09:28:29.518 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:29.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:29 smithi098 ceph-mon[102386]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-09-23T09:28:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:29 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/903765406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-23T09:28:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:29 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2891610201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-23T09:28:29.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:29 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1798909167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 -- 172.21.15.45:0/1834431036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 msgr2=0x7fd59010cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 --2- 172.21.15.45:0/1834431036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 0x7fd59010cc50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b0a0 tx=0x7fd58800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 -- 172.21.15.45:0/1834431036 shutdown_connections 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 --2- 172.21.15.45:0/1834431036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 0x7fd59010cc50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 --2- 172.21.15.45:0/1834431036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 0x7fd590100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 --2- 172.21.15.45:0/1834431036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd590100950 0x7fd5901049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 -- 172.21.15.45:0/1834431036 >> 172.21.15.45:0/1834431036 conn(0x7fd5900fb890 msgr2=0x7fd5900fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.833+0000 7fd59616e700 1 -- 172.21.15.45:0/1834431036 shutdown_connections 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.834+0000 7fd59616e700 1 -- 172.21.15.45:0/1834431036 wait complete. 2023-09-23T09:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.834+0000 7fd59616e700 1 Processor -- start 2023-09-23T09:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.834+0000 7fd59616e700 1 -- start start 2023-09-23T09:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd59616e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 0x7fd590195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd59616e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd590100950 0x7fd5901961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd59616e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 0x7fd5901966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd59616e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59019f070 con 0x7fd590105080 2023-09-23T09:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd59616e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901a9ad0 con 0x7fd5900fffa0 2023-09-23T09:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd59616e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901a9c10 con 0x7fd590100950 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 0x7fd590195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 0x7fd5901966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 0x7fd590195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:52500/0 (socket says 172.21.15.45:52500) 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58f7fe700 1 -- 172.21.15.45:0/1004413889 learned_addr learned my addr 172.21.15.45:0/1004413889 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58effd700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd590100950 0x7fd5901961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58ffff700 1 -- 172.21.15.45:0/1004413889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd590100950 msgr2=0x7fd5901961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58ffff700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd590100950 0x7fd5901961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58ffff700 1 -- 172.21.15.45:0/1004413889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 msgr2=0x7fd590195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58ffff700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 0x7fd590195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58ffff700 1 -- 172.21.15.45:0/1004413889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd588009d00 con 0x7fd590105080 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.835+0000 7fd58f7fe700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 0x7fd590195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.836+0000 7fd58ffff700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 0x7fd5901966e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b0a0 tx=0x7fd588007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.836+0000 7fd58cff9700 1 -- 172.21.15.45:0/1004413889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800e040 con 0x7fd590105080 2023-09-23T09:28:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.836+0000 7fd58cff9700 1 -- 172.21.15.45:0/1004413889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd58800be40 con 0x7fd590105080 2023-09-23T09:28:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.836+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59019f3a0 con 0x7fd590105080 2023-09-23T09:28:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.836+0000 7fd58cff9700 1 -- 172.21.15.45:0/1004413889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5880052a0 con 0x7fd590105080 2023-09-23T09:28:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.836+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59019f8a0 con 0x7fd590105080 2023-09-23T09:28:29.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:29 smithi123 ceph-mon[103577]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-09-23T09:28:29.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:29 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/903765406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-23T09:28:29.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:29 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2891610201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-23T09:28:29.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:29 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1798909167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-23T09:28:29.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.837+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59004eb20 con 0x7fd590105080 2023-09-23T09:28:29.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.838+0000 7fd58cff9700 1 -- 172.21.15.45:0/1004413889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fd58801d470 con 0x7fd590105080 2023-09-23T09:28:29.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.839+0000 7fd58cff9700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd5780a1630 0x7fd5780a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.839+0000 7fd58f7fe700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd5780a1630 0x7fd5780a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.839+0000 7fd58cff9700 1 -- 172.21.15.45:0/1004413889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd5880ce330 con 0x7fd590105080 2023-09-23T09:28:29.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.840+0000 7fd58f7fe700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd5780a1630 0x7fd5780a3af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd580005950 tx=0x7fd58000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:29.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.844+0000 7fd58cff9700 1 -- 172.21.15.45:0/1004413889 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd588097a00 con 0x7fd590105080 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.537+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/2948835816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 msgr2=0x7fd0f0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.537+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/2948835816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0101470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009990 tx=0x7fd0e4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.537+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/2948835816 shutdown_connections 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.537+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/2948835816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0f0106170 0x7fd0f010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.537+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/2948835816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0101470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.537+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/2948835816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0f0101a40 0x7fd0f0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.537+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/2948835816 >> 172.21.15.45:0/2948835816 conn(0x7fd0f00fc8c0 msgr2=0x7fd0f00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/2948835816 shutdown_connections 2023-09-23T09:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/2948835816 wait complete. 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 Processor -- start 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 -- start start 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0f0101a40 0x7fd0f0073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0f0106170 0x7fd0f0073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f01a5390 con 0x7fd0f0101090 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f01a5500 con 0x7fd0f0106170 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.538+0000 7fd0f6d7d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f01a56c0 con 0x7fd0f0101a40 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59368/0 (socket says 172.21.15.45:59368) 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 -- 172.21.15.45:0/725254792 learned_addr learned my addr 172.21.15.45:0/725254792 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0effff700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0f0101a40 0x7fd0f0073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f531a700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0f0106170 0x7fd0f0073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 -- 172.21.15.45:0/725254792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0f0101a40 msgr2=0x7fd0f0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0f0101a40 0x7fd0f0073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 -- 172.21.15.45:0/725254792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0f0106170 msgr2=0x7fd0f0073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0f0106170 0x7fd0f0073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 -- 172.21.15.45:0/725254792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40095f0 con 0x7fd0f0101090 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f4b19700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0073120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40095c0 tx=0x7fd0e400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0edffb700 1 -- 172.21.15.45:0/725254792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401d070 con 0x7fd0f0101090 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0edffb700 1 -- 172.21.15.45:0/725254792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e40054f0 con 0x7fd0f0101090 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f01a5940 con 0x7fd0f0101090 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0edffb700 1 -- 172.21.15.45:0/725254792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004a70 con 0x7fd0f0101090 2023-09-23T09:28:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.539+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f01a5de0 con 0x7fd0f0101090 2023-09-23T09:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.540+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f004eb20 con 0x7fd0f0101090 2023-09-23T09:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.541+0000 7fd0edffb700 1 -- 172.21.15.45:0/725254792 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fd0e4004bd0 con 0x7fd0f0101090 2023-09-23T09:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.542+0000 7fd0edffb700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd0e00a1630 0x7fd0e00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.542+0000 7fd0effff700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd0e00a1630 0x7fd0e00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.542+0000 7fd0edffb700 1 -- 172.21.15.45:0/725254792 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd0e40cdc50 con 0x7fd0f0101090 2023-09-23T09:28:29.859 INFO:teuthology.orchestra.run.smithi045.stdout:304942678039 2023-09-23T09:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.543+0000 7fd0effff700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd0e00a1630 0x7fd0e00a3af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0074910 tx=0x7fd0dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.546+0000 7fd0edffb700 1 -- 172.21.15.45:0/725254792 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd0e4097320 con 0x7fd0f0101090 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.750+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd0f0110fb0 con 0x7fd0f0101090 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.750+0000 7fd0edffb700 1 -- 172.21.15.45:0/725254792 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd0e4026020 con 0x7fd0f0101090 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd0e00a1630 msgr2=0x7fd0e00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd0e00a1630 0x7fd0e00a3af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0074910 tx=0x7fd0dc00b410 comp rx=0 tx=0).stop 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 msgr2=0x7fd0f0073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0073120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40095c0 tx=0x7fd0e400f710 comp rx=0 tx=0).stop 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 shutdown_connections 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0f0106170 0x7fd0f0073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f0101090 0x7fd0f0073120 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd0e00a1630 0x7fd0e00a3af0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 --2- 172.21.15.45:0/725254792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0f0101a40 0x7fd0f0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.752+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 >> 172.21.15.45:0/725254792 conn(0x7fd0f00fc8c0 msgr2=0x7fd0f00fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.753+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 shutdown_connections 2023-09-23T09:28:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:29.753+0000 7fd0f6d7d700 1 -- 172.21.15.45:0/725254792 wait complete. 2023-09-23T09:28:29.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364833 got 214748364835 for osd.7 2023-09-23T09:28:29.926 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.061+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd59010fd60 con 0x7fd590105080 2023-09-23T09:28:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.061+0000 7fd58cff9700 1 -- 172.21.15.45:0/1004413889 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd58809b020 con 0x7fd590105080 2023-09-23T09:28:30.063 INFO:teuthology.orchestra.run.smithi045.stdout:77309411385 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd5780a1630 msgr2=0x7fd5780a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd5780a1630 0x7fd5780a3af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd580005950 tx=0x7fd58000b500 comp rx=0 tx=0).stop 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 msgr2=0x7fd5901966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 0x7fd5901966e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b0a0 tx=0x7fd588007990 comp rx=0 tx=0).stop 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 shutdown_connections 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590105080 0x7fd5901966e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5900fffa0 0x7fd590195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd5780a1630 0x7fd5780a3af0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 --2- 172.21.15.45:0/1004413889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd590100950 0x7fd5901961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 >> 172.21.15.45:0/1004413889 conn(0x7fd5900fb890 msgr2=0x7fd5900fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 shutdown_connections 2023-09-23T09:28:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:30.063+0000 7fd59616e700 1 -- 172.21.15.45:0/1004413889 wait complete. 2023-09-23T09:28:30.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738431 got 34359738433 for osd.0 2023-09-23T09:28:30.392 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:30.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678037 got 304942678039 for osd.10 2023-09-23T09:28:30.392 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:30.737 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:30 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/725254792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-23T09:28:30.737 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:30 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1004413889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-23T09:28:30.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:30 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/725254792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-23T09:28:30.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:30 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1004413889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-23T09:28:30.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:30 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/725254792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-23T09:28:30.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:30 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1004413889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-23T09:28:30.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411383 got 77309411385 for osd.2 2023-09-23T09:28:30.873 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/3983255814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 msgr2=0x7f0f08105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/3983255814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08105a90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009990 tx=0x7f0ef8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/3983255814 shutdown_connections 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/3983255814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f08106170 0x7f0f0810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/3983255814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f08101090 0x7f0f08101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/3983255814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08105a90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/3983255814 >> 172.21.15.45:0/3983255814 conn(0x7f0f080fc920 msgr2=0x7f0f080fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/3983255814 shutdown_connections 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.558+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/3983255814 wait complete. 2023-09-23T09:28:31.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 Processor -- start 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 -- start start 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f08101090 0x7f0f08073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f08106170 0x7f0f08073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f081a52f0 con 0x7f0f08101a40 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f081a5460 con 0x7f0f08101090 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f0d5b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f081a5620 con 0x7f0f08106170 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59398/0 (socket says 172.21.15.45:59398) 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 -- 172.21.15.45:0/834575525 learned_addr learned my addr 172.21.15.45:0/834575525 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f077fe700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f08106170 0x7f0f08073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f06ffd700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f08101090 0x7f0f08073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:31.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 -- 172.21.15.45:0/834575525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f08106170 msgr2=0x7f0f08073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f08106170 0x7f0f08073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 -- 172.21.15.45:0/834575525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f08101090 msgr2=0x7f0f08073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f08101090 0x7f0f08073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f067fc700 1 -- 172.21.15.45:0/834575525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef80095f0 con 0x7f0f08101a40 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.559+0000 7f0f06ffd700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f08101090 0x7f0f08073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.560+0000 7f0f067fc700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08073660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.560+0000 7f0eeffff700 1 -- 172.21.15.45:0/834575525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef801d070 con 0x7f0f08101a40 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.560+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f081a58a0 con 0x7f0f08101a40 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.560+0000 7f0eeffff700 1 -- 172.21.15.45:0/834575525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef800fca0 con 0x7f0f08101a40 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.560+0000 7f0eeffff700 1 -- 172.21.15.45:0/834575525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8017760 con 0x7f0f08101a40 2023-09-23T09:28:31.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.560+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f081a5d10 con 0x7f0f08101a40 2023-09-23T09:28:31.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.561+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0804eb20 con 0x7f0f08101a40 2023-09-23T09:28:31.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.562+0000 7f0eeffff700 1 -- 172.21.15.45:0/834575525 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f0ef80178c0 con 0x7f0f08101a40 2023-09-23T09:28:31.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.562+0000 7f0eeffff700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ef00a1630 0x7f0ef00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:31.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.563+0000 7f0f06ffd700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ef00a1630 0x7f0ef00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:31.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.563+0000 7f0eeffff700 1 -- 172.21.15.45:0/834575525 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ef80ced90 con 0x7f0f08101a40 2023-09-23T09:28:31.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.563+0000 7f0f06ffd700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ef00a1630 0x7f0ef00a3af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0efc005fd0 tx=0x7f0efc005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:31.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.566+0000 7f0eeffff700 1 -- 172.21.15.45:0/834575525 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0ef8098360 con 0x7f0f08101a40 2023-09-23T09:28:31.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.765+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0f08110ed0 con 0x7f0f08101a40 2023-09-23T09:28:31.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.765+0000 7f0eeffff700 1 -- 172.21.15.45:0/834575525 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ef809b980 con 0x7f0f08101a40 2023-09-23T09:28:31.767 INFO:teuthology.orchestra.run.smithi045.stdout:163208757293 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ef00a1630 msgr2=0x7f0ef00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ef00a1630 0x7f0ef00a3af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0efc005fd0 tx=0x7f0efc005e20 comp rx=0 tx=0).stop 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 msgr2=0x7f0f08073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08073660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef800f7b0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 shutdown_connections 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f0ef00a1630 0x7f0ef00a3af0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f08106170 0x7f0f08073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f08101090 0x7f0f08073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 --2- 172.21.15.45:0/834575525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f08101a40 0x7f0f08073660 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:31.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 >> 172.21.15.45:0/834575525 conn(0x7f0f080fc920 msgr2=0x7f0f080fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:31.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 shutdown_connections 2023-09-23T09:28:31.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:31.767+0000 7f0f0d5b3700 1 -- 172.21.15.45:0/834575525 wait complete. 2023-09-23T09:28:31.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:31 smithi045 ceph-mon[98627]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2023-09-23T09:28:31.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:31 smithi098 ceph-mon[102386]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2023-09-23T09:28:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:31 smithi123 ceph-mon[103577]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2023-09-23T09:28:32.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757291 got 163208757293 for osd.5 2023-09-23T09:28:32.196 DEBUG:teuthology.parallel:result is None 2023-09-23T09:28:32.196 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-23T09:28:32.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph pg dump --format=json 2023-09-23T09:28:32.761 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:32.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:32 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/834575525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-23T09:28:32.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:32 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/834575525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-23T09:28:32.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:32 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/834575525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-23T09:28:33.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:33 smithi045 ceph-mon[98627]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:33.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:33 smithi098 ceph-mon[102386]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:33.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:33 smithi123 ceph-mon[103577]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 -- 172.21.15.45:0/3054076231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 msgr2=0x7f1f5c100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/3054076231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c100480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f500099c0 tx=0x7f1f50009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 -- 172.21.15.45:0/3054076231 shutdown_connections 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/3054076231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f5c102340 0x7f1f5c10cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/3054076231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c100480 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/3054076231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c101980 0x7f1f5c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 -- 172.21.15.45:0/3054076231 >> 172.21.15.45:0/3054076231 conn(0x7f1f5c076e10 msgr2=0x7f1f5c077220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 -- 172.21.15.45:0/3054076231 shutdown_connections 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.944+0000 7f1f62e0b700 1 -- 172.21.15.45:0/3054076231 wait complete. 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 Processor -- start 2023-09-23T09:28:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 -- start start 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f5c101980 0x7f1f5c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c102340 0x7f1f5c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c19bf80 con 0x7f1f5c1000a0 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c19c0f0 con 0x7f1f5c102340 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f62e0b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c19c2b0 con 0x7f1f5c101980 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f5c101980 0x7f1f5c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f60ba7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f60ba7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59408/0 (socket says 172.21.15.45:59408) 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.945+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f5c101980 0x7f1f5c197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:33284/0 (socket says 172.21.15.45:33284) 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f60ba7700 1 -- 172.21.15.45:0/571513606 learned_addr learned my addr 172.21.15.45:0/571513606 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f613a8700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c102340 0x7f1f5c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f60ba7700 1 -- 172.21.15.45:0/571513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f5c101980 msgr2=0x7f1f5c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f60ba7700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f5c101980 0x7f1f5c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f60ba7700 1 -- 172.21.15.45:0/571513606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c102340 msgr2=0x7f1f5c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f60ba7700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c102340 0x7f1f5c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f60ba7700 1 -- 172.21.15.45:0/571513606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500095f0 con 0x7f1f5c1000a0 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f60ba7700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c196fc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006010 tx=0x7f1f50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f59ffb700 1 -- 172.21.15.45:0/571513606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5001d070 con 0x7f1f5c1000a0 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f59ffb700 1 -- 172.21.15.45:0/571513606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f50004ca0 con 0x7f1f5c1000a0 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c19c530 con 0x7f1f5c1000a0 2023-09-23T09:28:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f59ffb700 1 -- 172.21.15.45:0/571513606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50003ae0 con 0x7f1f5c1000a0 2023-09-23T09:28:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.946+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c1a3df0 con 0x7f1f5c1000a0 2023-09-23T09:28:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.947+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c04eb20 con 0x7f1f5c1000a0 2023-09-23T09:28:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.948+0000 7f1f59ffb700 1 -- 172.21.15.45:0/571513606 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f1f50005030 con 0x7f1f5c1000a0 2023-09-23T09:28:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.949+0000 7f1f59ffb700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1f4c0a1630 0x7f1f4c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.949+0000 7f1f5bfff700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1f4c0a1630 0x7f1f4c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:33.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.949+0000 7f1f59ffb700 1 -- 172.21.15.45:0/571513606 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1f500ce0b0 con 0x7f1f5c1000a0 2023-09-23T09:28:33.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.950+0000 7f1f5bfff700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1f4c0a1630 0x7f1f4c0a3af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c1985e0 tx=0x7f1f4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:33.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:33.952+0000 7f1f59ffb700 1 -- 172.21.15.45:0/571513606 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1f50097780 con 0x7f1f5c1000a0 2023-09-23T09:28:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.146+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f5c110f50 con 0x7f1f4c0a1630 2023-09-23T09:28:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.158+0000 7f1f59ffb700 1 -- 172.21.15.45:0/571513606 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341063 (secure 0 0 0) 0x7f1f5c110f50 con 0x7f1f4c0a1630 2023-09-23T09:28:34.164 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1f4c0a1630 msgr2=0x7f1f4c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1f4c0a1630 0x7f1f4c0a3af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c1985e0 tx=0x7f1f4800b410 comp rx=0 tx=0).stop 2023-09-23T09:28:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 msgr2=0x7f1f5c196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c196fc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006010 tx=0x7f1f50004730 comp rx=0 tx=0).stop 2023-09-23T09:28:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 shutdown_connections 2023-09-23T09:28:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c102340 0x7f1f5c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f5c1000a0 0x7f1f5c196fc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1f4c0a1630 0x7f1f4c0a3af0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 --2- 172.21.15.45:0/571513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f5c101980 0x7f1f5c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 >> 172.21.15.45:0/571513606 conn(0x7f1f5c076e10 msgr2=0x7f1f5c103920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 shutdown_connections 2023-09-23T09:28:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:34.162+0000 7f1f62e0b700 1 -- 172.21.15.45:0/571513606 wait complete. 2023-09-23T09:28:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-09-23T09:28:34.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:34 smithi045 ceph-mon[98627]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:34.520 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2023-09-23T09:28:32.332289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651998,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":366,"num_read_kb":299,"num_write":332,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335164,"kb_used_data":10972,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606660,"statfs":{"total":1151940427776,"available":1151597219840,"internally_reserved":0,"allocated":11235328,"data_stored":7075574,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003344"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735132+0000","last_change":"2023-09-23T09:27:50.270599+0000","last_active":"2023-09-23T09:27:51.735132+0000","last_peered":"2023-09-23T09:27:51.735132+0000","last_clean":"2023-09-23T09:27:51.735132+0000","last_became_active":"2023-09-23T09:27:50.269926+0000","last_became_peered":"2023-09-23T09:27:50.269926+0000","last_unstale":"2023-09-23T09:27:51.735132+0000","last_undegraded":"2023-09-23T09:27:51.735132+0000","last_fullsized":"2023-09-23T09:27:51.735132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:12:08.908188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733803+0000","last_change":"2023-09-23T09:27:50.270451+0000","last_active":"2023-09-23T09:27:51.733803+0000","last_peered":"2023-09-23T09:27:51.733803+0000","last_clean":"2023-09-23T09:27:51.733803+0000","last_became_active":"2023-09-23T09:27:50.270055+0000","last_became_peered":"2023-09-23T09:27:50.270055+0000","last_unstale":"2023-09-23T09:27:51.733803+0000","last_undegraded":"2023-09-23T09:27:51.733803+0000","last_fullsized":"2023-09-23T09:27:51.733803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:56:43.388109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734671+0000","last_change":"2023-09-23T09:27:50.278101+0000","last_active":"2023-09-23T09:27:51.734671+0000","last_peered":"2023-09-23T09:27:51.734671+0000","last_clean":"2023-09-23T09:27:51.734671+0000","last_became_active":"2023-09-23T09:27:50.277739+0000","last_became_peered":"2023-09-23T09:27:50.277739+0000","last_unstale":"2023-09-23T09:27:51.734671+0000","last_undegraded":"2023-09-23T09:27:51.734671+0000","last_fullsized":"2023-09-23T09:27:51.734671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:52:31.582623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733930+0000","last_change":"2023-09-23T09:27:50.265962+0000","last_active":"2023-09-23T09:27:51.733930+0000","last_peered":"2023-09-23T09:27:51.733930+0000","last_clean":"2023-09-23T09:27:51.733930+0000","last_became_active":"2023-09-23T09:27:50.265681+0000","last_became_peered":"2023-09-23T09:27:50.265681+0000","last_unstale":"2023-09-23T09:27:51.733930+0000","last_undegraded":"2023-09-23T09:27:51.733930+0000","last_fullsized":"2023-09-23T09:27:51.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:12.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734705+0000","last_change":"2023-09-23T09:27:50.270356+0000","last_active":"2023-09-23T09:27:51.734705+0000","last_peered":"2023-09-23T09:27:51.734705+0000","last_clean":"2023-09-23T09:27:51.734705+0000","last_became_active":"2023-09-23T09:27:50.270124+0000","last_became_peered":"2023-09-23T09:27:50.270124+0000","last_unstale":"2023-09-23T09:27:51.734705+0000","last_undegraded":"2023-09-23T09:27:51.734705+0000","last_fullsized":"2023-09-23T09:27:51.734705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:14:00.508926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398274+0000","last_change":"2023-09-23T09:27:50.276122+0000","last_active":"2023-09-23T09:27:51.398274+0000","last_peered":"2023-09-23T09:27:51.398274+0000","last_clean":"2023-09-23T09:27:51.398274+0000","last_became_active":"2023-09-23T09:27:50.275847+0000","last_became_peered":"2023-09-23T09:27:50.275847+0000","last_unstale":"2023-09-23T09:27:51.398274+0000","last_undegraded":"2023-09-23T09:27:51.398274+0000","last_fullsized":"2023-09-23T09:27:51.398274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:04:03.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734273+0000","last_change":"2023-09-23T09:27:50.279420+0000","last_active":"2023-09-23T09:27:51.734273+0000","last_peered":"2023-09-23T09:27:51.734273+0000","last_clean":"2023-09-23T09:27:51.734273+0000","last_became_active":"2023-09-23T09:27:50.279120+0000","last_became_peered":"2023-09-23T09:27:50.279120+0000","last_unstale":"2023-09-23T09:27:51.734273+0000","last_undegraded":"2023-09-23T09:27:51.734273+0000","last_fullsized":"2023-09-23T09:27:51.734273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:48:03.461868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734300+0000","last_change":"2023-09-23T09:27:50.276364+0000","last_active":"2023-09-23T09:27:51.734300+0000","last_peered":"2023-09-23T09:27:51.734300+0000","last_clean":"2023-09-23T09:27:51.734300+0000","last_became_active":"2023-09-23T09:27:50.276060+0000","last_became_peered":"2023-09-23T09:27:50.276060+0000","last_unstale":"2023-09-23T09:27:51.734300+0000","last_undegraded":"2023-09-23T09:27:51.734300+0000","last_fullsized":"2023-09-23T09:27:51.734300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:26.998609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259923+0000","last_change":"2023-09-23T09:27:50.280302+0000","last_active":"2023-09-23T09:27:51.259923+0000","last_peered":"2023-09-23T09:27:51.259923+0000","last_clean":"2023-09-23T09:27:51.259923+0000","last_became_active":"2023-09-23T09:27:50.279295+0000","last_became_peered":"2023-09-23T09:27:50.279295+0000","last_unstale":"2023-09-23T09:27:51.259923+0000","last_undegraded":"2023-09-23T09:27:51.259923+0000","last_fullsized":"2023-09-23T09:27:51.259923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:15.329483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398526+0000","last_change":"2023-09-23T09:27:50.271876+0000","last_active":"2023-09-23T09:27:51.398526+0000","last_peered":"2023-09-23T09:27:51.398526+0000","last_clean":"2023-09-23T09:27:51.398526+0000","last_became_active":"2023-09-23T09:27:50.271576+0000","last_became_peered":"2023-09-23T09:27:50.271576+0000","last_unstale":"2023-09-23T09:27:51.398526+0000","last_undegraded":"2023-09-23T09:27:51.398526+0000","last_fullsized":"2023-09-23T09:27:51.398526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:17:22.656828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734649+0000","last_change":"2023-09-23T09:27:50.271405+0000","last_active":"2023-09-23T09:27:51.734649+0000","last_peered":"2023-09-23T09:27:51.734649+0000","last_clean":"2023-09-23T09:27:51.734649+0000","last_became_active":"2023-09-23T09:27:50.271087+0000","last_became_peered":"2023-09-23T09:27:50.271087+0000","last_unstale":"2023-09-23T09:27:51.734649+0000","last_undegraded":"2023-09-23T09:27:51.734649+0000","last_fullsized":"2023-09-23T09:27:51.734649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:09.363894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733876+0000","last_change":"2023-09-23T09:27:50.265855+0000","last_active":"2023-09-23T09:27:51.733876+0000","last_peered":"2023-09-23T09:27:51.733876+0000","last_clean":"2023-09-23T09:27:51.733876+0000","last_became_active":"2023-09-23T09:27:50.265101+0000","last_became_peered":"2023-09-23T09:27:50.265101+0000","last_unstale":"2023-09-23T09:27:51.733876+0000","last_undegraded":"2023-09-23T09:27:51.733876+0000","last_fullsized":"2023-09-23T09:27:51.733876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:52:40.456013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734201+0000","last_change":"2023-09-23T09:27:50.274405+0000","last_active":"2023-09-23T09:27:51.734201+0000","last_peered":"2023-09-23T09:27:51.734201+0000","last_clean":"2023-09-23T09:27:51.734201+0000","last_became_active":"2023-09-23T09:27:50.274036+0000","last_became_peered":"2023-09-23T09:27:50.274036+0000","last_unstale":"2023-09-23T09:27:51.734201+0000","last_undegraded":"2023-09-23T09:27:51.734201+0000","last_fullsized":"2023-09-23T09:27:51.734201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:38.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733331+0000","last_change":"2023-09-23T09:27:50.272237+0000","last_active":"2023-09-23T09:27:51.733331+0000","last_peered":"2023-09-23T09:27:51.733331+0000","last_clean":"2023-09-23T09:27:51.733331+0000","last_became_active":"2023-09-23T09:27:50.271897+0000","last_became_peered":"2023-09-23T09:27:50.271897+0000","last_unstale":"2023-09-23T09:27:51.733331+0000","last_undegraded":"2023-09-23T09:27:51.733331+0000","last_fullsized":"2023-09-23T09:27:51.733331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:31:31.115205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733452+0000","last_change":"2023-09-23T09:27:50.268902+0000","last_active":"2023-09-23T09:27:51.733452+0000","last_peered":"2023-09-23T09:27:51.733452+0000","last_clean":"2023-09-23T09:27:51.733452+0000","last_became_active":"2023-09-23T09:27:50.268590+0000","last_became_peered":"2023-09-23T09:27:50.268590+0000","last_unstale":"2023-09-23T09:27:51.733452+0000","last_undegraded":"2023-09-23T09:27:51.733452+0000","last_fullsized":"2023-09-23T09:27:51.733452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:54:39.384835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259359+0000","last_change":"2023-09-23T09:27:50.276934+0000","last_active":"2023-09-23T09:27:51.259359+0000","last_peered":"2023-09-23T09:27:51.259359+0000","last_clean":"2023-09-23T09:27:51.259359+0000","last_became_active":"2023-09-23T09:27:50.276645+0000","last_became_peered":"2023-09-23T09:27:50.276645+0000","last_unstale":"2023-09-23T09:27:51.259359+0000","last_undegraded":"2023-09-23T09:27:51.259359+0000","last_fullsized":"2023-09-23T09:27:51.259359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:07.100550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735229+0000","last_change":"2023-09-23T09:27:50.270781+0000","last_active":"2023-09-23T09:27:51.735229+0000","last_peered":"2023-09-23T09:27:51.735229+0000","last_clean":"2023-09-23T09:27:51.735229+0000","last_became_active":"2023-09-23T09:27:50.270319+0000","last_became_peered":"2023-09-23T09:27:50.270319+0000","last_unstale":"2023-09-23T09:27:51.735229+0000","last_undegraded":"2023-09-23T09:27:51.735229+0000","last_fullsized":"2023-09-23T09:27:51.735229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:42:44.882122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259692+0000","last_change":"2023-09-23T09:27:50.278775+0000","last_active":"2023-09-23T09:27:51.259692+0000","last_peered":"2023-09-23T09:27:51.259692+0000","last_clean":"2023-09-23T09:27:51.259692+0000","last_became_active":"2023-09-23T09:27:50.278375+0000","last_became_peered":"2023-09-23T09:27:50.278375+0000","last_unstale":"2023-09-23T09:27:51.259692+0000","last_undegraded":"2023-09-23T09:27:51.259692+0000","last_fullsized":"2023-09-23T09:27:51.259692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:14:00.076810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264221+0000","last_change":"2023-09-23T09:27:50.270141+0000","last_active":"2023-09-23T09:27:51.264221+0000","last_peered":"2023-09-23T09:27:51.264221+0000","last_clean":"2023-09-23T09:27:51.264221+0000","last_became_active":"2023-09-23T09:27:50.269849+0000","last_became_peered":"2023-09-23T09:27:50.269849+0000","last_unstale":"2023-09-23T09:27:51.264221+0000","last_undegraded":"2023-09-23T09:27:51.264221+0000","last_fullsized":"2023-09-23T09:27:51.264221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:49:41.837886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704055+0000","last_change":"2023-09-23T09:27:49.654458+0000","last_active":"2023-09-23T09:27:54.704055+0000","last_peered":"2023-09-23T09:27:54.704055+0000","last_clean":"2023-09-23T09:27:54.704055+0000","last_became_active":"2023-09-23T09:27:49.654084+0000","last_became_peered":"2023-09-23T09:27:49.654084+0000","last_unstale":"2023-09-23T09:27:54.704055+0000","last_undegraded":"2023-09-23T09:27:54.704055+0000","last_fullsized":"2023-09-23T09:27:54.704055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:47:32.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734956+0000","last_change":"2023-09-23T09:27:50.276558+0000","last_active":"2023-09-23T09:27:51.734956+0000","last_peered":"2023-09-23T09:27:51.734956+0000","last_clean":"2023-09-23T09:27:51.734956+0000","last_became_active":"2023-09-23T09:27:50.276255+0000","last_became_peered":"2023-09-23T09:27:50.276255+0000","last_unstale":"2023-09-23T09:27:51.734956+0000","last_undegraded":"2023-09-23T09:27:51.734956+0000","last_fullsized":"2023-09-23T09:27:51.734956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:13:09.870383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735166+0000","last_change":"2023-09-23T09:27:49.659256+0000","last_active":"2023-09-23T09:27:51.735166+0000","last_peered":"2023-09-23T09:27:51.735166+0000","last_clean":"2023-09-23T09:27:51.735166+0000","last_became_active":"2023-09-23T09:27:49.658793+0000","last_became_peered":"2023-09-23T09:27:49.658793+0000","last_unstale":"2023-09-23T09:27:51.735166+0000","last_undegraded":"2023-09-23T09:27:51.735166+0000","last_fullsized":"2023-09-23T09:27:51.735166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:03.416666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733674+0000","last_change":"2023-09-23T09:27:50.277338+0000","last_active":"2023-09-23T09:27:51.733674+0000","last_peered":"2023-09-23T09:27:51.733674+0000","last_clean":"2023-09-23T09:27:51.733674+0000","last_became_active":"2023-09-23T09:27:50.277011+0000","last_became_peered":"2023-09-23T09:27:50.277011+0000","last_unstale":"2023-09-23T09:27:51.733674+0000","last_undegraded":"2023-09-23T09:27:51.733674+0000","last_fullsized":"2023-09-23T09:27:51.733674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:52:23.028794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702688+0000","last_change":"2023-09-23T09:27:49.660679+0000","last_active":"2023-09-23T09:27:54.702688+0000","last_peered":"2023-09-23T09:27:54.702688+0000","last_clean":"2023-09-23T09:27:54.702688+0000","last_became_active":"2023-09-23T09:27:49.660201+0000","last_became_peered":"2023-09-23T09:27:49.660201+0000","last_unstale":"2023-09-23T09:27:54.702688+0000","last_undegraded":"2023-09-23T09:27:54.702688+0000","last_fullsized":"2023-09-23T09:27:54.702688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:33:56.205322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259494+0000","last_change":"2023-09-23T09:27:50.278373+0000","last_active":"2023-09-23T09:27:51.259494+0000","last_peered":"2023-09-23T09:27:51.259494+0000","last_clean":"2023-09-23T09:27:51.259494+0000","last_became_active":"2023-09-23T09:27:50.277837+0000","last_became_peered":"2023-09-23T09:27:50.277837+0000","last_unstale":"2023-09-23T09:27:51.259494+0000","last_undegraded":"2023-09-23T09:27:51.259494+0000","last_fullsized":"2023-09-23T09:27:51.259494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:24:26.914509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.117099+0000","last_change":"2023-09-23T09:27:49.657578+0000","last_active":"2023-09-23T09:27:51.117099+0000","last_peered":"2023-09-23T09:27:51.117099+0000","last_clean":"2023-09-23T09:27:51.117099+0000","last_became_active":"2023-09-23T09:27:49.657278+0000","last_became_peered":"2023-09-23T09:27:49.657278+0000","last_unstale":"2023-09-23T09:27:51.117099+0000","last_undegraded":"2023-09-23T09:27:51.117099+0000","last_fullsized":"2023-09-23T09:27:51.117099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:53.440997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734804+0000","last_change":"2023-09-23T09:27:50.279784+0000","last_active":"2023-09-23T09:27:51.734804+0000","last_peered":"2023-09-23T09:27:51.734804+0000","last_clean":"2023-09-23T09:27:51.734804+0000","last_became_active":"2023-09-23T09:27:50.279473+0000","last_became_peered":"2023-09-23T09:27:50.279473+0000","last_unstale":"2023-09-23T09:27:51.734804+0000","last_undegraded":"2023-09-23T09:27:51.734804+0000","last_fullsized":"2023-09-23T09:27:51.734804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:30:13.010556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:27.314754+0000","last_change":"2023-09-23T09:27:49.655391+0000","last_active":"2023-09-23T09:28:27.314754+0000","last_peered":"2023-09-23T09:28:27.314754+0000","last_clean":"2023-09-23T09:28:27.314754+0000","last_became_active":"2023-09-23T09:27:49.655067+0000","last_became_peered":"2023-09-23T09:27:49.655067+0000","last_unstale":"2023-09-23T09:28:27.314754+0000","last_undegraded":"2023-09-23T09:28:27.314754+0000","last_fullsized":"2023-09-23T09:28:27.314754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.701471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733903+0000","last_change":"2023-09-23T09:27:50.275688+0000","last_active":"2023-09-23T09:27:51.733903+0000","last_peered":"2023-09-23T09:27:51.733903+0000","last_clean":"2023-09-23T09:27:51.733903+0000","last_became_active":"2023-09-23T09:27:50.275356+0000","last_became_peered":"2023-09-23T09:27:50.275356+0000","last_unstale":"2023-09-23T09:27:51.733903+0000","last_undegraded":"2023-09-23T09:27:51.733903+0000","last_fullsized":"2023-09-23T09:27:51.733903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:43:31.829209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702167+0000","last_change":"2023-09-23T09:27:49.657066+0000","last_active":"2023-09-23T09:27:54.702167+0000","last_peered":"2023-09-23T09:27:54.702167+0000","last_clean":"2023-09-23T09:27:54.702167+0000","last_became_active":"2023-09-23T09:27:49.656754+0000","last_became_peered":"2023-09-23T09:27:49.656754+0000","last_unstale":"2023-09-23T09:27:54.702167+0000","last_undegraded":"2023-09-23T09:27:54.702167+0000","last_fullsized":"2023-09-23T09:27:54.702167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:52.277474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734458+0000","last_change":"2023-09-23T09:27:50.273705+0000","last_active":"2023-09-23T09:27:51.734458+0000","last_peered":"2023-09-23T09:27:51.734458+0000","last_clean":"2023-09-23T09:27:51.734458+0000","last_became_active":"2023-09-23T09:27:50.272245+0000","last_became_peered":"2023-09-23T09:27:50.272245+0000","last_unstale":"2023-09-23T09:27:51.734458+0000","last_undegraded":"2023-09-23T09:27:51.734458+0000","last_fullsized":"2023-09-23T09:27:51.734458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:45:57.318326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704059+0000","last_change":"2023-09-23T09:27:49.651958+0000","last_active":"2023-09-23T09:27:54.704059+0000","last_peered":"2023-09-23T09:27:54.704059+0000","last_clean":"2023-09-23T09:27:54.704059+0000","last_became_active":"2023-09-23T09:27:49.651058+0000","last_became_peered":"2023-09-23T09:27:49.651058+0000","last_unstale":"2023-09-23T09:27:54.704059+0000","last_undegraded":"2023-09-23T09:27:54.704059+0000","last_fullsized":"2023-09-23T09:27:54.704059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:26:56.360566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734149+0000","last_change":"2023-09-23T09:27:50.270804+0000","last_active":"2023-09-23T09:27:51.734149+0000","last_peered":"2023-09-23T09:27:51.734149+0000","last_clean":"2023-09-23T09:27:51.734149+0000","last_became_active":"2023-09-23T09:27:50.270044+0000","last_became_peered":"2023-09-23T09:27:50.270044+0000","last_unstale":"2023-09-23T09:27:51.734149+0000","last_undegraded":"2023-09-23T09:27:51.734149+0000","last_fullsized":"2023-09-23T09:27:51.734149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:59:20.821949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734563+0000","last_change":"2023-09-23T09:27:49.657030+0000","last_active":"2023-09-23T09:27:51.734563+0000","last_peered":"2023-09-23T09:27:51.734563+0000","last_clean":"2023-09-23T09:27:51.734563+0000","last_became_active":"2023-09-23T09:27:49.656710+0000","last_became_peered":"2023-09-23T09:27:49.656710+0000","last_unstale":"2023-09-23T09:27:51.734563+0000","last_undegraded":"2023-09-23T09:27:51.734563+0000","last_fullsized":"2023-09-23T09:27:51.734563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:24:59.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259415+0000","last_change":"2023-09-23T09:27:50.271688+0000","last_active":"2023-09-23T09:27:51.259415+0000","last_peered":"2023-09-23T09:27:51.259415+0000","last_clean":"2023-09-23T09:27:51.259415+0000","last_became_active":"2023-09-23T09:27:50.271276+0000","last_became_peered":"2023-09-23T09:27:50.271276+0000","last_unstale":"2023-09-23T09:27:51.259415+0000","last_undegraded":"2023-09-23T09:27:51.259415+0000","last_fullsized":"2023-09-23T09:27:51.259415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:40:48.333279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:27.314042+0000","last_change":"2023-09-23T09:27:49.655183+0000","last_active":"2023-09-23T09:28:27.314042+0000","last_peered":"2023-09-23T09:28:27.314042+0000","last_clean":"2023-09-23T09:28:27.314042+0000","last_became_active":"2023-09-23T09:27:49.654875+0000","last_became_peered":"2023-09-23T09:27:49.654875+0000","last_unstale":"2023-09-23T09:28:27.314042+0000","last_undegraded":"2023-09-23T09:28:27.314042+0000","last_fullsized":"2023-09-23T09:28:27.314042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:12:49.944732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259843+0000","last_change":"2023-09-23T09:27:50.276347+0000","last_active":"2023-09-23T09:27:51.259843+0000","last_peered":"2023-09-23T09:27:51.259843+0000","last_clean":"2023-09-23T09:27:51.259843+0000","last_became_active":"2023-09-23T09:27:50.276095+0000","last_became_peered":"2023-09-23T09:27:50.276095+0000","last_unstale":"2023-09-23T09:27:51.259843+0000","last_undegraded":"2023-09-23T09:27:51.259843+0000","last_fullsized":"2023-09-23T09:27:51.259843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:42.952808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.692327+0000","last_change":"2023-09-23T09:27:49.657115+0000","last_active":"2023-09-23T09:27:53.692327+0000","last_peered":"2023-09-23T09:27:53.692327+0000","last_clean":"2023-09-23T09:27:53.692327+0000","last_became_active":"2023-09-23T09:27:49.656816+0000","last_became_peered":"2023-09-23T09:27:49.656816+0000","last_unstale":"2023-09-23T09:27:53.692327+0000","last_undegraded":"2023-09-23T09:27:53.692327+0000","last_fullsized":"2023-09-23T09:27:53.692327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:37.205221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733619+0000","last_change":"2023-09-23T09:27:50.277477+0000","last_active":"2023-09-23T09:27:51.733619+0000","last_peered":"2023-09-23T09:27:51.733619+0000","last_clean":"2023-09-23T09:27:51.733619+0000","last_became_active":"2023-09-23T09:27:50.277200+0000","last_became_peered":"2023-09-23T09:27:50.277200+0000","last_unstale":"2023-09-23T09:27:51.733619+0000","last_undegraded":"2023-09-23T09:27:51.733619+0000","last_fullsized":"2023-09-23T09:27:51.733619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:45:03.167471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.691949+0000","last_change":"2023-09-23T09:27:49.663678+0000","last_active":"2023-09-23T09:27:53.691949+0000","last_peered":"2023-09-23T09:27:53.691949+0000","last_clean":"2023-09-23T09:27:53.691949+0000","last_became_active":"2023-09-23T09:27:49.663142+0000","last_became_peered":"2023-09-23T09:27:49.663142+0000","last_unstale":"2023-09-23T09:27:53.691949+0000","last_undegraded":"2023-09-23T09:27:53.691949+0000","last_fullsized":"2023-09-23T09:27:53.691949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:14.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259656+0000","last_change":"2023-09-23T09:27:50.275044+0000","last_active":"2023-09-23T09:27:51.259656+0000","last_peered":"2023-09-23T09:27:51.259656+0000","last_clean":"2023-09-23T09:27:51.259656+0000","last_became_active":"2023-09-23T09:27:50.274752+0000","last_became_peered":"2023-09-23T09:27:50.274752+0000","last_unstale":"2023-09-23T09:27:51.259656+0000","last_undegraded":"2023-09-23T09:27:51.259656+0000","last_fullsized":"2023-09-23T09:27:51.259656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:19.914752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":114,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:28.315639+0000","last_change":"2023-09-23T09:27:49.653694+0000","last_active":"2023-09-23T09:28:28.315639+0000","last_peered":"2023-09-23T09:28:28.315639+0000","last_clean":"2023-09-23T09:28:28.315639+0000","last_became_active":"2023-09-23T09:27:49.653355+0000","last_became_peered":"2023-09-23T09:27:49.653355+0000","last_unstale":"2023-09-23T09:28:28.315639+0000","last_undegraded":"2023-09-23T09:28:28.315639+0000","last_fullsized":"2023-09-23T09:28:28.315639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:19:53.282723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56994,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":92,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.272849+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.272411+0000","last_became_peered":"2023-09-23T09:27:50.272411+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:01:41.972357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702769+0000","last_change":"2023-09-23T09:27:49.652769+0000","last_active":"2023-09-23T09:27:54.702769+0000","last_peered":"2023-09-23T09:27:54.702769+0000","last_clean":"2023-09-23T09:27:54.702769+0000","last_became_active":"2023-09-23T09:27:49.652440+0000","last_became_peered":"2023-09-23T09:27:49.652440+0000","last_unstale":"2023-09-23T09:27:54.702769+0000","last_undegraded":"2023-09-23T09:27:54.702769+0000","last_fullsized":"2023-09-23T09:27:54.702769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:02.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734000+0000","last_change":"2023-09-23T09:27:50.272105+0000","last_active":"2023-09-23T09:27:51.734000+0000","last_peered":"2023-09-23T09:27:51.734000+0000","last_clean":"2023-09-23T09:27:51.734000+0000","last_became_active":"2023-09-23T09:27:50.271766+0000","last_became_peered":"2023-09-23T09:27:50.271766+0000","last_unstale":"2023-09-23T09:27:51.734000+0000","last_undegraded":"2023-09-23T09:27:51.734000+0000","last_fullsized":"2023-09-23T09:27:51.734000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:46:46.012381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690285+0000","last_change":"2023-09-23T09:27:49.663473+0000","last_active":"2023-09-23T09:27:53.690285+0000","last_peered":"2023-09-23T09:27:53.690285+0000","last_clean":"2023-09-23T09:27:53.690285+0000","last_became_active":"2023-09-23T09:27:49.663085+0000","last_became_peered":"2023-09-23T09:27:49.663085+0000","last_unstale":"2023-09-23T09:27:53.690285+0000","last_undegraded":"2023-09-23T09:27:53.690285+0000","last_fullsized":"2023-09-23T09:27:53.690285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:28:05.246720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734657+0000","last_change":"2023-09-23T09:27:50.274172+0000","last_active":"2023-09-23T09:27:51.734657+0000","last_peered":"2023-09-23T09:27:51.734657+0000","last_clean":"2023-09-23T09:27:51.734657+0000","last_became_active":"2023-09-23T09:27:50.273874+0000","last_became_peered":"2023-09-23T09:27:50.273874+0000","last_unstale":"2023-09-23T09:27:51.734657+0000","last_undegraded":"2023-09-23T09:27:51.734657+0000","last_fullsized":"2023-09-23T09:27:51.734657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:04:06.864055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:26.312655+0000","last_change":"2023-09-23T09:27:49.654341+0000","last_active":"2023-09-23T09:28:26.312655+0000","last_peered":"2023-09-23T09:28:26.312655+0000","last_clean":"2023-09-23T09:28:26.312655+0000","last_became_active":"2023-09-23T09:27:49.653936+0000","last_became_peered":"2023-09-23T09:27:49.653936+0000","last_unstale":"2023-09-23T09:28:26.312655+0000","last_undegraded":"2023-09-23T09:28:26.312655+0000","last_fullsized":"2023-09-23T09:28:26.312655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:48.876752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734194+0000","last_change":"2023-09-23T09:27:50.278620+0000","last_active":"2023-09-23T09:27:51.734194+0000","last_peered":"2023-09-23T09:27:51.734194+0000","last_clean":"2023-09-23T09:27:51.734194+0000","last_became_active":"2023-09-23T09:27:50.278289+0000","last_became_peered":"2023-09-23T09:27:50.278289+0000","last_unstale":"2023-09-23T09:27:51.734194+0000","last_undegraded":"2023-09-23T09:27:51.734194+0000","last_fullsized":"2023-09-23T09:27:51.734194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:20:44.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704421+0000","last_change":"2023-09-23T09:27:49.662970+0000","last_active":"2023-09-23T09:27:54.704421+0000","last_peered":"2023-09-23T09:27:54.704421+0000","last_clean":"2023-09-23T09:27:54.704421+0000","last_became_active":"2023-09-23T09:27:49.662664+0000","last_became_peered":"2023-09-23T09:27:49.662664+0000","last_unstale":"2023-09-23T09:27:54.704421+0000","last_undegraded":"2023-09-23T09:27:54.704421+0000","last_fullsized":"2023-09-23T09:27:54.704421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:13.189686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734127+0000","last_change":"2023-09-23T09:27:50.278475+0000","last_active":"2023-09-23T09:27:51.734127+0000","last_peered":"2023-09-23T09:27:51.734127+0000","last_clean":"2023-09-23T09:27:51.734127+0000","last_became_active":"2023-09-23T09:27:50.277802+0000","last_became_peered":"2023-09-23T09:27:50.277802+0000","last_unstale":"2023-09-23T09:27:51.734127+0000","last_undegraded":"2023-09-23T09:27:51.734127+0000","last_fullsized":"2023-09-23T09:27:51.734127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:44.608172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:55.261245+0000","last_change":"2023-09-23T09:27:49.657191+0000","last_active":"2023-09-23T09:27:55.261245+0000","last_peered":"2023-09-23T09:27:55.261245+0000","last_clean":"2023-09-23T09:27:55.261245+0000","last_became_active":"2023-09-23T09:27:49.656863+0000","last_became_peered":"2023-09-23T09:27:49.656863+0000","last_unstale":"2023-09-23T09:27:55.261245+0000","last_undegraded":"2023-09-23T09:27:55.261245+0000","last_fullsized":"2023-09-23T09:27:55.261245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:10.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734984+0000","last_change":"2023-09-23T09:27:50.274609+0000","last_active":"2023-09-23T09:27:51.734984+0000","last_peered":"2023-09-23T09:27:51.734984+0000","last_clean":"2023-09-23T09:27:51.734984+0000","last_became_active":"2023-09-23T09:27:50.274316+0000","last_became_peered":"2023-09-23T09:27:50.274316+0000","last_unstale":"2023-09-23T09:27:51.734984+0000","last_undegraded":"2023-09-23T09:27:51.734984+0000","last_fullsized":"2023-09-23T09:27:51.734984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:10:51.284306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259542+0000","last_change":"2023-09-23T09:27:49.659410+0000","last_active":"2023-09-23T09:27:51.259542+0000","last_peered":"2023-09-23T09:27:51.259542+0000","last_clean":"2023-09-23T09:27:51.259542+0000","last_became_active":"2023-09-23T09:27:49.659087+0000","last_became_peered":"2023-09-23T09:27:49.659087+0000","last_unstale":"2023-09-23T09:27:51.259542+0000","last_undegraded":"2023-09-23T09:27:51.259542+0000","last_fullsized":"2023-09-23T09:27:51.259542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:23:41.677416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.258804+0000","last_change":"2023-09-23T09:27:50.268568+0000","last_active":"2023-09-23T09:27:54.258804+0000","last_peered":"2023-09-23T09:27:54.258804+0000","last_clean":"2023-09-23T09:27:54.258804+0000","last_became_active":"2023-09-23T09:27:50.268268+0000","last_became_peered":"2023-09-23T09:27:50.268268+0000","last_unstale":"2023-09-23T09:27:54.258804+0000","last_undegraded":"2023-09-23T09:27:54.258804+0000","last_fullsized":"2023-09-23T09:27:54.258804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:59:22.516794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734411+0000","last_change":"2023-09-23T09:27:49.657160+0000","last_active":"2023-09-23T09:27:51.734411+0000","last_peered":"2023-09-23T09:27:51.734411+0000","last_clean":"2023-09-23T09:27:51.734411+0000","last_became_active":"2023-09-23T09:27:49.656853+0000","last_became_peered":"2023-09-23T09:27:49.656853+0000","last_unstale":"2023-09-23T09:27:51.734411+0000","last_undegraded":"2023-09-23T09:27:51.734411+0000","last_fullsized":"2023-09-23T09:27:51.734411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:17.489250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398429+0000","last_change":"2023-09-23T09:27:50.272495+0000","last_active":"2023-09-23T09:27:51.398429+0000","last_peered":"2023-09-23T09:27:51.398429+0000","last_clean":"2023-09-23T09:27:51.398429+0000","last_became_active":"2023-09-23T09:27:50.272207+0000","last_became_peered":"2023-09-23T09:27:50.272207+0000","last_unstale":"2023-09-23T09:27:51.398429+0000","last_undegraded":"2023-09-23T09:27:51.398429+0000","last_fullsized":"2023-09-23T09:27:51.398429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:24.952253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734722+0000","last_change":"2023-09-23T09:27:50.275551+0000","last_active":"2023-09-23T09:27:51.734722+0000","last_peered":"2023-09-23T09:27:51.734722+0000","last_clean":"2023-09-23T09:27:51.734722+0000","last_became_active":"2023-09-23T09:27:50.275235+0000","last_became_peered":"2023-09-23T09:27:50.275235+0000","last_unstale":"2023-09-23T09:27:51.734722+0000","last_undegraded":"2023-09-23T09:27:51.734722+0000","last_fullsized":"2023-09-23T09:27:51.734722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:41.009566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":51,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:27.313627+0000","last_change":"2023-09-23T09:27:49.652889+0000","last_active":"2023-09-23T09:28:27.313627+0000","last_peered":"2023-09-23T09:28:27.313627+0000","last_clean":"2023-09-23T09:28:27.313627+0000","last_became_active":"2023-09-23T09:27:49.652571+0000","last_became_peered":"2023-09-23T09:27:49.652571+0000","last_unstale":"2023-09-23T09:28:27.313627+0000","last_undegraded":"2023-09-23T09:28:27.313627+0000","last_fullsized":"2023-09-23T09:28:27.313627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:22.484052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734814+0000","last_change":"2023-09-23T09:27:50.278419+0000","last_active":"2023-09-23T09:27:51.734814+0000","last_peered":"2023-09-23T09:27:51.734814+0000","last_clean":"2023-09-23T09:27:51.734814+0000","last_became_active":"2023-09-23T09:27:50.278071+0000","last_became_peered":"2023-09-23T09:27:50.278071+0000","last_unstale":"2023-09-23T09:27:51.734814+0000","last_undegraded":"2023-09-23T09:27:51.734814+0000","last_fullsized":"2023-09-23T09:27:51.734814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:28:45.896479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:55.260221+0000","last_change":"2023-09-23T09:27:49.651251+0000","last_active":"2023-09-23T09:27:55.260221+0000","last_peered":"2023-09-23T09:27:55.260221+0000","last_clean":"2023-09-23T09:27:55.260221+0000","last_became_active":"2023-09-23T09:27:49.650933+0000","last_became_peered":"2023-09-23T09:27:49.650933+0000","last_unstale":"2023-09-23T09:27:55.260221+0000","last_undegraded":"2023-09-23T09:27:55.260221+0000","last_fullsized":"2023-09-23T09:27:55.260221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:00:54.100400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.264684+0000","last_change":"2023-09-23T09:26:51.296669+0000","last_active":"2023-09-23T09:27:50.264684+0000","last_peered":"2023-09-23T09:27:50.264684+0000","last_clean":"2023-09-23T09:27:50.264684+0000","last_became_active":"2023-09-23T09:26:51.296229+0000","last_became_peered":"2023-09-23T09:26:51.296229+0000","last_unstale":"2023-09-23T09:27:50.264684+0000","last_undegraded":"2023-09-23T09:27:50.264684+0000","last_fullsized":"2023-09-23T09:27:50.264684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_clean_scrub_stamp":"2023-09-23T09:24:01.204422+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:31:24.888152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734832+0000","last_change":"2023-09-23T09:27:50.267662+0000","last_active":"2023-09-23T09:27:51.734832+0000","last_peered":"2023-09-23T09:27:51.734832+0000","last_clean":"2023-09-23T09:27:51.734832+0000","last_became_active":"2023-09-23T09:27:50.267374+0000","last_became_peered":"2023-09-23T09:27:50.267374+0000","last_unstale":"2023-09-23T09:27:51.734832+0000","last_undegraded":"2023-09-23T09:27:51.734832+0000","last_fullsized":"2023-09-23T09:27:51.734832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:22.797939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.701373+0000","last_change":"2023-09-23T09:27:49.659773+0000","last_active":"2023-09-23T09:27:54.701373+0000","last_peered":"2023-09-23T09:27:54.701373+0000","last_clean":"2023-09-23T09:27:54.701373+0000","last_became_active":"2023-09-23T09:27:49.659526+0000","last_became_peered":"2023-09-23T09:27:49.659526+0000","last_unstale":"2023-09-23T09:27:54.701373+0000","last_undegraded":"2023-09-23T09:27:54.701373+0000","last_fullsized":"2023-09-23T09:27:54.701373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:27.423680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689202+0000","last_change":"2023-09-23T09:27:49.661336+0000","last_active":"2023-09-23T09:27:53.689202+0000","last_peered":"2023-09-23T09:27:53.689202+0000","last_clean":"2023-09-23T09:27:53.689202+0000","last_became_active":"2023-09-23T09:27:49.661038+0000","last_became_peered":"2023-09-23T09:27:49.661038+0000","last_unstale":"2023-09-23T09:27:53.689202+0000","last_undegraded":"2023-09-23T09:27:53.689202+0000","last_fullsized":"2023-09-23T09:27:53.689202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:20.634410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735024+0000","last_change":"2023-09-23T09:27:50.272485+0000","last_active":"2023-09-23T09:27:51.735024+0000","last_peered":"2023-09-23T09:27:51.735024+0000","last_clean":"2023-09-23T09:27:51.735024+0000","last_became_active":"2023-09-23T09:27:50.272164+0000","last_became_peered":"2023-09-23T09:27:50.272164+0000","last_unstale":"2023-09-23T09:27:51.735024+0000","last_undegraded":"2023-09-23T09:27:51.735024+0000","last_fullsized":"2023-09-23T09:27:51.735024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:48:00.596034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:58.934312+0000","last_change":"2023-09-23T09:27:49.657563+0000","last_active":"2023-09-23T09:27:58.934312+0000","last_peered":"2023-09-23T09:27:58.934312+0000","last_clean":"2023-09-23T09:27:58.934312+0000","last_became_active":"2023-09-23T09:27:49.657232+0000","last_became_peered":"2023-09-23T09:27:49.657232+0000","last_unstale":"2023-09-23T09:27:58.934312+0000","last_undegraded":"2023-09-23T09:27:58.934312+0000","last_fullsized":"2023-09-23T09:27:58.934312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:29:10.640013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735007+0000","last_change":"2023-09-23T09:27:50.274959+0000","last_active":"2023-09-23T09:27:51.735007+0000","last_peered":"2023-09-23T09:27:51.735007+0000","last_clean":"2023-09-23T09:27:51.735007+0000","last_became_active":"2023-09-23T09:27:50.274614+0000","last_became_peered":"2023-09-23T09:27:50.274614+0000","last_unstale":"2023-09-23T09:27:51.735007+0000","last_undegraded":"2023-09-23T09:27:51.735007+0000","last_fullsized":"2023-09-23T09:27:51.735007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:00:09.831103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690819+0000","last_change":"2023-09-23T09:27:49.654673+0000","last_active":"2023-09-23T09:27:53.690819+0000","last_peered":"2023-09-23T09:27:53.690819+0000","last_clean":"2023-09-23T09:27:53.690819+0000","last_became_active":"2023-09-23T09:27:49.654356+0000","last_became_peered":"2023-09-23T09:27:49.654356+0000","last_unstale":"2023-09-23T09:27:53.690819+0000","last_undegraded":"2023-09-23T09:27:53.690819+0000","last_fullsized":"2023-09-23T09:27:53.690819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:39:50.628346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.267368+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.266962+0000","last_became_peered":"2023-09-23T09:27:50.266962+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:24.660228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690757+0000","last_change":"2023-09-23T09:27:49.661200+0000","last_active":"2023-09-23T09:27:53.690757+0000","last_peered":"2023-09-23T09:27:53.690757+0000","last_clean":"2023-09-23T09:27:53.690757+0000","last_became_active":"2023-09-23T09:27:49.660902+0000","last_became_peered":"2023-09-23T09:27:49.660902+0000","last_unstale":"2023-09-23T09:27:53.690757+0000","last_undegraded":"2023-09-23T09:27:53.690757+0000","last_fullsized":"2023-09-23T09:27:53.690757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:41.036216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735005+0000","last_change":"2023-09-23T09:27:50.271834+0000","last_active":"2023-09-23T09:27:51.735005+0000","last_peered":"2023-09-23T09:27:51.735005+0000","last_clean":"2023-09-23T09:27:51.735005+0000","last_became_active":"2023-09-23T09:27:50.271519+0000","last_became_peered":"2023-09-23T09:27:50.271519+0000","last_unstale":"2023-09-23T09:27:51.735005+0000","last_undegraded":"2023-09-23T09:27:51.735005+0000","last_fullsized":"2023-09-23T09:27:51.735005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:08.036615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689622+0000","last_change":"2023-09-23T09:27:49.656794+0000","last_active":"2023-09-23T09:27:53.689622+0000","last_peered":"2023-09-23T09:27:53.689622+0000","last_clean":"2023-09-23T09:27:53.689622+0000","last_became_active":"2023-09-23T09:27:49.656480+0000","last_became_peered":"2023-09-23T09:27:49.656480+0000","last_unstale":"2023-09-23T09:27:53.689622+0000","last_undegraded":"2023-09-23T09:27:53.689622+0000","last_fullsized":"2023-09-23T09:27:53.689622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:35:46.948703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734135+0000","last_change":"2023-09-23T09:27:50.271805+0000","last_active":"2023-09-23T09:27:51.734135+0000","last_peered":"2023-09-23T09:27:51.734135+0000","last_clean":"2023-09-23T09:27:51.734135+0000","last_became_active":"2023-09-23T09:27:50.271463+0000","last_became_peered":"2023-09-23T09:27:50.271463+0000","last_unstale":"2023-09-23T09:27:51.734135+0000","last_undegraded":"2023-09-23T09:27:51.734135+0000","last_fullsized":"2023-09-23T09:27:51.734135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:00:03.608504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264108+0000","last_change":"2023-09-23T09:27:49.656988+0000","last_active":"2023-09-23T09:27:51.264108+0000","last_peered":"2023-09-23T09:27:51.264108+0000","last_clean":"2023-09-23T09:27:51.264108+0000","last_became_active":"2023-09-23T09:27:49.656655+0000","last_became_peered":"2023-09-23T09:27:49.656655+0000","last_unstale":"2023-09-23T09:27:51.264108+0000","last_undegraded":"2023-09-23T09:27:51.264108+0000","last_fullsized":"2023-09-23T09:27:51.264108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:07:33.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734679+0000","last_change":"2023-09-23T09:27:50.275021+0000","last_active":"2023-09-23T09:27:51.734679+0000","last_peered":"2023-09-23T09:27:51.734679+0000","last_clean":"2023-09-23T09:27:51.734679+0000","last_became_active":"2023-09-23T09:27:50.274551+0000","last_became_peered":"2023-09-23T09:27:50.274551+0000","last_unstale":"2023-09-23T09:27:51.734679+0000","last_undegraded":"2023-09-23T09:27:51.734679+0000","last_fullsized":"2023-09-23T09:27:51.734679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:50:45.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690216+0000","last_change":"2023-09-23T09:27:49.659603+0000","last_active":"2023-09-23T09:27:53.690216+0000","last_peered":"2023-09-23T09:27:53.690216+0000","last_clean":"2023-09-23T09:27:53.690216+0000","last_became_active":"2023-09-23T09:27:49.659293+0000","last_became_peered":"2023-09-23T09:27:49.659293+0000","last_unstale":"2023-09-23T09:27:53.690216+0000","last_undegraded":"2023-09-23T09:27:53.690216+0000","last_fullsized":"2023-09-23T09:27:53.690216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:31:25.165888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734478+0000","last_change":"2023-09-23T09:27:50.275069+0000","last_active":"2023-09-23T09:27:51.734478+0000","last_peered":"2023-09-23T09:27:51.734478+0000","last_clean":"2023-09-23T09:27:51.734478+0000","last_became_active":"2023-09-23T09:27:50.274731+0000","last_became_peered":"2023-09-23T09:27:50.274731+0000","last_unstale":"2023-09-23T09:27:51.734478+0000","last_undegraded":"2023-09-23T09:27:51.734478+0000","last_fullsized":"2023-09-23T09:27:51.734478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:45:23.794774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.703349+0000","last_change":"2023-09-23T09:27:49.659914+0000","last_active":"2023-09-23T09:27:54.703349+0000","last_peered":"2023-09-23T09:27:54.703349+0000","last_clean":"2023-09-23T09:27:54.703349+0000","last_became_active":"2023-09-23T09:27:49.659604+0000","last_became_peered":"2023-09-23T09:27:49.659604+0000","last_unstale":"2023-09-23T09:27:54.703349+0000","last_undegraded":"2023-09-23T09:27:54.703349+0000","last_fullsized":"2023-09-23T09:27:54.703349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:20.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734661+0000","last_change":"2023-09-23T09:27:50.271309+0000","last_active":"2023-09-23T09:27:51.734661+0000","last_peered":"2023-09-23T09:27:51.734661+0000","last_clean":"2023-09-23T09:27:51.734661+0000","last_became_active":"2023-09-23T09:27:50.270951+0000","last_became_peered":"2023-09-23T09:27:50.270951+0000","last_unstale":"2023-09-23T09:27:51.734661+0000","last_undegraded":"2023-09-23T09:27:51.734661+0000","last_fullsized":"2023-09-23T09:27:51.734661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:18.811428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734659+0000","last_change":"2023-09-23T09:27:49.663849+0000","last_active":"2023-09-23T09:27:51.734659+0000","last_peered":"2023-09-23T09:27:51.734659+0000","last_clean":"2023-09-23T09:27:51.734659+0000","last_became_active":"2023-09-23T09:27:49.663499+0000","last_became_peered":"2023-09-23T09:27:49.663499+0000","last_unstale":"2023-09-23T09:27:51.734659+0000","last_undegraded":"2023-09-23T09:27:51.734659+0000","last_fullsized":"2023-09-23T09:27:51.734659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:31:08.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734582+0000","last_change":"2023-09-23T09:27:50.275681+0000","last_active":"2023-09-23T09:27:51.734582+0000","last_peered":"2023-09-23T09:27:51.734582+0000","last_clean":"2023-09-23T09:27:51.734582+0000","last_became_active":"2023-09-23T09:27:50.275370+0000","last_became_peered":"2023-09-23T09:27:50.275370+0000","last_unstale":"2023-09-23T09:27:51.734582+0000","last_undegraded":"2023-09-23T09:27:51.734582+0000","last_fullsized":"2023-09-23T09:27:51.734582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:53:01.710377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.665421+0000","last_change":"2023-09-23T09:27:49.652127+0000","last_active":"2023-09-23T09:27:50.665421+0000","last_peered":"2023-09-23T09:27:50.665421+0000","last_clean":"2023-09-23T09:27:50.665421+0000","last_became_active":"2023-09-23T09:27:49.651792+0000","last_became_peered":"2023-09-23T09:27:49.651792+0000","last_unstale":"2023-09-23T09:27:50.665421+0000","last_undegraded":"2023-09-23T09:27:50.665421+0000","last_fullsized":"2023-09-23T09:27:50.665421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:23:39.299203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734734+0000","last_change":"2023-09-23T09:27:50.271063+0000","last_active":"2023-09-23T09:27:51.734734+0000","last_peered":"2023-09-23T09:27:51.734734+0000","last_clean":"2023-09-23T09:27:51.734734+0000","last_became_active":"2023-09-23T09:27:50.270487+0000","last_became_peered":"2023-09-23T09:27:50.270487+0000","last_unstale":"2023-09-23T09:27:51.734734+0000","last_undegraded":"2023-09-23T09:27:51.734734+0000","last_fullsized":"2023-09-23T09:27:51.734734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:11:05.621855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734264+0000","last_change":"2023-09-23T09:27:50.275900+0000","last_active":"2023-09-23T09:27:51.734264+0000","last_peered":"2023-09-23T09:27:51.734264+0000","last_clean":"2023-09-23T09:27:51.734264+0000","last_became_active":"2023-09-23T09:27:50.275565+0000","last_became_peered":"2023-09-23T09:27:50.275565+0000","last_unstale":"2023-09-23T09:27:51.734264+0000","last_undegraded":"2023-09-23T09:27:51.734264+0000","last_fullsized":"2023-09-23T09:27:51.734264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:57:37.004604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264041+0000","last_change":"2023-09-23T09:27:50.275911+0000","last_active":"2023-09-23T09:27:51.264041+0000","last_peered":"2023-09-23T09:27:51.264041+0000","last_clean":"2023-09-23T09:27:51.264041+0000","last_became_active":"2023-09-23T09:27:50.275626+0000","last_became_peered":"2023-09-23T09:27:50.275626+0000","last_unstale":"2023-09-23T09:27:51.264041+0000","last_undegraded":"2023-09-23T09:27:51.264041+0000","last_fullsized":"2023-09-23T09:27:51.264041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:46.790012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264037+0000","last_change":"2023-09-23T09:27:50.269483+0000","last_active":"2023-09-23T09:27:51.264037+0000","last_peered":"2023-09-23T09:27:51.264037+0000","last_clean":"2023-09-23T09:27:51.264037+0000","last_became_active":"2023-09-23T09:27:50.269199+0000","last_became_peered":"2023-09-23T09:27:50.269199+0000","last_unstale":"2023-09-23T09:27:51.264037+0000","last_undegraded":"2023-09-23T09:27:51.264037+0000","last_fullsized":"2023-09-23T09:27:51.264037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:58:34.304439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733479+0000","last_change":"2023-09-23T09:27:50.267917+0000","last_active":"2023-09-23T09:27:51.733479+0000","last_peered":"2023-09-23T09:27:51.733479+0000","last_clean":"2023-09-23T09:27:51.733479+0000","last_became_active":"2023-09-23T09:27:50.267036+0000","last_became_peered":"2023-09-23T09:27:50.267036+0000","last_unstale":"2023-09-23T09:27:51.733479+0000","last_undegraded":"2023-09-23T09:27:51.733479+0000","last_fullsized":"2023-09-23T09:27:51.733479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:42.866260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735091+0000","last_change":"2023-09-23T09:27:50.269934+0000","last_active":"2023-09-23T09:27:51.735091+0000","last_peered":"2023-09-23T09:27:51.735091+0000","last_clean":"2023-09-23T09:27:51.735091+0000","last_became_active":"2023-09-23T09:27:50.269060+0000","last_became_peered":"2023-09-23T09:27:50.269060+0000","last_unstale":"2023-09-23T09:27:51.735091+0000","last_undegraded":"2023-09-23T09:27:51.735091+0000","last_fullsized":"2023-09-23T09:27:51.735091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:47:31.191479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733829+0000","last_change":"2023-09-23T09:27:50.277885+0000","last_active":"2023-09-23T09:27:51.733829+0000","last_peered":"2023-09-23T09:27:51.733829+0000","last_clean":"2023-09-23T09:27:51.733829+0000","last_became_active":"2023-09-23T09:27:50.277576+0000","last_became_peered":"2023-09-23T09:27:50.277576+0000","last_unstale":"2023-09-23T09:27:51.733829+0000","last_undegraded":"2023-09-23T09:27:51.733829+0000","last_fullsized":"2023-09-23T09:27:51.733829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:03:43.338185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264326+0000","last_change":"2023-09-23T09:27:50.278225+0000","last_active":"2023-09-23T09:27:51.264326+0000","last_peered":"2023-09-23T09:27:51.264326+0000","last_clean":"2023-09-23T09:27:51.264326+0000","last_became_active":"2023-09-23T09:27:50.277891+0000","last_became_peered":"2023-09-23T09:27:50.277891+0000","last_unstale":"2023-09-23T09:27:51.264326+0000","last_undegraded":"2023-09-23T09:27:51.264326+0000","last_fullsized":"2023-09-23T09:27:51.264326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:27:08.961439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734854+0000","last_change":"2023-09-23T09:27:50.273445+0000","last_active":"2023-09-23T09:27:51.734854+0000","last_peered":"2023-09-23T09:27:51.734854+0000","last_clean":"2023-09-23T09:27:51.734854+0000","last_became_active":"2023-09-23T09:27:50.273124+0000","last_became_peered":"2023-09-23T09:27:50.273124+0000","last_unstale":"2023-09-23T09:27:51.734854+0000","last_undegraded":"2023-09-23T09:27:51.734854+0000","last_fullsized":"2023-09-23T09:27:51.734854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:46.030212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734647+0000","last_change":"2023-09-23T09:27:50.276164+0000","last_active":"2023-09-23T09:27:51.734647+0000","last_peered":"2023-09-23T09:27:51.734647+0000","last_clean":"2023-09-23T09:27:51.734647+0000","last_became_active":"2023-09-23T09:27:50.275860+0000","last_became_peered":"2023-09-23T09:27:50.275860+0000","last_unstale":"2023-09-23T09:27:51.734647+0000","last_undegraded":"2023-09-23T09:27:51.734647+0000","last_fullsized":"2023-09-23T09:27:51.734647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:53:50.521872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734518+0000","last_change":"2023-09-23T09:27:50.275221+0000","last_active":"2023-09-23T09:27:51.734518+0000","last_peered":"2023-09-23T09:27:51.734518+0000","last_clean":"2023-09-23T09:27:51.734518+0000","last_became_active":"2023-09-23T09:27:50.274920+0000","last_became_peered":"2023-09-23T09:27:50.274920+0000","last_unstale":"2023-09-23T09:27:51.734518+0000","last_undegraded":"2023-09-23T09:27:51.734518+0000","last_fullsized":"2023-09-23T09:27:51.734518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:45:35.023433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259534+0000","last_change":"2023-09-23T09:27:50.277776+0000","last_active":"2023-09-23T09:27:51.259534+0000","last_peered":"2023-09-23T09:27:51.259534+0000","last_clean":"2023-09-23T09:27:51.259534+0000","last_became_active":"2023-09-23T09:27:50.277291+0000","last_became_peered":"2023-09-23T09:27:50.277291+0000","last_unstale":"2023-09-23T09:27:51.259534+0000","last_undegraded":"2023-09-23T09:27:51.259534+0000","last_fullsized":"2023-09-23T09:27:51.259534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:23:46.984450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734180+0000","last_change":"2023-09-23T09:27:50.271219+0000","last_active":"2023-09-23T09:27:51.734180+0000","last_peered":"2023-09-23T09:27:51.734180+0000","last_clean":"2023-09-23T09:27:51.734180+0000","last_became_active":"2023-09-23T09:27:50.270747+0000","last_became_peered":"2023-09-23T09:27:50.270747+0000","last_unstale":"2023-09-23T09:27:51.734180+0000","last_undegraded":"2023-09-23T09:27:51.734180+0000","last_fullsized":"2023-09-23T09:27:51.734180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:19.237260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733913+0000","last_change":"2023-09-23T09:27:50.263794+0000","last_active":"2023-09-23T09:27:51.733913+0000","last_peered":"2023-09-23T09:27:51.733913+0000","last_clean":"2023-09-23T09:27:51.733913+0000","last_became_active":"2023-09-23T09:27:50.263459+0000","last_became_peered":"2023-09-23T09:27:50.263459+0000","last_unstale":"2023-09-23T09:27:51.733913+0000","last_undegraded":"2023-09-23T09:27:51.733913+0000","last_fullsized":"2023-09-23T09:27:51.733913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:05:37.618959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250071,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61384,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":230,"num_read_kb":180,"num_write":149,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":184152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":427264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.376},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.40999999999999998}]},{"osd":1,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.49399999999999999}]},{"osd":2,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.52900000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.443},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.71599999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.74199999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.72699999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678040,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1076135,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.55200000000000005}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.625}]},{"osd":9,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67100000000000004}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70699999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":427500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.32800000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51500000000000001}]},{"osd":3,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.313}]},{"osd":4,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.499}]},{"osd":5,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.39000000000000001}]},{"osd":7,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.47199999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.54100000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.61199999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":427420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.629},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.34699999999999998}]},{"osd":1,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.745}]},{"osd":2,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.745},"last":0.37},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.67400000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48999999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95099999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.71699999999999997}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76800000000000002}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1019472,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41199999999999998,"15min":0.40400000000000003},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.434},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.311,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.81000000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.53700000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99299999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79100000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.71299999999999997}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.752},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":426895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.41199999999999998},"min":{"1min":0.33300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.30099999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.33900000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.499,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.51200000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.496}]},{"osd":2,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.57099999999999995},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98399999999999999,"5min":1.006,"15min":1.006},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.36299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58699999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.79200000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":1.077,"15min":1.077},"last":0.76000000000000001}]},{"osd":5,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.61299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.56699999999999995,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.44800000000000001}]},{"osd":7,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.61899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.78600000000000003}]},{"osd":8,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":6.9850000000000003,"5min":6.9850000000000003,"15min":6.9850000000000003},"last":0.623},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.60199999999999998}]},{"osd":9,"last update":"Sat Sep 23 09:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71599999999999997}]},{"osd":10,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.66500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":483676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.37,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.503,"15min":0.48399999999999999},"min":{"1min":0.38700000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.53500000000000003,"15min":0.505},"min":{"1min":0.40899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.82999999999999996}]},{"osd":3,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.29499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.70699999999999996}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.872,"5min":0.877,"15min":0.877},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.91900000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.89100000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.64900000000000002,"15min":0.59699999999999998},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.7679999999999998,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64800000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.75600000000000001}]},{"osd":8,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88700000000000001,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91700000000000004,"5min":1.01,"15min":1.01},"last":0.96899999999999997}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.98599999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":428160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 09:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.55100000000000005,"15min":0.51300000000000001},"min":{"1min":0.46400000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.439,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.58099999999999996,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.81000000000000005}]},{"osd":3,"last update":"Sat Sep 23 09:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.26400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81100000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.57099999999999995,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.51300000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.97299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.718,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.718},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":8,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0649999999999999,"5min":1.101,"15min":1.101},"last":0.93600000000000005}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.90200000000000002}]},{"osd":10,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.60599999999999998}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.216}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":485416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.71199999999999997,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.32800000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.78000000000000003,"5min":1.655,"15min":1.655},"last":0.53700000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.505},"min":{"1min":0.35899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67700000000000005}]},{"osd":3,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.626},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.80800000000000005}]},{"osd":4,"last update":"Sat Sep 23 09:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73999999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.89300000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.46000000000000002,"15min":0.442},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.29499999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.57899999999999996}]},{"osd":6,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85699999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.014,"15min":1.014},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95899999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.64200000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.85699999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":426881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.36499999999999999,"15min":0.34399999999999997},"min":{"1min":0.29299999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.35799999999999998},"min":{"1min":0.26600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.28199999999999997}]},{"osd":1,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.438,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60699999999999998,"15min":0.58599999999999997},"min":{"1min":0.48499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.502},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.55300000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":1,"15min":1},"last":0.70799999999999996}]},{"osd":6,"last update":"Sat Sep 23 09:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.499,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.375}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.48799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.71099999999999997,"15min":0.69999999999999996},"min":{"1min":0.628,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.437},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54000000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.78800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":427716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.998,"15min":0.998},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.38},"min":{"1min":0.33900000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70599999999999996}]},{"osd":3,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.495,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.626}]},{"osd":5,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61299999999999999,"15min":0.58199999999999996},"min":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.877,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.58599999999999997,"15min":0.56599999999999995},"min":{"1min":0.44400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.496},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73499999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92300000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98399999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78300000000000003}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.76500000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.751}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":1019039,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.55300000000000005,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.52000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.51000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.92100000000000004}]},{"osd":2,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.56399999999999995,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.872},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.163,"5min":1.538,"15min":1.538},"last":0.85599999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.45400000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.83799999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.54800000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.81299999999999994}]},{"osd":8,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88100000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86199999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76700000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.752},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.97999999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T09:28:34.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph pg dump --format=json 2023-09-23T09:28:34.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:34 smithi098 ceph-mon[102386]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:34 smithi123 ceph-mon[103577]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:35.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:35 smithi045 ceph-mon[98627]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:35.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:35 smithi098 ceph-mon[102386]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:35 smithi123 ceph-mon[103577]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.285+0000 7f87052a7700 1 -- 172.21.15.45:0/2246304145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700100940 msgr2=0x7f8700104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.285+0000 7f87052a7700 1 --2- 172.21.15.45:0/2246304145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700100940 0x7f8700104990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009990 tx=0x7f86f4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.285+0000 7f87052a7700 1 -- 172.21.15.45:0/2246304145 shutdown_connections 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.285+0000 7f87052a7700 1 --2- 172.21.15.45:0/2246304145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700105070 0x7f870010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.285+0000 7f87052a7700 1 --2- 172.21.15.45:0/2246304145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87000fff90 0x7f8700100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.285+0000 7f87052a7700 1 --2- 172.21.15.45:0/2246304145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700100940 0x7f8700104990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.285+0000 7f87052a7700 1 -- 172.21.15.45:0/2246304145 >> 172.21.15.45:0/2246304145 conn(0x7f87000fb820 msgr2=0x7f87000fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.286+0000 7f87052a7700 1 -- 172.21.15.45:0/2246304145 shutdown_connections 2023-09-23T09:28:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.286+0000 7f87052a7700 1 -- 172.21.15.45:0/2246304145 wait complete. 2023-09-23T09:28:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.286+0000 7f87052a7700 1 Processor -- start 2023-09-23T09:28:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f87052a7700 1 -- start start 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f87052a7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87000fff90 0x7f8700197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f87052a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8700100940 0x7f87001982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f87052a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700105070 0x7f87001a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f87052a7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8700198be0 con 0x7f8700105070 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f87052a7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001a3320 con 0x7f87000fff90 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f87052a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001a34e0 con 0x7f8700100940 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700105070 0x7f87001a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700105070 0x7f87001a2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59426/0 (socket says 172.21.15.45:59426) 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f86ff7fe700 1 -- 172.21.15.45:0/351113092 learned_addr learned my addr 172.21.15.45:0/351113092 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f86feffd700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87000fff90 0x7f8700197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.287+0000 7f86fe7fc700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8700100940 0x7f87001982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86ff7fe700 1 -- 172.21.15.45:0/351113092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8700100940 msgr2=0x7f87001982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86ff7fe700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8700100940 0x7f87001982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86ff7fe700 1 -- 172.21.15.45:0/351113092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87000fff90 msgr2=0x7f8700197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86ff7fe700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87000fff90 0x7f8700197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86ff7fe700 1 -- 172.21.15.45:0/351113092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40095f0 con 0x7f8700105070 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86fe7fc700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8700100940 0x7f87001982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86ff7fe700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700105070 0x7f87001a2de0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f86f8007ee0 tx=0x7f86f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86e7fff700 1 -- 172.21.15.45:0/351113092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8017070 con 0x7f8700105070 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87001a37c0 con 0x7f8700105070 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86e7fff700 1 -- 172.21.15.45:0/351113092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86f800f040 con 0x7f8700105070 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f86e7fff700 1 -- 172.21.15.45:0/351113092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8005740 con 0x7f8700105070 2023-09-23T09:28:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.288+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a3c70 con 0x7f8700105070 2023-09-23T09:28:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.289+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870004eb20 con 0x7f8700105070 2023-09-23T09:28:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.290+0000 7f86e7fff700 1 -- 172.21.15.45:0/351113092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f86f800dc60 con 0x7f8700105070 2023-09-23T09:28:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.291+0000 7f86e7fff700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f86e80a1630 0x7f86e80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.291+0000 7f86feffd700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f86e80a1630 0x7f86e80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.291+0000 7f86e7fff700 1 -- 172.21.15.45:0/351113092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f86f80ccde0 con 0x7f8700105070 2023-09-23T09:28:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.291+0000 7f86feffd700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f86e80a1630 0x7f86e80a3af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f86f0004630 tx=0x7f86f000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.294+0000 7f86e7fff700 1 -- 172.21.15.45:0/351113092 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f86f8096320 con 0x7f8700105070 2023-09-23T09:28:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.495+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87000698b0 con 0x7f86e80a1630 2023-09-23T09:28:36.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.507+0000 7f86e7fff700 1 -- 172.21.15.45:0/351113092 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340752 (secure 0 0 0) 0x7f87000698b0 con 0x7f86e80a1630 2023-09-23T09:28:36.513 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f86e80a1630 msgr2=0x7f86e80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f86e80a1630 0x7f86e80a3af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f86f0004630 tx=0x7f86f000b3c0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700105070 msgr2=0x7f87001a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700105070 0x7f87001a2de0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f86f8007ee0 tx=0x7f86f800d3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 shutdown_connections 2023-09-23T09:28:36.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8700105070 0x7f87001a2de0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87000fff90 0x7f8700197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f86e80a1630 0x7f86e80a3af0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.510+0000 7f87052a7700 1 --2- 172.21.15.45:0/351113092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8700100940 0x7f87001982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.511+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 >> 172.21.15.45:0/351113092 conn(0x7f87000fb820 msgr2=0x7f87000fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.511+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 shutdown_connections 2023-09-23T09:28:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:36.511+0000 7f87052a7700 1 -- 172.21.15.45:0/351113092 wait complete. 2023-09-23T09:28:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-09-23T09:28:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-09-23T09:28:36.333182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651998,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":396,"num_read_kb":314,"num_write":332,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335164,"kb_used_data":10972,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606660,"statfs":{"total":1151940427776,"available":1151597219840,"internally_reserved":0,"allocated":11235328,"data_stored":7075574,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003163"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735132+0000","last_change":"2023-09-23T09:27:50.270599+0000","last_active":"2023-09-23T09:27:51.735132+0000","last_peered":"2023-09-23T09:27:51.735132+0000","last_clean":"2023-09-23T09:27:51.735132+0000","last_became_active":"2023-09-23T09:27:50.269926+0000","last_became_peered":"2023-09-23T09:27:50.269926+0000","last_unstale":"2023-09-23T09:27:51.735132+0000","last_undegraded":"2023-09-23T09:27:51.735132+0000","last_fullsized":"2023-09-23T09:27:51.735132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:12:08.908188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733803+0000","last_change":"2023-09-23T09:27:50.270451+0000","last_active":"2023-09-23T09:27:51.733803+0000","last_peered":"2023-09-23T09:27:51.733803+0000","last_clean":"2023-09-23T09:27:51.733803+0000","last_became_active":"2023-09-23T09:27:50.270055+0000","last_became_peered":"2023-09-23T09:27:50.270055+0000","last_unstale":"2023-09-23T09:27:51.733803+0000","last_undegraded":"2023-09-23T09:27:51.733803+0000","last_fullsized":"2023-09-23T09:27:51.733803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:56:43.388109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734671+0000","last_change":"2023-09-23T09:27:50.278101+0000","last_active":"2023-09-23T09:27:51.734671+0000","last_peered":"2023-09-23T09:27:51.734671+0000","last_clean":"2023-09-23T09:27:51.734671+0000","last_became_active":"2023-09-23T09:27:50.277739+0000","last_became_peered":"2023-09-23T09:27:50.277739+0000","last_unstale":"2023-09-23T09:27:51.734671+0000","last_undegraded":"2023-09-23T09:27:51.734671+0000","last_fullsized":"2023-09-23T09:27:51.734671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:52:31.582623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733930+0000","last_change":"2023-09-23T09:27:50.265962+0000","last_active":"2023-09-23T09:27:51.733930+0000","last_peered":"2023-09-23T09:27:51.733930+0000","last_clean":"2023-09-23T09:27:51.733930+0000","last_became_active":"2023-09-23T09:27:50.265681+0000","last_became_peered":"2023-09-23T09:27:50.265681+0000","last_unstale":"2023-09-23T09:27:51.733930+0000","last_undegraded":"2023-09-23T09:27:51.733930+0000","last_fullsized":"2023-09-23T09:27:51.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:12.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734705+0000","last_change":"2023-09-23T09:27:50.270356+0000","last_active":"2023-09-23T09:27:51.734705+0000","last_peered":"2023-09-23T09:27:51.734705+0000","last_clean":"2023-09-23T09:27:51.734705+0000","last_became_active":"2023-09-23T09:27:50.270124+0000","last_became_peered":"2023-09-23T09:27:50.270124+0000","last_unstale":"2023-09-23T09:27:51.734705+0000","last_undegraded":"2023-09-23T09:27:51.734705+0000","last_fullsized":"2023-09-23T09:27:51.734705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:14:00.508926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398274+0000","last_change":"2023-09-23T09:27:50.276122+0000","last_active":"2023-09-23T09:27:51.398274+0000","last_peered":"2023-09-23T09:27:51.398274+0000","last_clean":"2023-09-23T09:27:51.398274+0000","last_became_active":"2023-09-23T09:27:50.275847+0000","last_became_peered":"2023-09-23T09:27:50.275847+0000","last_unstale":"2023-09-23T09:27:51.398274+0000","last_undegraded":"2023-09-23T09:27:51.398274+0000","last_fullsized":"2023-09-23T09:27:51.398274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:04:03.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734273+0000","last_change":"2023-09-23T09:27:50.279420+0000","last_active":"2023-09-23T09:27:51.734273+0000","last_peered":"2023-09-23T09:27:51.734273+0000","last_clean":"2023-09-23T09:27:51.734273+0000","last_became_active":"2023-09-23T09:27:50.279120+0000","last_became_peered":"2023-09-23T09:27:50.279120+0000","last_unstale":"2023-09-23T09:27:51.734273+0000","last_undegraded":"2023-09-23T09:27:51.734273+0000","last_fullsized":"2023-09-23T09:27:51.734273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:48:03.461868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734300+0000","last_change":"2023-09-23T09:27:50.276364+0000","last_active":"2023-09-23T09:27:51.734300+0000","last_peered":"2023-09-23T09:27:51.734300+0000","last_clean":"2023-09-23T09:27:51.734300+0000","last_became_active":"2023-09-23T09:27:50.276060+0000","last_became_peered":"2023-09-23T09:27:50.276060+0000","last_unstale":"2023-09-23T09:27:51.734300+0000","last_undegraded":"2023-09-23T09:27:51.734300+0000","last_fullsized":"2023-09-23T09:27:51.734300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:26.998609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259923+0000","last_change":"2023-09-23T09:27:50.280302+0000","last_active":"2023-09-23T09:27:51.259923+0000","last_peered":"2023-09-23T09:27:51.259923+0000","last_clean":"2023-09-23T09:27:51.259923+0000","last_became_active":"2023-09-23T09:27:50.279295+0000","last_became_peered":"2023-09-23T09:27:50.279295+0000","last_unstale":"2023-09-23T09:27:51.259923+0000","last_undegraded":"2023-09-23T09:27:51.259923+0000","last_fullsized":"2023-09-23T09:27:51.259923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:15.329483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398526+0000","last_change":"2023-09-23T09:27:50.271876+0000","last_active":"2023-09-23T09:27:51.398526+0000","last_peered":"2023-09-23T09:27:51.398526+0000","last_clean":"2023-09-23T09:27:51.398526+0000","last_became_active":"2023-09-23T09:27:50.271576+0000","last_became_peered":"2023-09-23T09:27:50.271576+0000","last_unstale":"2023-09-23T09:27:51.398526+0000","last_undegraded":"2023-09-23T09:27:51.398526+0000","last_fullsized":"2023-09-23T09:27:51.398526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:17:22.656828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734649+0000","last_change":"2023-09-23T09:27:50.271405+0000","last_active":"2023-09-23T09:27:51.734649+0000","last_peered":"2023-09-23T09:27:51.734649+0000","last_clean":"2023-09-23T09:27:51.734649+0000","last_became_active":"2023-09-23T09:27:50.271087+0000","last_became_peered":"2023-09-23T09:27:50.271087+0000","last_unstale":"2023-09-23T09:27:51.734649+0000","last_undegraded":"2023-09-23T09:27:51.734649+0000","last_fullsized":"2023-09-23T09:27:51.734649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:09.363894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733876+0000","last_change":"2023-09-23T09:27:50.265855+0000","last_active":"2023-09-23T09:27:51.733876+0000","last_peered":"2023-09-23T09:27:51.733876+0000","last_clean":"2023-09-23T09:27:51.733876+0000","last_became_active":"2023-09-23T09:27:50.265101+0000","last_became_peered":"2023-09-23T09:27:50.265101+0000","last_unstale":"2023-09-23T09:27:51.733876+0000","last_undegraded":"2023-09-23T09:27:51.733876+0000","last_fullsized":"2023-09-23T09:27:51.733876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:52:40.456013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734201+0000","last_change":"2023-09-23T09:27:50.274405+0000","last_active":"2023-09-23T09:27:51.734201+0000","last_peered":"2023-09-23T09:27:51.734201+0000","last_clean":"2023-09-23T09:27:51.734201+0000","last_became_active":"2023-09-23T09:27:50.274036+0000","last_became_peered":"2023-09-23T09:27:50.274036+0000","last_unstale":"2023-09-23T09:27:51.734201+0000","last_undegraded":"2023-09-23T09:27:51.734201+0000","last_fullsized":"2023-09-23T09:27:51.734201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:38.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733331+0000","last_change":"2023-09-23T09:27:50.272237+0000","last_active":"2023-09-23T09:27:51.733331+0000","last_peered":"2023-09-23T09:27:51.733331+0000","last_clean":"2023-09-23T09:27:51.733331+0000","last_became_active":"2023-09-23T09:27:50.271897+0000","last_became_peered":"2023-09-23T09:27:50.271897+0000","last_unstale":"2023-09-23T09:27:51.733331+0000","last_undegraded":"2023-09-23T09:27:51.733331+0000","last_fullsized":"2023-09-23T09:27:51.733331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:31:31.115205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733452+0000","last_change":"2023-09-23T09:27:50.268902+0000","last_active":"2023-09-23T09:27:51.733452+0000","last_peered":"2023-09-23T09:27:51.733452+0000","last_clean":"2023-09-23T09:27:51.733452+0000","last_became_active":"2023-09-23T09:27:50.268590+0000","last_became_peered":"2023-09-23T09:27:50.268590+0000","last_unstale":"2023-09-23T09:27:51.733452+0000","last_undegraded":"2023-09-23T09:27:51.733452+0000","last_fullsized":"2023-09-23T09:27:51.733452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:54:39.384835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259359+0000","last_change":"2023-09-23T09:27:50.276934+0000","last_active":"2023-09-23T09:27:51.259359+0000","last_peered":"2023-09-23T09:27:51.259359+0000","last_clean":"2023-09-23T09:27:51.259359+0000","last_became_active":"2023-09-23T09:27:50.276645+0000","last_became_peered":"2023-09-23T09:27:50.276645+0000","last_unstale":"2023-09-23T09:27:51.259359+0000","last_undegraded":"2023-09-23T09:27:51.259359+0000","last_fullsized":"2023-09-23T09:27:51.259359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:07.100550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735229+0000","last_change":"2023-09-23T09:27:50.270781+0000","last_active":"2023-09-23T09:27:51.735229+0000","last_peered":"2023-09-23T09:27:51.735229+0000","last_clean":"2023-09-23T09:27:51.735229+0000","last_became_active":"2023-09-23T09:27:50.270319+0000","last_became_peered":"2023-09-23T09:27:50.270319+0000","last_unstale":"2023-09-23T09:27:51.735229+0000","last_undegraded":"2023-09-23T09:27:51.735229+0000","last_fullsized":"2023-09-23T09:27:51.735229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:42:44.882122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259692+0000","last_change":"2023-09-23T09:27:50.278775+0000","last_active":"2023-09-23T09:27:51.259692+0000","last_peered":"2023-09-23T09:27:51.259692+0000","last_clean":"2023-09-23T09:27:51.259692+0000","last_became_active":"2023-09-23T09:27:50.278375+0000","last_became_peered":"2023-09-23T09:27:50.278375+0000","last_unstale":"2023-09-23T09:27:51.259692+0000","last_undegraded":"2023-09-23T09:27:51.259692+0000","last_fullsized":"2023-09-23T09:27:51.259692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:14:00.076810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264221+0000","last_change":"2023-09-23T09:27:50.270141+0000","last_active":"2023-09-23T09:27:51.264221+0000","last_peered":"2023-09-23T09:27:51.264221+0000","last_clean":"2023-09-23T09:27:51.264221+0000","last_became_active":"2023-09-23T09:27:50.269849+0000","last_became_peered":"2023-09-23T09:27:50.269849+0000","last_unstale":"2023-09-23T09:27:51.264221+0000","last_undegraded":"2023-09-23T09:27:51.264221+0000","last_fullsized":"2023-09-23T09:27:51.264221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:49:41.837886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704055+0000","last_change":"2023-09-23T09:27:49.654458+0000","last_active":"2023-09-23T09:27:54.704055+0000","last_peered":"2023-09-23T09:27:54.704055+0000","last_clean":"2023-09-23T09:27:54.704055+0000","last_became_active":"2023-09-23T09:27:49.654084+0000","last_became_peered":"2023-09-23T09:27:49.654084+0000","last_unstale":"2023-09-23T09:27:54.704055+0000","last_undegraded":"2023-09-23T09:27:54.704055+0000","last_fullsized":"2023-09-23T09:27:54.704055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:47:32.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734956+0000","last_change":"2023-09-23T09:27:50.276558+0000","last_active":"2023-09-23T09:27:51.734956+0000","last_peered":"2023-09-23T09:27:51.734956+0000","last_clean":"2023-09-23T09:27:51.734956+0000","last_became_active":"2023-09-23T09:27:50.276255+0000","last_became_peered":"2023-09-23T09:27:50.276255+0000","last_unstale":"2023-09-23T09:27:51.734956+0000","last_undegraded":"2023-09-23T09:27:51.734956+0000","last_fullsized":"2023-09-23T09:27:51.734956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:13:09.870383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735166+0000","last_change":"2023-09-23T09:27:49.659256+0000","last_active":"2023-09-23T09:27:51.735166+0000","last_peered":"2023-09-23T09:27:51.735166+0000","last_clean":"2023-09-23T09:27:51.735166+0000","last_became_active":"2023-09-23T09:27:49.658793+0000","last_became_peered":"2023-09-23T09:27:49.658793+0000","last_unstale":"2023-09-23T09:27:51.735166+0000","last_undegraded":"2023-09-23T09:27:51.735166+0000","last_fullsized":"2023-09-23T09:27:51.735166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:03.416666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733674+0000","last_change":"2023-09-23T09:27:50.277338+0000","last_active":"2023-09-23T09:27:51.733674+0000","last_peered":"2023-09-23T09:27:51.733674+0000","last_clean":"2023-09-23T09:27:51.733674+0000","last_became_active":"2023-09-23T09:27:50.277011+0000","last_became_peered":"2023-09-23T09:27:50.277011+0000","last_unstale":"2023-09-23T09:27:51.733674+0000","last_undegraded":"2023-09-23T09:27:51.733674+0000","last_fullsized":"2023-09-23T09:27:51.733674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:52:23.028794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702688+0000","last_change":"2023-09-23T09:27:49.660679+0000","last_active":"2023-09-23T09:27:54.702688+0000","last_peered":"2023-09-23T09:27:54.702688+0000","last_clean":"2023-09-23T09:27:54.702688+0000","last_became_active":"2023-09-23T09:27:49.660201+0000","last_became_peered":"2023-09-23T09:27:49.660201+0000","last_unstale":"2023-09-23T09:27:54.702688+0000","last_undegraded":"2023-09-23T09:27:54.702688+0000","last_fullsized":"2023-09-23T09:27:54.702688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:33:56.205322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259494+0000","last_change":"2023-09-23T09:27:50.278373+0000","last_active":"2023-09-23T09:27:51.259494+0000","last_peered":"2023-09-23T09:27:51.259494+0000","last_clean":"2023-09-23T09:27:51.259494+0000","last_became_active":"2023-09-23T09:27:50.277837+0000","last_became_peered":"2023-09-23T09:27:50.277837+0000","last_unstale":"2023-09-23T09:27:51.259494+0000","last_undegraded":"2023-09-23T09:27:51.259494+0000","last_fullsized":"2023-09-23T09:27:51.259494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:24:26.914509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.117099+0000","last_change":"2023-09-23T09:27:49.657578+0000","last_active":"2023-09-23T09:27:51.117099+0000","last_peered":"2023-09-23T09:27:51.117099+0000","last_clean":"2023-09-23T09:27:51.117099+0000","last_became_active":"2023-09-23T09:27:49.657278+0000","last_became_peered":"2023-09-23T09:27:49.657278+0000","last_unstale":"2023-09-23T09:27:51.117099+0000","last_undegraded":"2023-09-23T09:27:51.117099+0000","last_fullsized":"2023-09-23T09:27:51.117099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:53.440997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734804+0000","last_change":"2023-09-23T09:27:50.279784+0000","last_active":"2023-09-23T09:27:51.734804+0000","last_peered":"2023-09-23T09:27:51.734804+0000","last_clean":"2023-09-23T09:27:51.734804+0000","last_became_active":"2023-09-23T09:27:50.279473+0000","last_became_peered":"2023-09-23T09:27:50.279473+0000","last_unstale":"2023-09-23T09:27:51.734804+0000","last_undegraded":"2023-09-23T09:27:51.734804+0000","last_fullsized":"2023-09-23T09:27:51.734804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:30:13.010556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:32.322495+0000","last_change":"2023-09-23T09:27:49.655391+0000","last_active":"2023-09-23T09:28:32.322495+0000","last_peered":"2023-09-23T09:28:32.322495+0000","last_clean":"2023-09-23T09:28:32.322495+0000","last_became_active":"2023-09-23T09:27:49.655067+0000","last_became_peered":"2023-09-23T09:27:49.655067+0000","last_unstale":"2023-09-23T09:28:32.322495+0000","last_undegraded":"2023-09-23T09:28:32.322495+0000","last_fullsized":"2023-09-23T09:28:32.322495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.701471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733903+0000","last_change":"2023-09-23T09:27:50.275688+0000","last_active":"2023-09-23T09:27:51.733903+0000","last_peered":"2023-09-23T09:27:51.733903+0000","last_clean":"2023-09-23T09:27:51.733903+0000","last_became_active":"2023-09-23T09:27:50.275356+0000","last_became_peered":"2023-09-23T09:27:50.275356+0000","last_unstale":"2023-09-23T09:27:51.733903+0000","last_undegraded":"2023-09-23T09:27:51.733903+0000","last_fullsized":"2023-09-23T09:27:51.733903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:43:31.829209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702167+0000","last_change":"2023-09-23T09:27:49.657066+0000","last_active":"2023-09-23T09:27:54.702167+0000","last_peered":"2023-09-23T09:27:54.702167+0000","last_clean":"2023-09-23T09:27:54.702167+0000","last_became_active":"2023-09-23T09:27:49.656754+0000","last_became_peered":"2023-09-23T09:27:49.656754+0000","last_unstale":"2023-09-23T09:27:54.702167+0000","last_undegraded":"2023-09-23T09:27:54.702167+0000","last_fullsized":"2023-09-23T09:27:54.702167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:52.277474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734458+0000","last_change":"2023-09-23T09:27:50.273705+0000","last_active":"2023-09-23T09:27:51.734458+0000","last_peered":"2023-09-23T09:27:51.734458+0000","last_clean":"2023-09-23T09:27:51.734458+0000","last_became_active":"2023-09-23T09:27:50.272245+0000","last_became_peered":"2023-09-23T09:27:50.272245+0000","last_unstale":"2023-09-23T09:27:51.734458+0000","last_undegraded":"2023-09-23T09:27:51.734458+0000","last_fullsized":"2023-09-23T09:27:51.734458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:45:57.318326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704059+0000","last_change":"2023-09-23T09:27:49.651958+0000","last_active":"2023-09-23T09:27:54.704059+0000","last_peered":"2023-09-23T09:27:54.704059+0000","last_clean":"2023-09-23T09:27:54.704059+0000","last_became_active":"2023-09-23T09:27:49.651058+0000","last_became_peered":"2023-09-23T09:27:49.651058+0000","last_unstale":"2023-09-23T09:27:54.704059+0000","last_undegraded":"2023-09-23T09:27:54.704059+0000","last_fullsized":"2023-09-23T09:27:54.704059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:26:56.360566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734149+0000","last_change":"2023-09-23T09:27:50.270804+0000","last_active":"2023-09-23T09:27:51.734149+0000","last_peered":"2023-09-23T09:27:51.734149+0000","last_clean":"2023-09-23T09:27:51.734149+0000","last_became_active":"2023-09-23T09:27:50.270044+0000","last_became_peered":"2023-09-23T09:27:50.270044+0000","last_unstale":"2023-09-23T09:27:51.734149+0000","last_undegraded":"2023-09-23T09:27:51.734149+0000","last_fullsized":"2023-09-23T09:27:51.734149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:59:20.821949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734563+0000","last_change":"2023-09-23T09:27:49.657030+0000","last_active":"2023-09-23T09:27:51.734563+0000","last_peered":"2023-09-23T09:27:51.734563+0000","last_clean":"2023-09-23T09:27:51.734563+0000","last_became_active":"2023-09-23T09:27:49.656710+0000","last_became_peered":"2023-09-23T09:27:49.656710+0000","last_unstale":"2023-09-23T09:27:51.734563+0000","last_undegraded":"2023-09-23T09:27:51.734563+0000","last_fullsized":"2023-09-23T09:27:51.734563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:24:59.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259415+0000","last_change":"2023-09-23T09:27:50.271688+0000","last_active":"2023-09-23T09:27:51.259415+0000","last_peered":"2023-09-23T09:27:51.259415+0000","last_clean":"2023-09-23T09:27:51.259415+0000","last_became_active":"2023-09-23T09:27:50.271276+0000","last_became_peered":"2023-09-23T09:27:50.271276+0000","last_unstale":"2023-09-23T09:27:51.259415+0000","last_undegraded":"2023-09-23T09:27:51.259415+0000","last_fullsized":"2023-09-23T09:27:51.259415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:40:48.333279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:32.321814+0000","last_change":"2023-09-23T09:27:49.655183+0000","last_active":"2023-09-23T09:28:32.321814+0000","last_peered":"2023-09-23T09:28:32.321814+0000","last_clean":"2023-09-23T09:28:32.321814+0000","last_became_active":"2023-09-23T09:27:49.654875+0000","last_became_peered":"2023-09-23T09:27:49.654875+0000","last_unstale":"2023-09-23T09:28:32.321814+0000","last_undegraded":"2023-09-23T09:28:32.321814+0000","last_fullsized":"2023-09-23T09:28:32.321814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:12:49.944732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259843+0000","last_change":"2023-09-23T09:27:50.276347+0000","last_active":"2023-09-23T09:27:51.259843+0000","last_peered":"2023-09-23T09:27:51.259843+0000","last_clean":"2023-09-23T09:27:51.259843+0000","last_became_active":"2023-09-23T09:27:50.276095+0000","last_became_peered":"2023-09-23T09:27:50.276095+0000","last_unstale":"2023-09-23T09:27:51.259843+0000","last_undegraded":"2023-09-23T09:27:51.259843+0000","last_fullsized":"2023-09-23T09:27:51.259843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:42.952808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.692327+0000","last_change":"2023-09-23T09:27:49.657115+0000","last_active":"2023-09-23T09:27:53.692327+0000","last_peered":"2023-09-23T09:27:53.692327+0000","last_clean":"2023-09-23T09:27:53.692327+0000","last_became_active":"2023-09-23T09:27:49.656816+0000","last_became_peered":"2023-09-23T09:27:49.656816+0000","last_unstale":"2023-09-23T09:27:53.692327+0000","last_undegraded":"2023-09-23T09:27:53.692327+0000","last_fullsized":"2023-09-23T09:27:53.692327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:37.205221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733619+0000","last_change":"2023-09-23T09:27:50.277477+0000","last_active":"2023-09-23T09:27:51.733619+0000","last_peered":"2023-09-23T09:27:51.733619+0000","last_clean":"2023-09-23T09:27:51.733619+0000","last_became_active":"2023-09-23T09:27:50.277200+0000","last_became_peered":"2023-09-23T09:27:50.277200+0000","last_unstale":"2023-09-23T09:27:51.733619+0000","last_undegraded":"2023-09-23T09:27:51.733619+0000","last_fullsized":"2023-09-23T09:27:51.733619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:45:03.167471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.691949+0000","last_change":"2023-09-23T09:27:49.663678+0000","last_active":"2023-09-23T09:27:53.691949+0000","last_peered":"2023-09-23T09:27:53.691949+0000","last_clean":"2023-09-23T09:27:53.691949+0000","last_became_active":"2023-09-23T09:27:49.663142+0000","last_became_peered":"2023-09-23T09:27:49.663142+0000","last_unstale":"2023-09-23T09:27:53.691949+0000","last_undegraded":"2023-09-23T09:27:53.691949+0000","last_fullsized":"2023-09-23T09:27:53.691949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:14.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259656+0000","last_change":"2023-09-23T09:27:50.275044+0000","last_active":"2023-09-23T09:27:51.259656+0000","last_peered":"2023-09-23T09:27:51.259656+0000","last_clean":"2023-09-23T09:27:51.259656+0000","last_became_active":"2023-09-23T09:27:50.274752+0000","last_became_peered":"2023-09-23T09:27:50.274752+0000","last_unstale":"2023-09-23T09:27:51.259656+0000","last_undegraded":"2023-09-23T09:27:51.259656+0000","last_fullsized":"2023-09-23T09:27:51.259656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:19.914752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:33.323181+0000","last_change":"2023-09-23T09:27:49.653694+0000","last_active":"2023-09-23T09:28:33.323181+0000","last_peered":"2023-09-23T09:28:33.323181+0000","last_clean":"2023-09-23T09:28:33.323181+0000","last_became_active":"2023-09-23T09:27:49.653355+0000","last_became_peered":"2023-09-23T09:27:49.653355+0000","last_unstale":"2023-09-23T09:28:33.323181+0000","last_undegraded":"2023-09-23T09:28:33.323181+0000","last_fullsized":"2023-09-23T09:28:33.323181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:19:53.282723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56994,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":85,"num_read_kb":97,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.272849+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.272411+0000","last_became_peered":"2023-09-23T09:27:50.272411+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:01:41.972357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.702769+0000","last_change":"2023-09-23T09:27:49.652769+0000","last_active":"2023-09-23T09:27:54.702769+0000","last_peered":"2023-09-23T09:27:54.702769+0000","last_clean":"2023-09-23T09:27:54.702769+0000","last_became_active":"2023-09-23T09:27:49.652440+0000","last_became_peered":"2023-09-23T09:27:49.652440+0000","last_unstale":"2023-09-23T09:27:54.702769+0000","last_undegraded":"2023-09-23T09:27:54.702769+0000","last_fullsized":"2023-09-23T09:27:54.702769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:02.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734000+0000","last_change":"2023-09-23T09:27:50.272105+0000","last_active":"2023-09-23T09:27:51.734000+0000","last_peered":"2023-09-23T09:27:51.734000+0000","last_clean":"2023-09-23T09:27:51.734000+0000","last_became_active":"2023-09-23T09:27:50.271766+0000","last_became_peered":"2023-09-23T09:27:50.271766+0000","last_unstale":"2023-09-23T09:27:51.734000+0000","last_undegraded":"2023-09-23T09:27:51.734000+0000","last_fullsized":"2023-09-23T09:27:51.734000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:46:46.012381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690285+0000","last_change":"2023-09-23T09:27:49.663473+0000","last_active":"2023-09-23T09:27:53.690285+0000","last_peered":"2023-09-23T09:27:53.690285+0000","last_clean":"2023-09-23T09:27:53.690285+0000","last_became_active":"2023-09-23T09:27:49.663085+0000","last_became_peered":"2023-09-23T09:27:49.663085+0000","last_unstale":"2023-09-23T09:27:53.690285+0000","last_undegraded":"2023-09-23T09:27:53.690285+0000","last_fullsized":"2023-09-23T09:27:53.690285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:28:05.246720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734657+0000","last_change":"2023-09-23T09:27:50.274172+0000","last_active":"2023-09-23T09:27:51.734657+0000","last_peered":"2023-09-23T09:27:51.734657+0000","last_clean":"2023-09-23T09:27:51.734657+0000","last_became_active":"2023-09-23T09:27:50.273874+0000","last_became_peered":"2023-09-23T09:27:50.273874+0000","last_unstale":"2023-09-23T09:27:51.734657+0000","last_undegraded":"2023-09-23T09:27:51.734657+0000","last_fullsized":"2023-09-23T09:27:51.734657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:04:06.864055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:31.320031+0000","last_change":"2023-09-23T09:27:49.654341+0000","last_active":"2023-09-23T09:28:31.320031+0000","last_peered":"2023-09-23T09:28:31.320031+0000","last_clean":"2023-09-23T09:28:31.320031+0000","last_became_active":"2023-09-23T09:27:49.653936+0000","last_became_peered":"2023-09-23T09:27:49.653936+0000","last_unstale":"2023-09-23T09:28:31.320031+0000","last_undegraded":"2023-09-23T09:28:31.320031+0000","last_fullsized":"2023-09-23T09:28:31.320031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:48.876752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734194+0000","last_change":"2023-09-23T09:27:50.278620+0000","last_active":"2023-09-23T09:27:51.734194+0000","last_peered":"2023-09-23T09:27:51.734194+0000","last_clean":"2023-09-23T09:27:51.734194+0000","last_became_active":"2023-09-23T09:27:50.278289+0000","last_became_peered":"2023-09-23T09:27:50.278289+0000","last_unstale":"2023-09-23T09:27:51.734194+0000","last_undegraded":"2023-09-23T09:27:51.734194+0000","last_fullsized":"2023-09-23T09:27:51.734194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:20:44.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.704421+0000","last_change":"2023-09-23T09:27:49.662970+0000","last_active":"2023-09-23T09:27:54.704421+0000","last_peered":"2023-09-23T09:27:54.704421+0000","last_clean":"2023-09-23T09:27:54.704421+0000","last_became_active":"2023-09-23T09:27:49.662664+0000","last_became_peered":"2023-09-23T09:27:49.662664+0000","last_unstale":"2023-09-23T09:27:54.704421+0000","last_undegraded":"2023-09-23T09:27:54.704421+0000","last_fullsized":"2023-09-23T09:27:54.704421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:13.189686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734127+0000","last_change":"2023-09-23T09:27:50.278475+0000","last_active":"2023-09-23T09:27:51.734127+0000","last_peered":"2023-09-23T09:27:51.734127+0000","last_clean":"2023-09-23T09:27:51.734127+0000","last_became_active":"2023-09-23T09:27:50.277802+0000","last_became_peered":"2023-09-23T09:27:50.277802+0000","last_unstale":"2023-09-23T09:27:51.734127+0000","last_undegraded":"2023-09-23T09:27:51.734127+0000","last_fullsized":"2023-09-23T09:27:51.734127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:44.608172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:55.261245+0000","last_change":"2023-09-23T09:27:49.657191+0000","last_active":"2023-09-23T09:27:55.261245+0000","last_peered":"2023-09-23T09:27:55.261245+0000","last_clean":"2023-09-23T09:27:55.261245+0000","last_became_active":"2023-09-23T09:27:49.656863+0000","last_became_peered":"2023-09-23T09:27:49.656863+0000","last_unstale":"2023-09-23T09:27:55.261245+0000","last_undegraded":"2023-09-23T09:27:55.261245+0000","last_fullsized":"2023-09-23T09:27:55.261245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:10.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734984+0000","last_change":"2023-09-23T09:27:50.274609+0000","last_active":"2023-09-23T09:27:51.734984+0000","last_peered":"2023-09-23T09:27:51.734984+0000","last_clean":"2023-09-23T09:27:51.734984+0000","last_became_active":"2023-09-23T09:27:50.274316+0000","last_became_peered":"2023-09-23T09:27:50.274316+0000","last_unstale":"2023-09-23T09:27:51.734984+0000","last_undegraded":"2023-09-23T09:27:51.734984+0000","last_fullsized":"2023-09-23T09:27:51.734984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:10:51.284306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259542+0000","last_change":"2023-09-23T09:27:49.659410+0000","last_active":"2023-09-23T09:27:51.259542+0000","last_peered":"2023-09-23T09:27:51.259542+0000","last_clean":"2023-09-23T09:27:51.259542+0000","last_became_active":"2023-09-23T09:27:49.659087+0000","last_became_peered":"2023-09-23T09:27:49.659087+0000","last_unstale":"2023-09-23T09:27:51.259542+0000","last_undegraded":"2023-09-23T09:27:51.259542+0000","last_fullsized":"2023-09-23T09:27:51.259542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:23:41.677416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.258804+0000","last_change":"2023-09-23T09:27:50.268568+0000","last_active":"2023-09-23T09:27:54.258804+0000","last_peered":"2023-09-23T09:27:54.258804+0000","last_clean":"2023-09-23T09:27:54.258804+0000","last_became_active":"2023-09-23T09:27:50.268268+0000","last_became_peered":"2023-09-23T09:27:50.268268+0000","last_unstale":"2023-09-23T09:27:54.258804+0000","last_undegraded":"2023-09-23T09:27:54.258804+0000","last_fullsized":"2023-09-23T09:27:54.258804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:59:22.516794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734411+0000","last_change":"2023-09-23T09:27:49.657160+0000","last_active":"2023-09-23T09:27:51.734411+0000","last_peered":"2023-09-23T09:27:51.734411+0000","last_clean":"2023-09-23T09:27:51.734411+0000","last_became_active":"2023-09-23T09:27:49.656853+0000","last_became_peered":"2023-09-23T09:27:49.656853+0000","last_unstale":"2023-09-23T09:27:51.734411+0000","last_undegraded":"2023-09-23T09:27:51.734411+0000","last_fullsized":"2023-09-23T09:27:51.734411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:17.489250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.398429+0000","last_change":"2023-09-23T09:27:50.272495+0000","last_active":"2023-09-23T09:27:51.398429+0000","last_peered":"2023-09-23T09:27:51.398429+0000","last_clean":"2023-09-23T09:27:51.398429+0000","last_became_active":"2023-09-23T09:27:50.272207+0000","last_became_peered":"2023-09-23T09:27:50.272207+0000","last_unstale":"2023-09-23T09:27:51.398429+0000","last_undegraded":"2023-09-23T09:27:51.398429+0000","last_fullsized":"2023-09-23T09:27:51.398429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:24.952253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734722+0000","last_change":"2023-09-23T09:27:50.275551+0000","last_active":"2023-09-23T09:27:51.734722+0000","last_peered":"2023-09-23T09:27:51.734722+0000","last_clean":"2023-09-23T09:27:51.734722+0000","last_became_active":"2023-09-23T09:27:50.275235+0000","last_became_peered":"2023-09-23T09:27:50.275235+0000","last_unstale":"2023-09-23T09:27:51.734722+0000","last_undegraded":"2023-09-23T09:27:51.734722+0000","last_fullsized":"2023-09-23T09:27:51.734722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:41.009566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:28:32.320741+0000","last_change":"2023-09-23T09:27:49.652889+0000","last_active":"2023-09-23T09:28:32.320741+0000","last_peered":"2023-09-23T09:28:32.320741+0000","last_clean":"2023-09-23T09:28:32.320741+0000","last_became_active":"2023-09-23T09:27:49.652571+0000","last_became_peered":"2023-09-23T09:27:49.652571+0000","last_unstale":"2023-09-23T09:28:32.320741+0000","last_undegraded":"2023-09-23T09:28:32.320741+0000","last_fullsized":"2023-09-23T09:28:32.320741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:22.484052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734814+0000","last_change":"2023-09-23T09:27:50.278419+0000","last_active":"2023-09-23T09:27:51.734814+0000","last_peered":"2023-09-23T09:27:51.734814+0000","last_clean":"2023-09-23T09:27:51.734814+0000","last_became_active":"2023-09-23T09:27:50.278071+0000","last_became_peered":"2023-09-23T09:27:50.278071+0000","last_unstale":"2023-09-23T09:27:51.734814+0000","last_undegraded":"2023-09-23T09:27:51.734814+0000","last_fullsized":"2023-09-23T09:27:51.734814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:28:45.896479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:55.260221+0000","last_change":"2023-09-23T09:27:49.651251+0000","last_active":"2023-09-23T09:27:55.260221+0000","last_peered":"2023-09-23T09:27:55.260221+0000","last_clean":"2023-09-23T09:27:55.260221+0000","last_became_active":"2023-09-23T09:27:49.650933+0000","last_became_peered":"2023-09-23T09:27:49.650933+0000","last_unstale":"2023-09-23T09:27:55.260221+0000","last_undegraded":"2023-09-23T09:27:55.260221+0000","last_fullsized":"2023-09-23T09:27:55.260221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:00:54.100400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.264684+0000","last_change":"2023-09-23T09:26:51.296669+0000","last_active":"2023-09-23T09:27:50.264684+0000","last_peered":"2023-09-23T09:27:50.264684+0000","last_clean":"2023-09-23T09:27:50.264684+0000","last_became_active":"2023-09-23T09:26:51.296229+0000","last_became_peered":"2023-09-23T09:26:51.296229+0000","last_unstale":"2023-09-23T09:27:50.264684+0000","last_undegraded":"2023-09-23T09:27:50.264684+0000","last_fullsized":"2023-09-23T09:27:50.264684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_clean_scrub_stamp":"2023-09-23T09:24:01.204422+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:31:24.888152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734832+0000","last_change":"2023-09-23T09:27:50.267662+0000","last_active":"2023-09-23T09:27:51.734832+0000","last_peered":"2023-09-23T09:27:51.734832+0000","last_clean":"2023-09-23T09:27:51.734832+0000","last_became_active":"2023-09-23T09:27:50.267374+0000","last_became_peered":"2023-09-23T09:27:50.267374+0000","last_unstale":"2023-09-23T09:27:51.734832+0000","last_undegraded":"2023-09-23T09:27:51.734832+0000","last_fullsized":"2023-09-23T09:27:51.734832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:22.797939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.701373+0000","last_change":"2023-09-23T09:27:49.659773+0000","last_active":"2023-09-23T09:27:54.701373+0000","last_peered":"2023-09-23T09:27:54.701373+0000","last_clean":"2023-09-23T09:27:54.701373+0000","last_became_active":"2023-09-23T09:27:49.659526+0000","last_became_peered":"2023-09-23T09:27:49.659526+0000","last_unstale":"2023-09-23T09:27:54.701373+0000","last_undegraded":"2023-09-23T09:27:54.701373+0000","last_fullsized":"2023-09-23T09:27:54.701373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:27.423680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689202+0000","last_change":"2023-09-23T09:27:49.661336+0000","last_active":"2023-09-23T09:27:53.689202+0000","last_peered":"2023-09-23T09:27:53.689202+0000","last_clean":"2023-09-23T09:27:53.689202+0000","last_became_active":"2023-09-23T09:27:49.661038+0000","last_became_peered":"2023-09-23T09:27:49.661038+0000","last_unstale":"2023-09-23T09:27:53.689202+0000","last_undegraded":"2023-09-23T09:27:53.689202+0000","last_fullsized":"2023-09-23T09:27:53.689202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:20.634410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735024+0000","last_change":"2023-09-23T09:27:50.272485+0000","last_active":"2023-09-23T09:27:51.735024+0000","last_peered":"2023-09-23T09:27:51.735024+0000","last_clean":"2023-09-23T09:27:51.735024+0000","last_became_active":"2023-09-23T09:27:50.272164+0000","last_became_peered":"2023-09-23T09:27:50.272164+0000","last_unstale":"2023-09-23T09:27:51.735024+0000","last_undegraded":"2023-09-23T09:27:51.735024+0000","last_fullsized":"2023-09-23T09:27:51.735024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:48:00.596034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:58.934312+0000","last_change":"2023-09-23T09:27:49.657563+0000","last_active":"2023-09-23T09:27:58.934312+0000","last_peered":"2023-09-23T09:27:58.934312+0000","last_clean":"2023-09-23T09:27:58.934312+0000","last_became_active":"2023-09-23T09:27:49.657232+0000","last_became_peered":"2023-09-23T09:27:49.657232+0000","last_unstale":"2023-09-23T09:27:58.934312+0000","last_undegraded":"2023-09-23T09:27:58.934312+0000","last_fullsized":"2023-09-23T09:27:58.934312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:29:10.640013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735007+0000","last_change":"2023-09-23T09:27:50.274959+0000","last_active":"2023-09-23T09:27:51.735007+0000","last_peered":"2023-09-23T09:27:51.735007+0000","last_clean":"2023-09-23T09:27:51.735007+0000","last_became_active":"2023-09-23T09:27:50.274614+0000","last_became_peered":"2023-09-23T09:27:50.274614+0000","last_unstale":"2023-09-23T09:27:51.735007+0000","last_undegraded":"2023-09-23T09:27:51.735007+0000","last_fullsized":"2023-09-23T09:27:51.735007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:00:09.831103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690819+0000","last_change":"2023-09-23T09:27:49.654673+0000","last_active":"2023-09-23T09:27:53.690819+0000","last_peered":"2023-09-23T09:27:53.690819+0000","last_clean":"2023-09-23T09:27:53.690819+0000","last_became_active":"2023-09-23T09:27:49.654356+0000","last_became_peered":"2023-09-23T09:27:49.654356+0000","last_unstale":"2023-09-23T09:27:53.690819+0000","last_undegraded":"2023-09-23T09:27:53.690819+0000","last_fullsized":"2023-09-23T09:27:53.690819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:39:50.628346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733670+0000","last_change":"2023-09-23T09:27:50.267368+0000","last_active":"2023-09-23T09:27:51.733670+0000","last_peered":"2023-09-23T09:27:51.733670+0000","last_clean":"2023-09-23T09:27:51.733670+0000","last_became_active":"2023-09-23T09:27:50.266962+0000","last_became_peered":"2023-09-23T09:27:50.266962+0000","last_unstale":"2023-09-23T09:27:51.733670+0000","last_undegraded":"2023-09-23T09:27:51.733670+0000","last_fullsized":"2023-09-23T09:27:51.733670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:24.660228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690757+0000","last_change":"2023-09-23T09:27:49.661200+0000","last_active":"2023-09-23T09:27:53.690757+0000","last_peered":"2023-09-23T09:27:53.690757+0000","last_clean":"2023-09-23T09:27:53.690757+0000","last_became_active":"2023-09-23T09:27:49.660902+0000","last_became_peered":"2023-09-23T09:27:49.660902+0000","last_unstale":"2023-09-23T09:27:53.690757+0000","last_undegraded":"2023-09-23T09:27:53.690757+0000","last_fullsized":"2023-09-23T09:27:53.690757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:41.036216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735005+0000","last_change":"2023-09-23T09:27:50.271834+0000","last_active":"2023-09-23T09:27:51.735005+0000","last_peered":"2023-09-23T09:27:51.735005+0000","last_clean":"2023-09-23T09:27:51.735005+0000","last_became_active":"2023-09-23T09:27:50.271519+0000","last_became_peered":"2023-09-23T09:27:50.271519+0000","last_unstale":"2023-09-23T09:27:51.735005+0000","last_undegraded":"2023-09-23T09:27:51.735005+0000","last_fullsized":"2023-09-23T09:27:51.735005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:08.036615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.689622+0000","last_change":"2023-09-23T09:27:49.656794+0000","last_active":"2023-09-23T09:27:53.689622+0000","last_peered":"2023-09-23T09:27:53.689622+0000","last_clean":"2023-09-23T09:27:53.689622+0000","last_became_active":"2023-09-23T09:27:49.656480+0000","last_became_peered":"2023-09-23T09:27:49.656480+0000","last_unstale":"2023-09-23T09:27:53.689622+0000","last_undegraded":"2023-09-23T09:27:53.689622+0000","last_fullsized":"2023-09-23T09:27:53.689622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:35:46.948703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734135+0000","last_change":"2023-09-23T09:27:50.271805+0000","last_active":"2023-09-23T09:27:51.734135+0000","last_peered":"2023-09-23T09:27:51.734135+0000","last_clean":"2023-09-23T09:27:51.734135+0000","last_became_active":"2023-09-23T09:27:50.271463+0000","last_became_peered":"2023-09-23T09:27:50.271463+0000","last_unstale":"2023-09-23T09:27:51.734135+0000","last_undegraded":"2023-09-23T09:27:51.734135+0000","last_fullsized":"2023-09-23T09:27:51.734135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:00:03.608504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264108+0000","last_change":"2023-09-23T09:27:49.656988+0000","last_active":"2023-09-23T09:27:51.264108+0000","last_peered":"2023-09-23T09:27:51.264108+0000","last_clean":"2023-09-23T09:27:51.264108+0000","last_became_active":"2023-09-23T09:27:49.656655+0000","last_became_peered":"2023-09-23T09:27:49.656655+0000","last_unstale":"2023-09-23T09:27:51.264108+0000","last_undegraded":"2023-09-23T09:27:51.264108+0000","last_fullsized":"2023-09-23T09:27:51.264108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:07:33.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734679+0000","last_change":"2023-09-23T09:27:50.275021+0000","last_active":"2023-09-23T09:27:51.734679+0000","last_peered":"2023-09-23T09:27:51.734679+0000","last_clean":"2023-09-23T09:27:51.734679+0000","last_became_active":"2023-09-23T09:27:50.274551+0000","last_became_peered":"2023-09-23T09:27:50.274551+0000","last_unstale":"2023-09-23T09:27:51.734679+0000","last_undegraded":"2023-09-23T09:27:51.734679+0000","last_fullsized":"2023-09-23T09:27:51.734679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:50:45.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:53.690216+0000","last_change":"2023-09-23T09:27:49.659603+0000","last_active":"2023-09-23T09:27:53.690216+0000","last_peered":"2023-09-23T09:27:53.690216+0000","last_clean":"2023-09-23T09:27:53.690216+0000","last_became_active":"2023-09-23T09:27:49.659293+0000","last_became_peered":"2023-09-23T09:27:49.659293+0000","last_unstale":"2023-09-23T09:27:53.690216+0000","last_undegraded":"2023-09-23T09:27:53.690216+0000","last_fullsized":"2023-09-23T09:27:53.690216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:31:25.165888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734478+0000","last_change":"2023-09-23T09:27:50.275069+0000","last_active":"2023-09-23T09:27:51.734478+0000","last_peered":"2023-09-23T09:27:51.734478+0000","last_clean":"2023-09-23T09:27:51.734478+0000","last_became_active":"2023-09-23T09:27:50.274731+0000","last_became_peered":"2023-09-23T09:27:50.274731+0000","last_unstale":"2023-09-23T09:27:51.734478+0000","last_undegraded":"2023-09-23T09:27:51.734478+0000","last_fullsized":"2023-09-23T09:27:51.734478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:45:23.794774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:54.703349+0000","last_change":"2023-09-23T09:27:49.659914+0000","last_active":"2023-09-23T09:27:54.703349+0000","last_peered":"2023-09-23T09:27:54.703349+0000","last_clean":"2023-09-23T09:27:54.703349+0000","last_became_active":"2023-09-23T09:27:49.659604+0000","last_became_peered":"2023-09-23T09:27:49.659604+0000","last_unstale":"2023-09-23T09:27:54.703349+0000","last_undegraded":"2023-09-23T09:27:54.703349+0000","last_fullsized":"2023-09-23T09:27:54.703349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:20.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734661+0000","last_change":"2023-09-23T09:27:50.271309+0000","last_active":"2023-09-23T09:27:51.734661+0000","last_peered":"2023-09-23T09:27:51.734661+0000","last_clean":"2023-09-23T09:27:51.734661+0000","last_became_active":"2023-09-23T09:27:50.270951+0000","last_became_peered":"2023-09-23T09:27:50.270951+0000","last_unstale":"2023-09-23T09:27:51.734661+0000","last_undegraded":"2023-09-23T09:27:51.734661+0000","last_fullsized":"2023-09-23T09:27:51.734661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:18.811428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734659+0000","last_change":"2023-09-23T09:27:49.663849+0000","last_active":"2023-09-23T09:27:51.734659+0000","last_peered":"2023-09-23T09:27:51.734659+0000","last_clean":"2023-09-23T09:27:51.734659+0000","last_became_active":"2023-09-23T09:27:49.663499+0000","last_became_peered":"2023-09-23T09:27:49.663499+0000","last_unstale":"2023-09-23T09:27:51.734659+0000","last_undegraded":"2023-09-23T09:27:51.734659+0000","last_fullsized":"2023-09-23T09:27:51.734659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:31:08.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734582+0000","last_change":"2023-09-23T09:27:50.275681+0000","last_active":"2023-09-23T09:27:51.734582+0000","last_peered":"2023-09-23T09:27:51.734582+0000","last_clean":"2023-09-23T09:27:51.734582+0000","last_became_active":"2023-09-23T09:27:50.275370+0000","last_became_peered":"2023-09-23T09:27:50.275370+0000","last_unstale":"2023-09-23T09:27:51.734582+0000","last_undegraded":"2023-09-23T09:27:51.734582+0000","last_fullsized":"2023-09-23T09:27:51.734582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:53:01.710377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T09:27:50.665421+0000","last_change":"2023-09-23T09:27:49.652127+0000","last_active":"2023-09-23T09:27:50.665421+0000","last_peered":"2023-09-23T09:27:50.665421+0000","last_clean":"2023-09-23T09:27:50.665421+0000","last_became_active":"2023-09-23T09:27:49.651792+0000","last_became_peered":"2023-09-23T09:27:49.651792+0000","last_unstale":"2023-09-23T09:27:50.665421+0000","last_undegraded":"2023-09-23T09:27:50.665421+0000","last_fullsized":"2023-09-23T09:27:50.665421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:23:39.299203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734734+0000","last_change":"2023-09-23T09:27:50.271063+0000","last_active":"2023-09-23T09:27:51.734734+0000","last_peered":"2023-09-23T09:27:51.734734+0000","last_clean":"2023-09-23T09:27:51.734734+0000","last_became_active":"2023-09-23T09:27:50.270487+0000","last_became_peered":"2023-09-23T09:27:50.270487+0000","last_unstale":"2023-09-23T09:27:51.734734+0000","last_undegraded":"2023-09-23T09:27:51.734734+0000","last_fullsized":"2023-09-23T09:27:51.734734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:11:05.621855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734264+0000","last_change":"2023-09-23T09:27:50.275900+0000","last_active":"2023-09-23T09:27:51.734264+0000","last_peered":"2023-09-23T09:27:51.734264+0000","last_clean":"2023-09-23T09:27:51.734264+0000","last_became_active":"2023-09-23T09:27:50.275565+0000","last_became_peered":"2023-09-23T09:27:50.275565+0000","last_unstale":"2023-09-23T09:27:51.734264+0000","last_undegraded":"2023-09-23T09:27:51.734264+0000","last_fullsized":"2023-09-23T09:27:51.734264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:57:37.004604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264041+0000","last_change":"2023-09-23T09:27:50.275911+0000","last_active":"2023-09-23T09:27:51.264041+0000","last_peered":"2023-09-23T09:27:51.264041+0000","last_clean":"2023-09-23T09:27:51.264041+0000","last_became_active":"2023-09-23T09:27:50.275626+0000","last_became_peered":"2023-09-23T09:27:50.275626+0000","last_unstale":"2023-09-23T09:27:51.264041+0000","last_undegraded":"2023-09-23T09:27:51.264041+0000","last_fullsized":"2023-09-23T09:27:51.264041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:46.790012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264037+0000","last_change":"2023-09-23T09:27:50.269483+0000","last_active":"2023-09-23T09:27:51.264037+0000","last_peered":"2023-09-23T09:27:51.264037+0000","last_clean":"2023-09-23T09:27:51.264037+0000","last_became_active":"2023-09-23T09:27:50.269199+0000","last_became_peered":"2023-09-23T09:27:50.269199+0000","last_unstale":"2023-09-23T09:27:51.264037+0000","last_undegraded":"2023-09-23T09:27:51.264037+0000","last_fullsized":"2023-09-23T09:27:51.264037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:58:34.304439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733479+0000","last_change":"2023-09-23T09:27:50.267917+0000","last_active":"2023-09-23T09:27:51.733479+0000","last_peered":"2023-09-23T09:27:51.733479+0000","last_clean":"2023-09-23T09:27:51.733479+0000","last_became_active":"2023-09-23T09:27:50.267036+0000","last_became_peered":"2023-09-23T09:27:50.267036+0000","last_unstale":"2023-09-23T09:27:51.733479+0000","last_undegraded":"2023-09-23T09:27:51.733479+0000","last_fullsized":"2023-09-23T09:27:51.733479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:42.866260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.735091+0000","last_change":"2023-09-23T09:27:50.269934+0000","last_active":"2023-09-23T09:27:51.735091+0000","last_peered":"2023-09-23T09:27:51.735091+0000","last_clean":"2023-09-23T09:27:51.735091+0000","last_became_active":"2023-09-23T09:27:50.269060+0000","last_became_peered":"2023-09-23T09:27:50.269060+0000","last_unstale":"2023-09-23T09:27:51.735091+0000","last_undegraded":"2023-09-23T09:27:51.735091+0000","last_fullsized":"2023-09-23T09:27:51.735091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:47:31.191479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733829+0000","last_change":"2023-09-23T09:27:50.277885+0000","last_active":"2023-09-23T09:27:51.733829+0000","last_peered":"2023-09-23T09:27:51.733829+0000","last_clean":"2023-09-23T09:27:51.733829+0000","last_became_active":"2023-09-23T09:27:50.277576+0000","last_became_peered":"2023-09-23T09:27:50.277576+0000","last_unstale":"2023-09-23T09:27:51.733829+0000","last_undegraded":"2023-09-23T09:27:51.733829+0000","last_fullsized":"2023-09-23T09:27:51.733829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:03:43.338185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.264326+0000","last_change":"2023-09-23T09:27:50.278225+0000","last_active":"2023-09-23T09:27:51.264326+0000","last_peered":"2023-09-23T09:27:51.264326+0000","last_clean":"2023-09-23T09:27:51.264326+0000","last_became_active":"2023-09-23T09:27:50.277891+0000","last_became_peered":"2023-09-23T09:27:50.277891+0000","last_unstale":"2023-09-23T09:27:51.264326+0000","last_undegraded":"2023-09-23T09:27:51.264326+0000","last_fullsized":"2023-09-23T09:27:51.264326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:27:08.961439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734854+0000","last_change":"2023-09-23T09:27:50.273445+0000","last_active":"2023-09-23T09:27:51.734854+0000","last_peered":"2023-09-23T09:27:51.734854+0000","last_clean":"2023-09-23T09:27:51.734854+0000","last_became_active":"2023-09-23T09:27:50.273124+0000","last_became_peered":"2023-09-23T09:27:50.273124+0000","last_unstale":"2023-09-23T09:27:51.734854+0000","last_undegraded":"2023-09-23T09:27:51.734854+0000","last_fullsized":"2023-09-23T09:27:51.734854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:46.030212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734647+0000","last_change":"2023-09-23T09:27:50.276164+0000","last_active":"2023-09-23T09:27:51.734647+0000","last_peered":"2023-09-23T09:27:51.734647+0000","last_clean":"2023-09-23T09:27:51.734647+0000","last_became_active":"2023-09-23T09:27:50.275860+0000","last_became_peered":"2023-09-23T09:27:50.275860+0000","last_unstale":"2023-09-23T09:27:51.734647+0000","last_undegraded":"2023-09-23T09:27:51.734647+0000","last_fullsized":"2023-09-23T09:27:51.734647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:53:50.521872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734518+0000","last_change":"2023-09-23T09:27:50.275221+0000","last_active":"2023-09-23T09:27:51.734518+0000","last_peered":"2023-09-23T09:27:51.734518+0000","last_clean":"2023-09-23T09:27:51.734518+0000","last_became_active":"2023-09-23T09:27:50.274920+0000","last_became_peered":"2023-09-23T09:27:50.274920+0000","last_unstale":"2023-09-23T09:27:51.734518+0000","last_undegraded":"2023-09-23T09:27:51.734518+0000","last_fullsized":"2023-09-23T09:27:51.734518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:45:35.023433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.259534+0000","last_change":"2023-09-23T09:27:50.277776+0000","last_active":"2023-09-23T09:27:51.259534+0000","last_peered":"2023-09-23T09:27:51.259534+0000","last_clean":"2023-09-23T09:27:51.259534+0000","last_became_active":"2023-09-23T09:27:50.277291+0000","last_became_peered":"2023-09-23T09:27:50.277291+0000","last_unstale":"2023-09-23T09:27:51.259534+0000","last_undegraded":"2023-09-23T09:27:51.259534+0000","last_fullsized":"2023-09-23T09:27:51.259534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:23:46.984450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.734180+0000","last_change":"2023-09-23T09:27:50.271219+0000","last_active":"2023-09-23T09:27:51.734180+0000","last_peered":"2023-09-23T09:27:51.734180+0000","last_clean":"2023-09-23T09:27:51.734180+0000","last_became_active":"2023-09-23T09:27:50.270747+0000","last_became_peered":"2023-09-23T09:27:50.270747+0000","last_unstale":"2023-09-23T09:27:51.734180+0000","last_undegraded":"2023-09-23T09:27:51.734180+0000","last_fullsized":"2023-09-23T09:27:51.734180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:19.237260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T09:27:51.733913+0000","last_change":"2023-09-23T09:27:50.263794+0000","last_active":"2023-09-23T09:27:51.733913+0000","last_peered":"2023-09-23T09:27:51.733913+0000","last_clean":"2023-09-23T09:27:51.733913+0000","last_became_active":"2023-09-23T09:27:50.263459+0000","last_became_peered":"2023-09-23T09:27:50.263459+0000","last_unstale":"2023-09-23T09:27:51.733913+0000","last_undegraded":"2023-09-23T09:27:51.733913+0000","last_fullsized":"2023-09-23T09:27:51.733913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:05:37.618959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250071,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61384,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":260,"num_read_kb":195,"num_write":149,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":184152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":427264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77700000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.502}]},{"osd":6,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.76300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":9,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.747}]}]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1076135,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.503}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.77700000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.621},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57699999999999996}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.85399999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.75800000000000001}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.86899999999999999}]},{"osd":9,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.89500000000000002}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.93100000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841565,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":427500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.497},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51400000000000001,"15min":0.51800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.67700000000000005,"15min":0.66100000000000003},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.139,"5min":0.79800000000000004,"15min":0.74099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.616,"5min":11.616,"15min":11.616},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":0.83999999999999997,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":11.647,"5min":11.647,"15min":11.647},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.74299999999999999}]},{"osd":3,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.9889999999999999,"5min":3.9889999999999999,"15min":3.9889999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.78800000000000003,"15min":0.751},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":11.702,"5min":11.702,"15min":11.702},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.6789999999999998,"5min":3.6789999999999998,"15min":3.6789999999999998},"last":0.71199999999999997}]},{"osd":5,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":0.876}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82099999999999995,"15min":0.83199999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.77700000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88,"15min":0.88},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.1930000000000001,"5min":0.999,"15min":0.96699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":11.557,"5min":11.557,"15min":11.557},"last":0.68500000000000005}]},{"osd":8,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.004,"15min":0.96699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":11.675000000000001,"5min":11.675000000000001,"15min":11.675000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.89700000000000002,"15min":0.91600000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.80100000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85299999999999998}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":427420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.79500000000000004}]},{"osd":1,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.71899999999999997}]},{"osd":2,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.745},"last":0.434},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45900000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.67400000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.628}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51200000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95099999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.83799999999999997}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.77800000000000002}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.87},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1019472,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41199999999999998,"15min":0.40400000000000003},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60599999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.628}]},{"osd":4,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.311,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46899999999999997}]},{"osd":5,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71399999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64500000000000002}]},{"osd":8,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.53700000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99299999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.871,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":426895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.48199999999999998,"15min":0.432},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.442,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.373}]},{"osd":1,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.65900000000000003,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.63600000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.48299999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.48099999999999998},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.66700000000000004,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.312,"15min":0.312},"max":{"1min":11.151999999999999,"5min":11.151999999999999,"15min":11.151999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.622}]},{"osd":5,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.749,"15min":0.69799999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":6.952,"5min":6.952,"15min":6.952},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.61899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70999999999999996}]},{"osd":8,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":6.9850000000000003,"5min":6.9850000000000003,"15min":6.9850000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.60899999999999999}]},{"osd":9,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":0.746},{"interface":"front","average":{"1min":1,"5min":0.85599999999999998,"15min":0.83199999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":7.016,"5min":7.016,"15min":7.016},"last":0.70099999999999996}]},{"osd":10,"last update":"Sat Sep 23 09:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":483676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.37,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.73699999999999999}]},{"osd":2,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.503,"15min":0.48399999999999999},"min":{"1min":0.38700000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.53500000000000003,"15min":0.505},"min":{"1min":0.40899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67400000000000004}]},{"osd":3,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.754},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.84399999999999997}]},{"osd":4,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.29499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.497}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.872,"5min":0.877,"15min":0.877},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.91900000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.64900000000000002,"15min":0.59699999999999998},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.7679999999999998,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69399999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51800000000000002}]},{"osd":8,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88700000000000001,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91700000000000004,"5min":1.01,"15min":1.01},"last":0.86499999999999999}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":428160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 09:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.55100000000000005,"15min":0.51300000000000001},"min":{"1min":0.46400000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.439,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.58099999999999996,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.81000000000000005}]},{"osd":3,"last update":"Sat Sep 23 09:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.26400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81100000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.57099999999999995,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.51300000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.97299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.718,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.718},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":8,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0649999999999999,"5min":1.101,"15min":1.101},"last":0.93600000000000005}]},{"osd":9,"last update":"Sat Sep 23 09:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.90200000000000002}]},{"osd":10,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.60599999999999998}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.216}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":485416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.71199999999999997,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.32800000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.78000000000000003,"5min":1.655,"15min":1.655},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.505},"min":{"1min":0.35899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.747}]},{"osd":3,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.627},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73999999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.46000000000000002,"15min":0.442},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.29499999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85699999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.014,"15min":1.014},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95899999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51200000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93000000000000005,"5min":0.995,"15min":0.995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68000000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.83299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":426881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.39100000000000001,"15min":0.35299999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.28199999999999997,"5min":0.191,"15min":0.191},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.39900000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55000000000000004,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96599999999999997,"5min":1.036,"15min":1.036},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79500000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Sep 23 09:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.55300000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":1,"15min":1},"last":0.54400000000000004}]},{"osd":6,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.53500000000000003},"min":{"1min":0.438,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4990000000000001,"5min":2.4990000000000001,"15min":2.4990000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.436}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.48799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.71099999999999997,"15min":0.69999999999999996},"min":{"1min":0.628,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78300000000000003}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.55400000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.81100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":427716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.998,"15min":0.998},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.38},"min":{"1min":0.33900000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.57599999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69199999999999995}]},{"osd":3,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.495,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61299999999999999,"15min":0.58199999999999996},"min":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.877,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76900000000000002}]},{"osd":6,"last update":"Sat Sep 23 09:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.874},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81499999999999995}]},{"osd":8,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92300000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98399999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":1.089,"15min":1.089},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.998}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.98399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":1019039,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.55300000000000005,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.52000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.51000000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.51000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.63500000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.59899999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.56399999999999995,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.163,"5min":1.538,"15min":1.538},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.88},{"interface":"front","average":{"1min":0.874,"5min":0.72199999999999998,"15min":0.67700000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.54800000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.73299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88100000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86199999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.745}]},{"osd":9,"last update":"Sat Sep 23 09:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.95199999999999996}]},{"osd":10,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T09:28:36.958 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-23T09:28:36.958 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-23T09:28:36.958 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-23T09:28:36.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph health --format=json 2023-09-23T09:28:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:37.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:37 smithi045 ceph-mon[98627]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:37.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:37 smithi045 ceph-mon[98627]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:37.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:37 smithi098 ceph-mon[102386]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:37.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:37 smithi098 ceph-mon[102386]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:37.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:37 smithi123 ceph-mon[103577]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:37.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:37 smithi123 ceph-mon[103577]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:28:38.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 -- 172.21.15.45:0/3416888616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0ffd60 msgr2=0x7f639c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/3416888616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0ffd60 0x7f639c1001e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6390009990 tx=0x7f6390009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 -- 172.21.15.45:0/3416888616 shutdown_connections 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/3416888616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f639c100720 0x7f639c10cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/3416888616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c105e10 0x7f639c1061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/3416888616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0ffd60 0x7f639c1001e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 -- 172.21.15.45:0/3416888616 >> 172.21.15.45:0/3416888616 conn(0x7f639c076e20 msgr2=0x7f639c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 -- 172.21.15.45:0/3416888616 shutdown_connections 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 -- 172.21.15.45:0/3416888616 wait complete. 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.865+0000 7f63a0ebe700 1 Processor -- start 2023-09-23T09:28:38.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f63a0ebe700 1 -- start start 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f63a0ebe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0ffd60 0x7f639c196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f63a0ebe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f639c100720 0x7f639c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f63a0ebe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c105e10 0x7f639c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f63a0ebe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19bed0 con 0x7f639c105e10 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f63a0ebe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19c040 con 0x7f639c0ffd60 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f63a0ebe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19c200 con 0x7f639c100720 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c105e10 0x7f639c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c105e10 0x7f639c19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59438/0 (socket says 172.21.15.45:59438) 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 -- 172.21.15.45:0/613241069 learned_addr learned my addr 172.21.15.45:0/613241069 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639a59c700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0ffd60 0x7f639c196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f6399d9b700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f639c100720 0x7f639c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:38.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 -- 172.21.15.45:0/613241069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f639c100720 msgr2=0x7f639c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f639c100720 0x7f639c197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 -- 172.21.15.45:0/613241069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0ffd60 msgr2=0x7f639c196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0ffd60 0x7f639c196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 -- 172.21.15.45:0/613241069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900095f0 con 0x7f639c105e10 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f6399d9b700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f639c100720 0x7f639c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.866+0000 7f639ad9d700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c105e10 0x7f639c19b8c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6394007f10 tx=0x7f639400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.867+0000 7f638b7fe700 1 -- 172.21.15.45:0/613241069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394017070 con 0x7f639c105e10 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.867+0000 7f638b7fe700 1 -- 172.21.15.45:0/613241069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f639400f040 con 0x7f639c105e10 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.867+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c19c4e0 con 0x7f639c105e10 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.867+0000 7f638b7fe700 1 -- 172.21.15.45:0/613241069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394005740 con 0x7f639c105e10 2023-09-23T09:28:38.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.867+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c1a3d40 con 0x7f639c105e10 2023-09-23T09:28:38.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.868+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c04eb20 con 0x7f639c105e10 2023-09-23T09:28:38.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.869+0000 7f638b7fe700 1 -- 172.21.15.45:0/613241069 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f639400dc60 con 0x7f639c105e10 2023-09-23T09:28:38.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.870+0000 7f638b7fe700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63840a1630 0x7f63840a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:38.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.870+0000 7f639a59c700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63840a1630 0x7f63840a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:38.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.870+0000 7f638b7fe700 1 -- 172.21.15.45:0/613241069 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63940cc6b0 con 0x7f639c105e10 2023-09-23T09:28:38.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.870+0000 7f639a59c700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63840a1630 0x7f63840a3af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f638c005950 tx=0x7f638c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:38.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:38.874+0000 7f638b7fe700 1 -- 172.21.15.45:0/613241069 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6394095cf0 con 0x7f639c105e10 2023-09-23T09:28:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.118+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f639c198270 con 0x7f639c105e10 2023-09-23T09:28:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.118+0000 7f638b7fe700 1 -- 172.21.15.45:0/613241069 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6394099310 con 0x7f639c105e10 2023-09-23T09:28:39.120 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-23T09:28:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63840a1630 msgr2=0x7f63840a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63840a1630 0x7f63840a3af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f638c005950 tx=0x7f638c00b410 comp rx=0 tx=0).stop 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c105e10 msgr2=0x7f639c19b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c105e10 0x7f639c19b8c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6394007f10 tx=0x7f639400d3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 shutdown_connections 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c105e10 0x7f639c19b8c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0ffd60 0x7f639c196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f63840a1630 0x7f63840a3af0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 --2- 172.21.15.45:0/613241069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f639c100720 0x7f639c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.120+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 >> 172.21.15.45:0/613241069 conn(0x7f639c076e20 msgr2=0x7f639c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.121+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 shutdown_connections 2023-09-23T09:28:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.121+0000 7f63a0ebe700 1 -- 172.21.15.45:0/613241069 wait complete. 2023-09-23T09:28:39.569 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-23T09:28:39.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:28:39.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 -- 172.21.15.45:0/2759109321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101890 msgr2=0x7f531c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:39.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 --2- 172.21.15.45:0/2759109321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101890 0x7f531c108140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f531400b0d0 tx=0x7f531400b3e0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 -- 172.21.15.45:0/2759109321 shutdown_connections 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 --2- 172.21.15.45:0/2759109321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101890 0x7f531c108140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 --2- 172.21.15.45:0/2759109321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f531c0ffd00 0x7f531c100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 --2- 172.21.15.45:0/2759109321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c100ef0 0x7f531c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 -- 172.21.15.45:0/2759109321 >> 172.21.15.45:0/2759109321 conn(0x7f531c0fb250 msgr2=0x7f531c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 -- 172.21.15.45:0/2759109321 shutdown_connections 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 -- 172.21.15.45:0/2759109321 wait complete. 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.698+0000 7f5323240700 1 Processor -- start 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5323240700 1 -- start start 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5323240700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c0ffd00 0x7f531c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5323240700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f531c100ef0 0x7f531c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5323240700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c101890 0x7f531c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5323240700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c19e830 con 0x7f531c0ffd00 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5323240700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c19e9a0 con 0x7f531c100ef0 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5323240700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c19eb60 con 0x7f531c101890 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c0ffd00 0x7f531c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c0ffd00 0x7f531c196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59444/0 (socket says 172.21.15.45:59444) 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 -- 172.21.15.45:0/2429758616 learned_addr learned my addr 172.21.15.45:0/2429758616 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f53217dd700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c101890 0x7f531c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f531bfff700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f531c100ef0 0x7f531c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 -- 172.21.15.45:0/2429758616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c101890 msgr2=0x7f531c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c101890 0x7f531c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 -- 172.21.15.45:0/2429758616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f531c100ef0 msgr2=0x7f531c197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f531c100ef0 0x7f531c197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 -- 172.21.15.45:0/2429758616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314009d00 con 0x7f531c0ffd00 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f53217dd700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c101890 0x7f531c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f531bfff700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f531c100ef0 0x7f531c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5320fdc700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c0ffd00 0x7f531c196cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f531000b500 tx=0x7f531000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.699+0000 7f5319ffb700 1 -- 172.21.15.45:0/2429758616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310011830 con 0x7f531c0ffd00 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.700+0000 7f5319ffb700 1 -- 172.21.15.45:0/2429758616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5310011e50 con 0x7f531c0ffd00 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.700+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c19ee40 con 0x7f531c0ffd00 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.700+0000 7f5319ffb700 1 -- 172.21.15.45:0/2429758616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000f2e0 con 0x7f531c0ffd00 2023-09-23T09:28:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.700+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c19f260 con 0x7f531c0ffd00 2023-09-23T09:28:39.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.701+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c107250 con 0x7f531c0ffd00 2023-09-23T09:28:39.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.702+0000 7f5319ffb700 1 -- 172.21.15.45:0/2429758616 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f5310010580 con 0x7f531c0ffd00 2023-09-23T09:28:39.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.702+0000 7f5319ffb700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f530c0a1670 0x7f530c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:39.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.702+0000 7f531bfff700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f530c0a1670 0x7f530c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:39.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.702+0000 7f5319ffb700 1 -- 172.21.15.45:0/2429758616 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f53100cc8b0 con 0x7f531c0ffd00 2023-09-23T09:28:39.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.703+0000 7f531bfff700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f530c0a1670 0x7f530c0a3b30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5308005fd0 tx=0x7f5308005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:39.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.707+0000 7f5319ffb700 1 -- 172.21.15.45:0/2429758616 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5310096000 con 0x7f531c0ffd00 2023-09-23T09:28:39.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:39 smithi098 ceph-mon[102386]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-23T09:28:39.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:39 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/613241069' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-23T09:28:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:39 smithi045 ceph-mon[98627]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-23T09:28:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:39 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/613241069' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-23T09:28:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:39 smithi123 ceph-mon[103577]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-23T09:28:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:39 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/613241069' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-23T09:28:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.912+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f531c10c930 con 0x7f531c0ffd00 2023-09-23T09:28:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.914+0000 7f5319ffb700 1 -- 172.21.15.45:0/2429758616 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7f5310015090 con 0x7f531c0ffd00 2023-09-23T09:28:39.916 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:39.916 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:51.253687+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:28:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.916+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f530c0a1670 msgr2=0x7f530c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.916+0000 7f5323240700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f530c0a1670 0x7f530c0a3b30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5308005fd0 tx=0x7f5308005dc0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.916+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c0ffd00 msgr2=0x7f531c196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.916+0000 7f5323240700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c0ffd00 0x7f531c196cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f531000b500 tx=0x7f531000b8c0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.916+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 shutdown_connections 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.916+0000 7f5323240700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f530c0a1670 0x7f530c0a3b30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.916+0000 7f5323240700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c101890 0x7f531c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.917+0000 7f5323240700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c0ffd00 0x7f531c196cc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.917+0000 7f5323240700 1 --2- 172.21.15.45:0/2429758616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f531c100ef0 0x7f531c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.917+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 >> 172.21.15.45:0/2429758616 conn(0x7f531c0fb250 msgr2=0x7f531c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.917+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 shutdown_connections 2023-09-23T09:28:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:39.917+0000 7f5323240700 1 -- 172.21.15.45:0/2429758616 wait complete. 2023-09-23T09:28:39.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:28:39.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:28:40.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 -- 172.21.15.45:0/4017308758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0101f60 msgr2=0x7f8cf01023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/4017308758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0101f60 0x7f8cf01023e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009990 tx=0x7f8cd8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 -- 172.21.15.45:0/4017308758 shutdown_connections 2023-09-23T09:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/4017308758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf0102920 0x7f8cf0109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/4017308758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cf0100e10 0x7f8cf0101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/4017308758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0101f60 0x7f8cf01023e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 -- 172.21.15.45:0/4017308758 >> 172.21.15.45:0/4017308758 conn(0x7f8cf00fc3a0 msgr2=0x7f8cf00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 -- 172.21.15.45:0/4017308758 shutdown_connections 2023-09-23T09:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 -- 172.21.15.45:0/4017308758 wait complete. 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 Processor -- start 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.058+0000 7f8cf64d7700 1 -- start start 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cf64d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0100e10 0x7f8cf0196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cf64d7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf0101f60 0x7f8cf0196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cf64d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cf0102920 0x7f8cf019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cf64d7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf019e5d0 con 0x7f8cf0100e10 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cf64d7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf019e710 con 0x7f8cf0101f60 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cf64d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf019e8a0 con 0x7f8cf0102920 2023-09-23T09:28:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0100e10 0x7f8cf0196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0100e10 0x7f8cf0196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59458/0 (socket says 172.21.15.45:59458) 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 -- 172.21.15.45:0/2959779866 learned_addr learned my addr 172.21.15.45:0/2959779866 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cf4a74700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cf0102920 0x7f8cf019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cef7fe700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf0101f60 0x7f8cf0196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 -- 172.21.15.45:0/2959779866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cf0102920 msgr2=0x7f8cf019e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cf0102920 0x7f8cf019e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 -- 172.21.15.45:0/2959779866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf0101f60 msgr2=0x7f8cf0196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf0101f60 0x7f8cf0196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 -- 172.21.15.45:0/2959779866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80095f0 con 0x7f8cf0100e10 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8cef7fe700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf0101f60 0x7f8cf0196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.059+0000 7f8ceffff700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0100e10 0x7f8cf0196a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000d980 tx=0x7f8ce000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:40.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.060+0000 7f8ced7fa700 1 -- 172.21.15.45:0/2959779866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000bb60 con 0x7f8cf0100e10 2023-09-23T09:28:40.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.060+0000 7f8ced7fa700 1 -- 172.21.15.45:0/2959779866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ce0007640 con 0x7f8cf0100e10 2023-09-23T09:28:40.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.060+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf019eb80 con 0x7f8cf0100e10 2023-09-23T09:28:40.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.060+0000 7f8ced7fa700 1 -- 172.21.15.45:0/2959779866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0004e90 con 0x7f8cf0100e10 2023-09-23T09:28:40.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.060+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf019f0e0 con 0x7f8cf0100e10 2023-09-23T09:28:40.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.061+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf0108370 con 0x7f8cf0100e10 2023-09-23T09:28:40.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.062+0000 7f8ced7fa700 1 -- 172.21.15.45:0/2959779866 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f8ce000bcc0 con 0x7f8cf0100e10 2023-09-23T09:28:40.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.062+0000 7f8ced7fa700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8cdc0a16b0 0x7f8cdc0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.062+0000 7f8cef7fe700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8cdc0a16b0 0x7f8cdc0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.062+0000 7f8ced7fa700 1 -- 172.21.15.45:0/2959779866 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ce00cd2a0 con 0x7f8cf0100e10 2023-09-23T09:28:40.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.063+0000 7f8cef7fe700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8cdc0a16b0 0x7f8cdc0a3b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8000c00 tx=0x7f8cd8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:40.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.066+0000 7f8ced7fa700 1 -- 172.21.15.45:0/2959779866 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8ce00968e0 con 0x7f8cf0100e10 2023-09-23T09:28:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.266+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cf010c7a0 con 0x7f8cf0100e10 2023-09-23T09:28:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.267+0000 7f8ced7fa700 1 -- 172.21.15.45:0/2959779866 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ce0016020 con 0x7f8cf0100e10 2023-09-23T09:28:40.269 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:28:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8cdc0a16b0 msgr2=0x7f8cdc0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8cdc0a16b0 0x7f8cdc0a3b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8000c00 tx=0x7f8cd8005fb0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0100e10 msgr2=0x7f8cf0196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0100e10 0x7f8cf0196a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000d980 tx=0x7f8ce000dd40 comp rx=0 tx=0).stop 2023-09-23T09:28:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 shutdown_connections 2023-09-23T09:28:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8cdc0a16b0 0x7f8cdc0a3b70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cf0102920 0x7f8cf019e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf0100e10 0x7f8cf0196a50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 --2- 172.21.15.45:0/2959779866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf0101f60 0x7f8cf0196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.269+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 >> 172.21.15.45:0/2959779866 conn(0x7f8cf00fc3a0 msgr2=0x7f8cf01054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.270+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 shutdown_connections 2023-09-23T09:28:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.270+0000 7f8cf64d7700 1 -- 172.21.15.45:0/2959779866 wait complete. 2023-09-23T09:28:40.284 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 -- 172.21.15.45:0/735093976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 msgr2=0x7fb6640ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/735093976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb6640ffb00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb6540099c0 tx=0x7fb654009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 -- 172.21.15.45:0/735093976 shutdown_connections 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/735093976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb664100a00 0x7fb664100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/735093976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb6640ffb00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/735093976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb664100040 0x7fb6641004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 -- 172.21.15.45:0/735093976 >> 172.21.15.45:0/735093976 conn(0x7fb6640fb280 msgr2=0x7fb6640fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 -- 172.21.15.45:0/735093976 shutdown_connections 2023-09-23T09:28:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.411+0000 7fb66b3c3700 1 -- 172.21.15.45:0/735093976 wait complete. 2023-09-23T09:28:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 Processor -- start 2023-09-23T09:28:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 -- start start 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb664198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb664100040 0x7fb664199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb664100a00 0x7fb6641a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a0830 con 0x7fb6640ff6e0 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a09a0 con 0x7fb664100a00 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66b3c3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a0b60 con 0x7fb664100040 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb664198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb664198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59472/0 (socket says 172.21.15.45:59472) 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 -- 172.21.15.45:0/3819643543 learned_addr learned my addr 172.21.15.45:0/3819643543 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66895e700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb664100040 0x7fb664199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb669960700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb664100a00 0x7fb6641a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 -- 172.21.15.45:0/3819643543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb664100040 msgr2=0x7fb664199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb664100040 0x7fb664199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 -- 172.21.15.45:0/3819643543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb664100a00 msgr2=0x7fb6641a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb664100a00 0x7fb6641a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 -- 172.21.15.45:0/3819643543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540095f0 con 0x7fb6640ff6e0 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb669960700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb664100a00 0x7fb6641a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66895e700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb664100040 0x7fb664199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.412+0000 7fb66915f700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb664198cc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb654009990 tx=0x7fb654005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.413+0000 7fb65a7fc700 1 -- 172.21.15.45:0/3819643543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65401d070 con 0x7fb6640ff6e0 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.413+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6641a0de0 con 0x7fb6640ff6e0 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.413+0000 7fb65a7fc700 1 -- 172.21.15.45:0/3819643543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb654022470 con 0x7fb6640ff6e0 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.413+0000 7fb65a7fc700 1 -- 172.21.15.45:0/3819643543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654004230 con 0x7fb6640ff6e0 2023-09-23T09:28:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.413+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641a1220 con 0x7fb6640ff6e0 2023-09-23T09:28:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.414+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66410a4c0 con 0x7fb6640ff6e0 2023-09-23T09:28:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.415+0000 7fb65a7fc700 1 -- 172.21.15.45:0/3819643543 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fb6540225e0 con 0x7fb6640ff6e0 2023-09-23T09:28:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.415+0000 7fb65a7fc700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb6500a1630 0x7fb6500a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.416+0000 7fb66895e700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb6500a1630 0x7fb6500a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.416+0000 7fb65a7fc700 1 -- 172.21.15.45:0/3819643543 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb6540ce5e0 con 0x7fb6640ff6e0 2023-09-23T09:28:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.416+0000 7fb66895e700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb6500a1630 0x7fb6500a3af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fd0 tx=0x7fb660005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.419+0000 7fb65a7fc700 1 -- 172.21.15.45:0/3819643543 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb654097bb0 con 0x7fb6640ff6e0 2023-09-23T09:28:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:40 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2429758616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:40.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:40 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2959779866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.622+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb66410ea50 con 0x7fb6640ff6e0 2023-09-23T09:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.623+0000 7fb65a7fc700 1 -- 172.21.15.45:0/3819643543 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb65409b1d0 con 0x7fb6640ff6e0 2023-09-23T09:28:40.625 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:28:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb6500a1630 msgr2=0x7fb6500a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb6500a1630 0x7fb6500a3af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fd0 tx=0x7fb660005f60 comp rx=0 tx=0).stop 2023-09-23T09:28:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 msgr2=0x7fb664198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb664198cc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb654009990 tx=0x7fb654005e80 comp rx=0 tx=0).stop 2023-09-23T09:28:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 shutdown_connections 2023-09-23T09:28:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb664100a00 0x7fb6641a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6640ff6e0 0x7fb664198cc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb6500a1630 0x7fb6500a3af0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 --2- 172.21.15.45:0/3819643543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb664100040 0x7fb664199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 >> 172.21.15.45:0/3819643543 conn(0x7fb6640fb280 msgr2=0x7fb6640fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 shutdown_connections 2023-09-23T09:28:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.625+0000 7fb66b3c3700 1 -- 172.21.15.45:0/3819643543 wait complete. 2023-09-23T09:28:40.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 -- 172.21.15.45:0/4109244390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e4100f00 msgr2=0x7fc5e4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 --2- 172.21.15.45:0/4109244390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e4100f00 0x7fc5e4101360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40099c0 tx=0x7fc5d4009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 -- 172.21.15.45:0/4109244390 shutdown_connections 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 --2- 172.21.15.45:0/4109244390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e41018a0 0x7fc5e4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 --2- 172.21.15.45:0/4109244390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5e40ffd10 0x7fc5e4100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 --2- 172.21.15.45:0/4109244390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e4100f00 0x7fc5e4101360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 -- 172.21.15.45:0/4109244390 >> 172.21.15.45:0/4109244390 conn(0x7fc5e40fb280 msgr2=0x7fc5e40fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 -- 172.21.15.45:0/4109244390 shutdown_connections 2023-09-23T09:28:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.765+0000 7fc5ec753700 1 -- 172.21.15.45:0/4109244390 wait complete. 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 Processor -- start 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 -- start start 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5e40ffd10 0x7fc5e4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e4100f00 0x7fc5e41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e41018a0 0x7fc5e41a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e41a08e0 con 0x7fc5e41018a0 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e41a0a20 con 0x7fc5e40ffd10 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ec753700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e41a0bb0 con 0x7fc5e4100f00 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5eacf0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e41018a0 0x7fc5e41a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5eacf0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e41018a0 0x7fc5e41a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59490/0 (socket says 172.21.15.45:59490) 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5eacf0700 1 -- 172.21.15.45:0/1680156473 learned_addr learned my addr 172.21.15.45:0/1680156473 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5e9cee700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e4100f00 0x7fc5e41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.766+0000 7fc5ea4ef700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5e40ffd10 0x7fc5e4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5eacf0700 1 -- 172.21.15.45:0/1680156473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e4100f00 msgr2=0x7fc5e41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5eacf0700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e4100f00 0x7fc5e41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5eacf0700 1 -- 172.21.15.45:0/1680156473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5e40ffd10 msgr2=0x7fc5e4198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5eacf0700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5e40ffd10 0x7fc5e4198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5eacf0700 1 -- 172.21.15.45:0/1680156473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d40095f0 con 0x7fc5e41018a0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5eacf0700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e41018a0 0x7fc5e41a0310 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00ef30 tx=0x7fc5dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5db7fe700 1 -- 172.21.15.45:0/1680156473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc018070 con 0x7fc5e41018a0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5db7fe700 1 -- 172.21.15.45:0/1680156473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5dc00f040 con 0x7fc5e41018a0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e41a0e30 con 0x7fc5e41018a0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5db7fe700 1 -- 172.21.15.45:0/1680156473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00cd00 con 0x7fc5e41018a0 2023-09-23T09:28:40.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.767+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e41a1260 con 0x7fc5e41018a0 2023-09-23T09:28:40.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.768+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e4193460 con 0x7fc5e41018a0 2023-09-23T09:28:40.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.769+0000 7fc5db7fe700 1 -- 172.21.15.45:0/1680156473 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fc5dc003a00 con 0x7fc5e41018a0 2023-09-23T09:28:40.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.769+0000 7fc5db7fe700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5d00a1630 0x7fc5d00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:40.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.770+0000 7fc5db7fe700 1 -- 172.21.15.45:0/1680156473 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5dc0cd380 con 0x7fc5e41018a0 2023-09-23T09:28:40.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.770+0000 7fc5ea4ef700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5d00a1630 0x7fc5d00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:40.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.770+0000 7fc5ea4ef700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5d00a1630 0x7fc5d00a3af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4100d60 tx=0x7fc5e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.773+0000 7fc5db7fe700 1 -- 172.21.15.45:0/1680156473 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc5dc096a50 con 0x7fc5e41018a0 2023-09-23T09:28:40.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:40 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2429758616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:40.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:40 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2959779866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:28:40.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:40 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2429758616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:40.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:40 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2959779866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:28:40.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.972+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5e410c800 con 0x7fc5e41018a0 2023-09-23T09:28:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.972+0000 7fc5db7fe700 1 -- 172.21.15.45:0/1680156473 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5dc09a070 con 0x7fc5e41018a0 2023-09-23T09:28:40.974 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:28:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5d00a1630 msgr2=0x7fc5d00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5d00a1630 0x7fc5d00a3af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4100d60 tx=0x7fc5e000b410 comp rx=0 tx=0).stop 2023-09-23T09:28:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e41018a0 msgr2=0x7fc5e41a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e41018a0 0x7fc5e41a0310 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00ef30 tx=0x7fc5dc00c3a0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 shutdown_connections 2023-09-23T09:28:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5e41018a0 0x7fc5e41a0310 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5e40ffd10 0x7fc5e4198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5d00a1630 0x7fc5d00a3af0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 --2- 172.21.15.45:0/1680156473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e4100f00 0x7fc5e41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.974+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 >> 172.21.15.45:0/1680156473 conn(0x7fc5e40fb280 msgr2=0x7fc5e4106860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.975+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 shutdown_connections 2023-09-23T09:28:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:40.975+0000 7fc5ec753700 1 -- 172.21.15.45:0/1680156473 wait complete. 2023-09-23T09:28:40.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:28:41.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 -- 172.21.15.45:0/579227099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a04100ef0 msgr2=0x7f8a04101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 --2- 172.21.15.45:0/579227099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a04100ef0 0x7f8a04101350 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f89f80099c0 tx=0x7f89f8009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 -- 172.21.15.45:0/579227099 shutdown_connections 2023-09-23T09:28:41.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 --2- 172.21.15.45:0/579227099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a04101890 0x7f8a04108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 --2- 172.21.15.45:0/579227099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a040ffd00 0x7f8a04100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 --2- 172.21.15.45:0/579227099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a04100ef0 0x7f8a04101350 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 -- 172.21.15.45:0/579227099 >> 172.21.15.45:0/579227099 conn(0x7f8a040fb250 msgr2=0x7f8a040fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 -- 172.21.15.45:0/579227099 shutdown_connections 2023-09-23T09:28:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.112+0000 7f8a0a895700 1 -- 172.21.15.45:0/579227099 wait complete. 2023-09-23T09:28:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 Processor -- start 2023-09-23T09:28:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 -- start start 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a040ffd00 0x7f8a04068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a04100ef0 0x7f8a0419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a04101890 0x7f8a0419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0419e5f0 con 0x7f8a040ffd00 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0419e760 con 0x7f8a04100ef0 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a0a895700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0419e920 con 0x7f8a04101890 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a040ffd00 0x7f8a04068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a040ffd00 0x7f8a04068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59500/0 (socket says 172.21.15.45:59500) 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 -- 172.21.15.45:0/2165488342 learned_addr learned my addr 172.21.15.45:0/2165488342 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a037fe700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a04100ef0 0x7f8a0419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a08e32700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a04101890 0x7f8a0419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 -- 172.21.15.45:0/2165488342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a04101890 msgr2=0x7f8a0419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a04101890 0x7f8a0419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 -- 172.21.15.45:0/2165488342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a04100ef0 msgr2=0x7f8a0419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a04100ef0 0x7f8a0419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.113+0000 7f8a03fff700 1 -- 172.21.15.45:0/2165488342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80095f0 con 0x7f8a040ffd00 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.114+0000 7f8a03fff700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a040ffd00 0x7f8a04068af0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f89f400e970 tx=0x7f89f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.114+0000 7f8a017fa700 1 -- 172.21.15.45:0/2165488342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4009790 con 0x7f8a040ffd00 2023-09-23T09:28:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.114+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0419ec00 con 0x7f8a040ffd00 2023-09-23T09:28:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.114+0000 7f8a017fa700 1 -- 172.21.15.45:0/2165488342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89f400eec0 con 0x7f8a040ffd00 2023-09-23T09:28:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.114+0000 7f8a017fa700 1 -- 172.21.15.45:0/2165488342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4003cf0 con 0x7f8a040ffd00 2023-09-23T09:28:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.114+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0419f110 con 0x7f8a040ffd00 2023-09-23T09:28:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.115+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a04107290 con 0x7f8a040ffd00 2023-09-23T09:28:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.116+0000 7f8a017fa700 1 -- 172.21.15.45:0/2165488342 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f89f400ccd0 con 0x7f8a040ffd00 2023-09-23T09:28:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.116+0000 7f8a017fa700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89ec0a1670 0x7f89ec0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.117+0000 7f8a037fe700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89ec0a1670 0x7f89ec0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.117+0000 7f8a017fa700 1 -- 172.21.15.45:0/2165488342 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89f4014070 con 0x7f8a040ffd00 2023-09-23T09:28:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.117+0000 7f8a037fe700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89ec0a1670 0x7f89ec0a3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b580 tx=0x7f89f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.121+0000 7f8a017fa700 1 -- 172.21.15.45:0/2165488342 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f89f4096770 con 0x7f8a040ffd00 2023-09-23T09:28:41.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.371+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a0402cfe0 con 0x7f8a040ffd00 2023-09-23T09:28:41.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.372+0000 7f8a017fa700 1 -- 172.21.15.45:0/2165488342 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7f89f4099d90 con 0x7f8a040ffd00 2023-09-23T09:28:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:41.374 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89ec0a1670 msgr2=0x7f89ec0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89ec0a1670 0x7f89ec0a3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b580 tx=0x7f89f8005fb0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a040ffd00 msgr2=0x7f8a04068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a040ffd00 0x7f8a04068af0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f89f400e970 tx=0x7f89f400ec80 comp rx=0 tx=0).stop 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 shutdown_connections 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89ec0a1670 0x7f89ec0a3b30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a04101890 0x7f8a0419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a040ffd00 0x7f8a04068af0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 --2- 172.21.15.45:0/2165488342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a04100ef0 0x7f8a0419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 >> 172.21.15.45:0/2165488342 conn(0x7f8a040fb250 msgr2=0x7f8a041043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 shutdown_connections 2023-09-23T09:28:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.374+0000 7f8a0a895700 1 -- 172.21.15.45:0/2165488342 wait complete. 2023-09-23T09:28:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:28:41.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 -- 172.21.15.45:0/3631148737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268106ae0 msgr2=0x7fc268108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/3631148737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268106ae0 0x7fc268108f50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b0a0 tx=0x7fc26000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 -- 172.21.15.45:0/3631148737 shutdown_connections 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/3631148737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268106ae0 0x7fc268108f50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/3631148737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc268101880 0x7fc268103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/3631148737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2681041b0 0x7fc2681065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 -- 172.21.15.45:0/3631148737 >> 172.21.15.45:0/3631148737 conn(0x7fc2680fb250 msgr2=0x7fc2680fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 -- 172.21.15.45:0/3631148737 shutdown_connections 2023-09-23T09:28:41.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.514+0000 7fc26ccc1700 1 -- 172.21.15.45:0/3631148737 wait complete. 2023-09-23T09:28:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 Processor -- start 2023-09-23T09:28:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 -- start start 2023-09-23T09:28:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268101880 0x7fc268196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2681041b0 0x7fc268197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc268106ae0 0x7fc26819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26819e680 con 0x7fc268101880 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26819e7c0 con 0x7fc268106ae0 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26ccc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26819e950 con 0x7fc2681041b0 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268101880 0x7fc268196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268101880 0x7fc268196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59514/0 (socket says 172.21.15.45:59514) 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 -- 172.21.15.45:0/101606967 learned_addr learned my addr 172.21.15.45:0/101606967 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc265d9b700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2681041b0 0x7fc268197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc266d9d700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc268106ae0 0x7fc26819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 -- 172.21.15.45:0/101606967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2681041b0 msgr2=0x7fc268197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2681041b0 0x7fc268197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 -- 172.21.15.45:0/101606967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc268106ae0 msgr2=0x7fc26819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc268106ae0 0x7fc26819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc26659c700 1 -- 172.21.15.45:0/101606967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009d00 con 0x7fc268101880 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.515+0000 7fc266d9d700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc268106ae0 0x7fc26819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.516+0000 7fc26659c700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268101880 0x7fc268196b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9a0 tx=0x7fc25800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.516+0000 7fc25f7fe700 1 -- 172.21.15.45:0/101606967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258009790 con 0x7fc268101880 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.516+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26819ec30 con 0x7fc268101880 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.516+0000 7fc25f7fe700 1 -- 172.21.15.45:0/101606967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc25800cbd0 con 0x7fc268101880 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.516+0000 7fc25f7fe700 1 -- 172.21.15.45:0/101606967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258018680 con 0x7fc268101880 2023-09-23T09:28:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.516+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26819f140 con 0x7fc268101880 2023-09-23T09:28:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.517+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2680fcdd0 con 0x7fc268101880 2023-09-23T09:28:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.518+0000 7fc25f7fe700 1 -- 172.21.15.45:0/101606967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fc25800cd40 con 0x7fc268101880 2023-09-23T09:28:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.519+0000 7fc25f7fe700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2540a1670 0x7fc2540a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.519+0000 7fc25f7fe700 1 -- 172.21.15.45:0/101606967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc258014070 con 0x7fc268101880 2023-09-23T09:28:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.519+0000 7fc265d9b700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2540a1670 0x7fc2540a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.519+0000 7fc265d9b700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2540a1670 0x7fc2540a3b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc25000ac50 tx=0x7fc25000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:41.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.522+0000 7fc25f7fe700 1 -- 172.21.15.45:0/101606967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc258096aa0 con 0x7fc268101880 2023-09-23T09:28:41.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.724+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc26810c820 con 0x7fc268101880 2023-09-23T09:28:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.726+0000 7fc25f7fe700 1 -- 172.21.15.45:0/101606967 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7fc25809a0c0 con 0x7fc268101880 2023-09-23T09:28:41.728 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:41.728 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:51.253687+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2540a1670 msgr2=0x7fc2540a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2540a1670 0x7fc2540a3b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc25000ac50 tx=0x7fc25000a380 comp rx=0 tx=0).stop 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268101880 msgr2=0x7fc268196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268101880 0x7fc268196b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9a0 tx=0x7fc25800ecb0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 shutdown_connections 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc268106ae0 0x7fc26819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc268101880 0x7fc268196b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc2540a1670 0x7fc2540a3b30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 --2- 172.21.15.45:0/101606967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2681041b0 0x7fc268197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 >> 172.21.15.45:0/101606967 conn(0x7fc2680fb250 msgr2=0x7fc2681024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 shutdown_connections 2023-09-23T09:28:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.728+0000 7fc26ccc1700 1 -- 172.21.15.45:0/101606967 wait complete. 2023-09-23T09:28:41.741 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:41 smithi045 ceph-mon[98627]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:41.741 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:41 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3819643543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:28:41.741 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:41 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1680156473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:28:41.741 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:41 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2165488342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:41.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:28:41.743 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:28:41.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:41 smithi098 ceph-mon[102386]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:41.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:41 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3819643543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:28:41.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:41 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1680156473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:28:41.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:41 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2165488342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:41 smithi123 ceph-mon[103577]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:41 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3819643543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:28:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:41 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1680156473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:28:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:41 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2165488342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2442847036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 msgr2=0x7f1cac108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2442847036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac108180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b0d0 tx=0x7f1ca400b3e0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2442847036 shutdown_connections 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2442847036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac108180 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2442847036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cac0ffd40 0x7f1cac100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2442847036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cac100f30 0x7f1cac101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2442847036 >> 172.21.15.45:0/2442847036 conn(0x7f1cac0fb2b0 msgr2=0x7f1cac0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2442847036 shutdown_connections 2023-09-23T09:28:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.869+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2442847036 wait complete. 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 Processor -- start 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 -- start start 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cac0ffd40 0x7f1cac195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cac100f30 0x7f1cac196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac19d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac19d6b0 con 0x7f1cac1018d0 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac19d7f0 con 0x7f1cac100f30 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb41ae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac19d980 con 0x7f1cac0ffd40 2023-09-23T09:28:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb274b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac19d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb274b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac19d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59532/0 (socket says 172.21.15.45:59532) 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.870+0000 7f1cb274b700 1 -- 172.21.15.45:0/2478000102 learned_addr learned my addr 172.21.15.45:0/2478000102 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb1f4a700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cac0ffd40 0x7f1cac195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb1749700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cac100f30 0x7f1cac196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb274b700 1 -- 172.21.15.45:0/2478000102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cac0ffd40 msgr2=0x7f1cac195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb274b700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cac0ffd40 0x7f1cac195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb274b700 1 -- 172.21.15.45:0/2478000102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cac100f30 msgr2=0x7f1cac196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb274b700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cac100f30 0x7f1cac196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb274b700 1 -- 172.21.15.45:0/2478000102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009d00 con 0x7f1cac1018d0 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb1749700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cac100f30 0x7f1cac196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb274b700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac19d0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4007990 tx=0x7f1ca4008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:41.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1c9effd700 1 -- 172.21.15.45:0/2478000102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400e040 con 0x7f1cac1018d0 2023-09-23T09:28:41.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1c9effd700 1 -- 172.21.15.45:0/2478000102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca400be30 con 0x7f1cac1018d0 2023-09-23T09:28:41.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac19dc00 con 0x7f1cac1018d0 2023-09-23T09:28:41.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1c9effd700 1 -- 172.21.15.45:0/2478000102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca401d6d0 con 0x7f1cac1018d0 2023-09-23T09:28:41.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.871+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac19e0d0 con 0x7f1cac1018d0 2023-09-23T09:28:41.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.872+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac106c20 con 0x7f1cac1018d0 2023-09-23T09:28:41.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.873+0000 7f1c9effd700 1 -- 172.21.15.45:0/2478000102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f1ca40051b0 con 0x7f1cac1018d0 2023-09-23T09:28:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.874+0000 7f1c9effd700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1c980a16b0 0x7f1c980a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.874+0000 7f1c9effd700 1 -- 172.21.15.45:0/2478000102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ca40ce4c0 con 0x7f1cac1018d0 2023-09-23T09:28:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.874+0000 7f1cb1f4a700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1c980a16b0 0x7f1c980a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.874+0000 7f1cb1f4a700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1c980a16b0 0x7f1c980a3b70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1cac100d90 tx=0x7f1ca8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:41.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:41.877+0000 7f1c9effd700 1 -- 172.21.15.45:0/2478000102 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1ca4097b90 con 0x7f1cac1018d0 2023-09-23T09:28:42.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.077+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cac10b740 con 0x7f1cac1018d0 2023-09-23T09:28:42.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.077+0000 7f1c9effd700 1 -- 172.21.15.45:0/2478000102 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ca409b1b0 con 0x7f1cac1018d0 2023-09-23T09:28:42.079 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:28:42.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1c980a16b0 msgr2=0x7f1c980a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1c980a16b0 0x7f1c980a3b70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1cac100d90 tx=0x7f1ca8009260 comp rx=0 tx=0).stop 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 msgr2=0x7f1cac19d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac19d0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4007990 tx=0x7f1ca4008eb0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 shutdown_connections 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac1018d0 0x7f1cac19d0e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f1c980a16b0 0x7f1c980a3b70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cac0ffd40 0x7f1cac195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 --2- 172.21.15.45:0/2478000102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cac100f30 0x7f1cac196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 >> 172.21.15.45:0/2478000102 conn(0x7f1cac0fb2b0 msgr2=0x7f1cac0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 shutdown_connections 2023-09-23T09:28:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.080+0000 7f1cb41ae700 1 -- 172.21.15.45:0/2478000102 wait complete. 2023-09-23T09:28:42.094 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:28:42.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.220+0000 7f5d74b24700 1 -- 172.21.15.45:0/2369952690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d70100f30 msgr2=0x7f5d70101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.220+0000 7f5d74b24700 1 --2- 172.21.15.45:0/2369952690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d70100f30 0x7f5d70101390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d600099c0 tx=0x7f5d60009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 -- 172.21.15.45:0/2369952690 shutdown_connections 2023-09-23T09:28:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 --2- 172.21.15.45:0/2369952690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d701018d0 0x7f5d70108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 --2- 172.21.15.45:0/2369952690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d700ffd40 0x7f5d70100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 --2- 172.21.15.45:0/2369952690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d70100f30 0x7f5d70101390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 -- 172.21.15.45:0/2369952690 >> 172.21.15.45:0/2369952690 conn(0x7f5d700fb2b0 msgr2=0x7f5d700fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 -- 172.21.15.45:0/2369952690 shutdown_connections 2023-09-23T09:28:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 -- 172.21.15.45:0/2369952690 wait complete. 2023-09-23T09:28:42.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 Processor -- start 2023-09-23T09:28:42.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.221+0000 7f5d74b24700 1 -- start start 2023-09-23T09:28:42.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d700ffd40 0x7f5d70103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d70100f30 0x7f5d701025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d701018d0 0x7f5d70102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d701032b0 con 0x7f5d701018d0 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d70103420 con 0x7f5d700ffd40 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d701a0b60 con 0x7f5d70100f30 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d701018d0 0x7f5d70102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d701018d0 0x7f5d70102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59546/0 (socket says 172.21.15.45:59546) 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 -- 172.21.15.45:0/769620527 learned_addr learned my addr 172.21.15.45:0/769620527 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6e59c700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d700ffd40 0x7f5d70103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6dd9b700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d70100f30 0x7f5d701025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 -- 172.21.15.45:0/769620527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d70100f30 msgr2=0x7f5d701025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d70100f30 0x7f5d701025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 -- 172.21.15.45:0/769620527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d700ffd40 msgr2=0x7f5d70103f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d700ffd40 0x7f5d70103f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 -- 172.21.15.45:0/769620527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600095f0 con 0x7f5d701018d0 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6dd9b700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d70100f30 0x7f5d701025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d6ed9d700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d701018d0 0x7f5d70102b10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800caf0 tx=0x7f5d6800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d5f7fe700 1 -- 172.21.15.45:0/769620527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800a9a0 con 0x7f5d701018d0 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d5f7fe700 1 -- 172.21.15.45:0/769620527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d68012420 con 0x7f5d701018d0 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d701a0e40 con 0x7f5d701018d0 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d5f7fe700 1 -- 172.21.15.45:0/769620527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680115e0 con 0x7f5d701018d0 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.222+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d701a12c0 con 0x7f5d701018d0 2023-09-23T09:28:42.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.223+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d70107dd0 con 0x7f5d701018d0 2023-09-23T09:28:42.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.224+0000 7f5d5f7fe700 1 -- 172.21.15.45:0/769620527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f5d68012590 con 0x7f5d701018d0 2023-09-23T09:28:42.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.225+0000 7f5d5f7fe700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5d580a1630 0x7f5d580a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.225+0000 7f5d5f7fe700 1 -- 172.21.15.45:0/769620527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5d680cd040 con 0x7f5d701018d0 2023-09-23T09:28:42.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.225+0000 7f5d6e59c700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5d580a1630 0x7f5d580a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.225+0000 7f5d6e59c700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5d580a1630 0x7f5d580a3af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d70100d90 tx=0x7f5d6400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:42.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.229+0000 7f5d5f7fe700 1 -- 172.21.15.45:0/769620527 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5d68096790 con 0x7f5d701018d0 2023-09-23T09:28:42.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.432+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d7010b730 con 0x7f5d701018d0 2023-09-23T09:28:42.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.432+0000 7f5d5f7fe700 1 -- 172.21.15.45:0/769620527 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d68007b20 con 0x7f5d701018d0 2023-09-23T09:28:42.434 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:28:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5d580a1630 msgr2=0x7f5d580a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5d580a1630 0x7f5d580a3af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d70100d90 tx=0x7f5d6400b410 comp rx=0 tx=0).stop 2023-09-23T09:28:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d701018d0 msgr2=0x7f5d70102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d701018d0 0x7f5d70102b10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800caf0 tx=0x7f5d6800ce00 comp rx=0 tx=0).stop 2023-09-23T09:28:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 shutdown_connections 2023-09-23T09:28:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d701018d0 0x7f5d70102b10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d700ffd40 0x7f5d70103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5d580a1630 0x7f5d580a3af0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 --2- 172.21.15.45:0/769620527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d70100f30 0x7f5d701025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 >> 172.21.15.45:0/769620527 conn(0x7f5d700fb2b0 msgr2=0x7f5d700fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 shutdown_connections 2023-09-23T09:28:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.434+0000 7f5d74b24700 1 -- 172.21.15.45:0/769620527 wait complete. 2023-09-23T09:28:42.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:28:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 -- 172.21.15.45:0/448650506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 msgr2=0x7f7294100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 --2- 172.21.15.45:0/448650506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294100130 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72900099c0 tx=0x7f7290009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 -- 172.21.15.45:0/448650506 shutdown_connections 2023-09-23T09:28:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 --2- 172.21.15.45:0/448650506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72941018a0 0x7f7294108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 --2- 172.21.15.45:0/448650506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294100130 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 --2- 172.21.15.45:0/448650506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7294100f00 0x7f7294101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 -- 172.21.15.45:0/448650506 >> 172.21.15.45:0/448650506 conn(0x7f72940fb280 msgr2=0x7f72940fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 -- 172.21.15.45:0/448650506 shutdown_connections 2023-09-23T09:28:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.575+0000 7f729b9b0700 1 -- 172.21.15.45:0/448650506 wait complete. 2023-09-23T09:28:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 Processor -- start 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 -- start start 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7294100f00 0x7f729419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72941018a0 0x7f729419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419e630 con 0x7f72940ffd10 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419e7a0 con 0x7f7294100f00 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729b9b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419e960 con 0x7f72941018a0 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59556/0 (socket says 172.21.15.45:59556) 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 -- 172.21.15.45:0/775387794 learned_addr learned my addr 172.21.15.45:0/775387794 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f7298f4b700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7294100f00 0x7f729419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 -- 172.21.15.45:0/775387794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72941018a0 msgr2=0x7f729419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72941018a0 0x7f729419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 -- 172.21.15.45:0/775387794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7294100f00 msgr2=0x7f729419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7294100f00 0x7f729419b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.576+0000 7f729974c700 1 -- 172.21.15.45:0/775387794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900095f0 con 0x7f72940ffd10 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.577+0000 7f729974c700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294068af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72900036f0 tx=0x7f72900057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.577+0000 7f728a7fc700 1 -- 172.21.15.45:0/775387794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729001d070 con 0x7f72940ffd10 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.577+0000 7f728a7fc700 1 -- 172.21.15.45:0/775387794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72900041c0 con 0x7f72940ffd10 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.577+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729419ebe0 con 0x7f72940ffd10 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.577+0000 7f728a7fc700 1 -- 172.21.15.45:0/775387794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000f460 con 0x7f72940ffd10 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.577+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729419f090 con 0x7f72940ffd10 2023-09-23T09:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.578+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729410b5a0 con 0x7f72940ffd10 2023-09-23T09:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.579+0000 7f728a7fc700 1 -- 172.21.15.45:0/775387794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f7290022600 con 0x7f72940ffd10 2023-09-23T09:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.579+0000 7f728a7fc700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f72800a1630 0x7f72800a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.579+0000 7f7298f4b700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f72800a1630 0x7f72800a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.579+0000 7f728a7fc700 1 -- 172.21.15.45:0/775387794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72900ce4b0 con 0x7f72940ffd10 2023-09-23T09:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.580+0000 7f7298f4b700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f72800a1630 0x7f72800a3af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7284005ea0 tx=0x7f7284005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.584+0000 7f728a7fc700 1 -- 172.21.15.45:0/775387794 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7290097b80 con 0x7f72940ffd10 2023-09-23T09:28:42.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.784+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7294106bc0 con 0x7f72940ffd10 2023-09-23T09:28:42.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.785+0000 7f728a7fc700 1 -- 172.21.15.45:0/775387794 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f729009b1a0 con 0x7f72940ffd10 2023-09-23T09:28:42.787 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f72800a1630 msgr2=0x7f72800a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f72800a1630 0x7f72800a3af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7284005ea0 tx=0x7f7284005e30 comp rx=0 tx=0).stop 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 msgr2=0x7f7294068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294068af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72900036f0 tx=0x7f72900057d0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 shutdown_connections 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f72800a1630 0x7f72800a3af0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72941018a0 0x7f729419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72940ffd10 0x7f7294068af0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 --2- 172.21.15.45:0/775387794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7294100f00 0x7f729419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 >> 172.21.15.45:0/775387794 conn(0x7f72940fb280 msgr2=0x7f72940fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:42.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 shutdown_connections 2023-09-23T09:28:42.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.787+0000 7f729b9b0700 1 -- 172.21.15.45:0/775387794 wait complete. 2023-09-23T09:28:42.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:42 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/101606967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:42.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:42 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2478000102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:28:42.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:42 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/101606967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:42.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:42 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2478000102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:28:42.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:28:42.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:42 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/101606967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:42.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:42 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2478000102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:28:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 -- 172.21.15.45:0/3171825042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00ffd00 msgr2=0x7f58c0100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 --2- 172.21.15.45:0/3171825042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00ffd00 0x7f58c0100120 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58b40099c0 tx=0x7f58b4009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 -- 172.21.15.45:0/3171825042 shutdown_connections 2023-09-23T09:28:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 --2- 172.21.15.45:0/3171825042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c0101890 0x7f58c0108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 --2- 172.21.15.45:0/3171825042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00ffd00 0x7f58c0100120 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 --2- 172.21.15.45:0/3171825042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58c0100ef0 0x7f58c0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 -- 172.21.15.45:0/3171825042 >> 172.21.15.45:0/3171825042 conn(0x7f58c00fb250 msgr2=0x7f58c00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:42.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 -- 172.21.15.45:0/3171825042 shutdown_connections 2023-09-23T09:28:42.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.924+0000 7f58c71b0700 1 -- 172.21.15.45:0/3171825042 wait complete. 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 Processor -- start 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 -- start start 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c00ffd00 0x7f58c0196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58c0100ef0 0x7f58c0197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c0101890 0x7f58c019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c019e680 con 0x7f58c0101890 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c019e7c0 con 0x7f58c0100ef0 2023-09-23T09:28:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c71b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c019e950 con 0x7f58c00ffd00 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c574d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c0101890 0x7f58c019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c574d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c0101890 0x7f58c019e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59574/0 (socket says 172.21.15.45:59574) 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58c574d700 1 -- 172.21.15.45:0/1159359498 learned_addr learned my addr 172.21.15.45:0/1159359498 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.925+0000 7f58bffff700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58c0100ef0 0x7f58c0197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c4f4c700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c00ffd00 0x7f58c0196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c574d700 1 -- 172.21.15.45:0/1159359498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c00ffd00 msgr2=0x7f58c0196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c574d700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c00ffd00 0x7f58c0196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c574d700 1 -- 172.21.15.45:0/1159359498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58c0100ef0 msgr2=0x7f58c0197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c574d700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58c0100ef0 0x7f58c0197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c574d700 1 -- 172.21.15.45:0/1159359498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40095f0 con 0x7f58c0101890 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c4f4c700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c00ffd00 0x7f58c0196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:42.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c574d700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c0101890 0x7f58c019e0b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f58b8007ee0 tx=0x7f58b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58bdffb700 1 -- 172.21.15.45:0/1159359498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b800ddd0 con 0x7f58c0101890 2023-09-23T09:28:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58bdffb700 1 -- 172.21.15.45:0/1159359498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58b800f040 con 0x7f58c0101890 2023-09-23T09:28:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c019ec30 con 0x7f58c0101890 2023-09-23T09:28:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58bdffb700 1 -- 172.21.15.45:0/1159359498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b80048e0 con 0x7f58c0101890 2023-09-23T09:28:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.926+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c019f140 con 0x7f58c0101890 2023-09-23T09:28:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.927+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c0106c80 con 0x7f58c0101890 2023-09-23T09:28:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.928+0000 7f58bdffb700 1 -- 172.21.15.45:0/1159359498 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f58b8003a00 con 0x7f58c0101890 2023-09-23T09:28:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.929+0000 7f58bdffb700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f58b00a16f0 0x7f58b00a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.929+0000 7f58c4f4c700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f58b00a16f0 0x7f58b00a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.929+0000 7f58bdffb700 1 -- 172.21.15.45:0/1159359498 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58b80cc770 con 0x7f58c0101890 2023-09-23T09:28:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.929+0000 7f58c4f4c700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f58b00a16f0 0x7f58b00a3bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009990 tx=0x7f58b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:42.933+0000 7f58bdffb700 1 -- 172.21.15.45:0/1159359498 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f58b804eec0 con 0x7f58c0101890 2023-09-23T09:28:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.180+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c010c610 con 0x7f58c0101890 2023-09-23T09:28:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.181+0000 7f58bdffb700 1 -- 172.21.15.45:0/1159359498 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7f58b8095db0 con 0x7f58c0101890 2023-09-23T09:28:43.183 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:43.184 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:28:43.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f58b00a16f0 msgr2=0x7f58b00a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f58b00a16f0 0x7f58b00a3bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009990 tx=0x7f58b4005fd0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c0101890 msgr2=0x7f58c019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c0101890 0x7f58c019e0b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f58b8007ee0 tx=0x7f58b800d3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 shutdown_connections 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c0101890 0x7f58c019e0b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f58b00a16f0 0x7f58b00a3bb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c00ffd00 0x7f58c0196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 --2- 172.21.15.45:0/1159359498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58c0100ef0 0x7f58c0197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 >> 172.21.15.45:0/1159359498 conn(0x7f58c00fb250 msgr2=0x7f58c00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 shutdown_connections 2023-09-23T09:28:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.183+0000 7f58c71b0700 1 -- 172.21.15.45:0/1159359498 wait complete. 2023-09-23T09:28:43.187 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:28:43.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:28:43.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2845979110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0102930 msgr2=0x7fa8e0109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2845979110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0102930 0x7fa8e0109250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800b0a0 tx=0x7fa8d800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2845979110 shutdown_connections 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2845979110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0102930 0x7fa8e0109250 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2845979110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0100e20 0x7fa8e0101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2845979110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8e0101f70 0x7fa8e01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2845979110 >> 172.21.15.45:0/2845979110 conn(0x7fa8e00fc370 msgr2=0x7fa8e00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2845979110 shutdown_connections 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2845979110 wait complete. 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 Processor -- start 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.321+0000 7fa8e6f2f700 1 -- start start 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e6f2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8e0100e20 0x7fa8e0196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e6f2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0101f70 0x7fa8e01971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e6f2f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0102930 0x7fa8e019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e6f2f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e019e810 con 0x7fa8e0101f70 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e6f2f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e019e980 con 0x7fa8e0102930 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e6f2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e019eb40 con 0x7fa8e0100e20 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0101f70 0x7fa8e01971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0101f70 0x7fa8e01971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59586/0 (socket says 172.21.15.45:59586) 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 -- 172.21.15.45:0/2525302267 learned_addr learned my addr 172.21.15.45:0/2525302267 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e4ccb700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8e0100e20 0x7fa8e0196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e54cc700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0102930 0x7fa8e019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 -- 172.21.15.45:0/2525302267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8e0100e20 msgr2=0x7fa8e0196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8e0100e20 0x7fa8e0196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 -- 172.21.15.45:0/2525302267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0102930 msgr2=0x7fa8e019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0102930 0x7fa8e019e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 -- 172.21.15.45:0/2525302267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8009d00 con 0x7fa8e0101f70 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e54cc700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0102930 0x7fa8e019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8dffff700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0101f70 0x7fa8e01971e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00d980 tx=0x7fa8cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8ddffb700 1 -- 172.21.15.45:0/2525302267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc004a30 con 0x7fa8e0101f70 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8ddffb700 1 -- 172.21.15.45:0/2525302267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8cc0054e0 con 0x7fa8e0101f70 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e019ee20 con 0x7fa8e0101f70 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.322+0000 7fa8ddffb700 1 -- 172.21.15.45:0/2525302267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc009c90 con 0x7fa8e0101f70 2023-09-23T09:28:43.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.323+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e019f2a0 con 0x7fa8e0101f70 2023-09-23T09:28:43.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.324+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e0108360 con 0x7fa8e0101f70 2023-09-23T09:28:43.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.325+0000 7fa8ddffb700 1 -- 172.21.15.45:0/2525302267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fa8cc004c20 con 0x7fa8e0101f70 2023-09-23T09:28:43.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.325+0000 7fa8ddffb700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8d00a16f0 0x7fa8d00a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.325+0000 7fa8e4ccb700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8d00a16f0 0x7fa8d00a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.326+0000 7fa8ddffb700 1 -- 172.21.15.45:0/2525302267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8cc0ccca0 con 0x7fa8e0101f70 2023-09-23T09:28:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.326+0000 7fa8e4ccb700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8d00a16f0 0x7fa8d00a3bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0197b70 tx=0x7fa8d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:43.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.330+0000 7fa8ddffb700 1 -- 172.21.15.45:0/2525302267 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa8cc0d1070 con 0x7fa8e0101f70 2023-09-23T09:28:43.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.534+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8e010c6a0 con 0x7fa8e0101f70 2023-09-23T09:28:43.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.536+0000 7fa8ddffb700 1 -- 172.21.15.45:0/2525302267 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7fa8cc017020 con 0x7fa8e0101f70 2023-09-23T09:28:43.539 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:43.539 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:27:51.253687+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:28:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8d00a16f0 msgr2=0x7fa8d00a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8d00a16f0 0x7fa8d00a3bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0197b70 tx=0x7fa8d400b410 comp rx=0 tx=0).stop 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0101f70 msgr2=0x7fa8e01971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0101f70 0x7fa8e01971e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00d980 tx=0x7fa8cc00dc90 comp rx=0 tx=0).stop 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 shutdown_connections 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0102930 0x7fa8e019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fa8d00a16f0 0x7fa8d00a3bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8e0100e20 0x7fa8e0196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 --2- 172.21.15.45:0/2525302267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8e0101f70 0x7fa8e01971e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 >> 172.21.15.45:0/2525302267 conn(0x7fa8e00fc370 msgr2=0x7fa8e01054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 shutdown_connections 2023-09-23T09:28:43.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.538+0000 7fa8e6f2f700 1 -- 172.21.15.45:0/2525302267 wait complete. 2023-09-23T09:28:43.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:43 smithi045 ceph-mon[98627]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:43.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:43 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/769620527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:28:43.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:43 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/775387794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:28:43.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:43 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1159359498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:43.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 -- 172.21.15.45:0/2975195637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020100e20 msgr2=0x7f2020101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 --2- 172.21.15.45:0/2975195637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020100e20 0x7f2020101240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2014009990 tx=0x7f2014009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 -- 172.21.15.45:0/2975195637 shutdown_connections 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 --2- 172.21.15.45:0/2975195637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020102930 0x7f2020109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 --2- 172.21.15.45:0/2975195637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020100e20 0x7f2020101240 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 --2- 172.21.15.45:0/2975195637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2020101f70 0x7f20201023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 -- 172.21.15.45:0/2975195637 >> 172.21.15.45:0/2975195637 conn(0x7f20200fc370 msgr2=0x7f20200fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 -- 172.21.15.45:0/2975195637 shutdown_connections 2023-09-23T09:28:43.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.676+0000 7f2026aea700 1 -- 172.21.15.45:0/2975195637 wait complete. 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 Processor -- start 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 -- start start 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100e20 0x7f2020196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020101f70 0x7f2020197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2020102930 0x7f202019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202019e680 con 0x7f2020101f70 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202019e7c0 con 0x7f2020102930 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2026aea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202019e950 con 0x7f2020100e20 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f201ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020101f70 0x7f2020197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f201ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020101f70 0x7f2020197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59600/0 (socket says 172.21.15.45:59600) 2023-09-23T09:28:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f201ffff700 1 -- 172.21.15.45:0/1239062797 learned_addr learned my addr 172.21.15.45:0/1239062797 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2025087700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2020102930 0x7f202019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f2024886700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100e20 0x7f2020196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f201ffff700 1 -- 172.21.15.45:0/1239062797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100e20 msgr2=0x7f2020196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.677+0000 7f201ffff700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100e20 0x7f2020196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f201ffff700 1 -- 172.21.15.45:0/1239062797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2020102930 msgr2=0x7f202019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f201ffff700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2020102930 0x7f202019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f201ffff700 1 -- 172.21.15.45:0/1239062797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20140095f0 con 0x7f2020101f70 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f2024886700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100e20 0x7f2020196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f201ffff700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020101f70 0x7f2020197040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f200c00d950 tx=0x7f200c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f201dffb700 1 -- 172.21.15.45:0/1239062797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c009940 con 0x7f2020101f70 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f201dffb700 1 -- 172.21.15.45:0/1239062797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f200c010460 con 0x7f2020101f70 2023-09-23T09:28:43.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f201dffb700 1 -- 172.21.15.45:0/1239062797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c00f620 con 0x7f2020101f70 2023-09-23T09:28:43.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202019ec30 con 0x7f2020101f70 2023-09-23T09:28:43.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.678+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202019f140 con 0x7f2020101f70 2023-09-23T09:28:43.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.679+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202010c670 con 0x7f2020101f70 2023-09-23T09:28:43.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.680+0000 7f201dffb700 1 -- 172.21.15.45:0/1239062797 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f200c0108d0 con 0x7f2020101f70 2023-09-23T09:28:43.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.681+0000 7f201dffb700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f20100a16f0 0x7f20100a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:43.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.681+0000 7f201dffb700 1 -- 172.21.15.45:0/1239062797 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f200c0ccc90 con 0x7f2020101f70 2023-09-23T09:28:43.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.681+0000 7f2024886700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f20100a16f0 0x7f20100a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:43.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.681+0000 7f2024886700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f20100a16f0 0x7f20100a3bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f201400b5c0 tx=0x7f2014005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.685+0000 7f201dffb700 1 -- 172.21.15.45:0/1239062797 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f200c0962d0 con 0x7f2020101f70 2023-09-23T09:28:43.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:43 smithi098 ceph-mon[102386]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:43.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:43 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/769620527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:28:43.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:43 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/775387794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:28:43.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:43 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1159359498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:43.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:43 smithi123 ceph-mon[103577]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:43.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:43 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/769620527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:28:43.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:43 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/775387794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:28:43.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:43 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1159359498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.930+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f202019f2d0 con 0x7f2020101f70 2023-09-23T09:28:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.931+0000 7f201dffb700 1 -- 172.21.15.45:0/1239062797 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7f200c016090 con 0x7f2020101f70 2023-09-23T09:28:43.933 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:43.934 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f20100a16f0 msgr2=0x7f20100a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f20100a16f0 0x7f20100a3bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f201400b5c0 tx=0x7f2014005fb0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020101f70 msgr2=0x7f2020197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020101f70 0x7f2020197040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f200c00d950 tx=0x7f200c00dc60 comp rx=0 tx=0).stop 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 shutdown_connections 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2020102930 0x7f202019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f20100a16f0 0x7f20100a3bb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100e20 0x7f2020196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 --2- 172.21.15.45:0/1239062797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2020101f70 0x7f2020197040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 >> 172.21.15.45:0/1239062797 conn(0x7f20200fc370 msgr2=0x7f20200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 shutdown_connections 2023-09-23T09:28:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:43.933+0000 7f2026aea700 1 -- 172.21.15.45:0/1239062797 wait complete. 2023-09-23T09:28:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:28:43.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-23T09:27:50.073600+0000', 'modified': '2023-09-23T09:27:55.252601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24530, 'mds_1': 24500, 'mds_2': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/3506316120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 3506316120}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 3506316120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/3236511886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3236511886}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3236511886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/2272642085', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2272642085}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2272642085}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/4054887711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 4054887711}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 4054887711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.98:6837/767375327', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 767375327}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 767375327}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/2348516337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 2348516337}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 2348516337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T09:28:43.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-23T09:28:43.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-23T09:28:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 -- 172.21.15.45:0/4270654384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 msgr2=0x7f4d301001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4270654384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d301001a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4d200099c0 tx=0x7f4d20009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 -- 172.21.15.45:0/4270654384 shutdown_connections 2023-09-23T09:28:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4270654384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d30101910 0x7f4d301081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4270654384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d301001a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4270654384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d30100f70 0x7f4d301013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 -- 172.21.15.45:0/4270654384 >> 172.21.15.45:0/4270654384 conn(0x7f4d300fb2f0 msgr2=0x7f4d300fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 -- 172.21.15.45:0/4270654384 shutdown_connections 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.072+0000 7f4d37b04700 1 -- 172.21.15.45:0/4270654384 wait complete. 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 Processor -- start 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 -- start start 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d30069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d30100f70 0x7f4d3019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d30101910 0x7f4d3019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019e720 con 0x7f4d300ffd80 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019e890 con 0x7f4d30101910 2023-09-23T09:28:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d37b04700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019ea50 con 0x7f4d30100f70 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d358a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d30069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d358a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d30069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59612/0 (socket says 172.21.15.45:59612) 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.073+0000 7f4d358a0700 1 -- 172.21.15.45:0/4240808509 learned_addr learned my addr 172.21.15.45:0/4240808509 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d360a1700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d30101910 0x7f4d3019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d3509f700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d30100f70 0x7f4d3019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d358a0700 1 -- 172.21.15.45:0/4240808509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d30100f70 msgr2=0x7f4d3019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d358a0700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d30100f70 0x7f4d3019b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d358a0700 1 -- 172.21.15.45:0/4240808509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d30101910 msgr2=0x7f4d3019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d358a0700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d30101910 0x7f4d3019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d358a0700 1 -- 172.21.15.45:0/4240808509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200095f0 con 0x7f4d300ffd80 2023-09-23T09:28:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d358a0700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d30069ca0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009990 tx=0x7f4d20003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d26ffd700 1 -- 172.21.15.45:0/4240808509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2001d070 con 0x7f4d300ffd80 2023-09-23T09:28:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3019ecd0 con 0x7f4d300ffd80 2023-09-23T09:28:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d26ffd700 1 -- 172.21.15.45:0/4240808509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d20004640 con 0x7f4d300ffd80 2023-09-23T09:28:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d26ffd700 1 -- 172.21.15.45:0/4240808509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000f460 con 0x7f4d300ffd80 2023-09-23T09:28:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.074+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3019f180 con 0x7f4d300ffd80 2023-09-23T09:28:44.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.076+0000 7f4d26ffd700 1 -- 172.21.15.45:0/4240808509 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f4d20004050 con 0x7f4d300ffd80 2023-09-23T09:28:44.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.077+0000 7f4d26ffd700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4d1c0a1630 0x7f4d1c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.077+0000 7f4d26ffd700 1 -- 172.21.15.45:0/4240808509 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4d200ce3e0 con 0x7f4d300ffd80 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.077+0000 7f4d3509f700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4d1c0a1630 0x7f4d1c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.077+0000 7f4d3509f700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4d1c0a1630 0x7f4d1c0a3af0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006fd0 tx=0x7f4d2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.077+0000 7f4d43ceeb80 1 Processor -- start 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d43ceeb80 1 -- start start 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540be550 0x55eb540be970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540beeb0 0x55eb540bf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb54026410 con 0x55eb540bfd50 2023-09-23T09:28:44.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb54017bd0 con 0x55eb540beeb0 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb540154f0 con 0x55eb540be550 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59624/0 (socket says 172.21.15.45:59624) 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 -- 172.21.15.45:0/843151837 learned_addr learned my addr 172.21.15.45:0/843151837 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d3509f700 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540beeb0 0x55eb540bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d358a0700 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540be550 0x55eb540be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 -- 172.21.15.45:0/843151837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540be550 msgr2=0x55eb540be970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540be550 0x55eb540be970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 -- 172.21.15.45:0/843151837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540beeb0 msgr2=0x55eb540bf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540beeb0 0x55eb540bf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 -- 172.21.15.45:0/843151837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb54073880 con 0x55eb540bfd50 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d3509f700 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540beeb0 0x55eb540bf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d358a0700 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540be550 0x55eb540be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.078+0000 7f4d360a1700 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540c01d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009f20 tx=0x7f4d28009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701305fa21922f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d1bfff700 1 -- 172.21.15.45:0/843151837 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28011070 con 0x55eb540bfd50 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d1bfff700 1 -- 172.21.15.45:0/843151837 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d28009a50 con 0x55eb540bfd50 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d1bfff700 1 -- 172.21.15.45:0/843151837 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800a400 con 0x55eb540bfd50 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/843151837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 msgr2=0x55eb540c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540c01d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009f20 tx=0x7f4d28009f50 comp rx=0 tx=0).stop 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/843151837 shutdown_connections 2023-09-23T09:28:44.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540c01d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540be550 0x55eb540be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/843151837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540beeb0 0x55eb540bf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/843151837 >> 172.21.15.45:0/843151837 conn(0x55eb54032210 msgr2=0x55eb540b9430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/843151837 shutdown_connections 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.079+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/843151837 wait complete. 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 -- start start 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 Processor -- start 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540be550 0x55eb540c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb54010de0 con 0x55eb540bfd50 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb53fad6c0 con 0x55eb540be550 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb53fad360 con 0x55eb540beeb0 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d43ceeb80 10 client.0 _ll_register_callbacks cb 0x55eb53e0a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.080+0000 7f4d360a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59630/0 (socket says 172.21.15.45:59630) 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 -- 172.21.15.45:0/309367775 learned_addr learned my addr 172.21.15.45:0/309367775 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d358a0700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540be550 0x55eb540c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d3509f700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 -- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 msgr2=0x55eb540c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 -- 172.21.15.45:0/309367775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540be550 msgr2=0x55eb540c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540be550 0x55eb540c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 -- 172.21.15.45:0/309367775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d28009bd0 con 0x55eb540bfd50 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d358a0700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540be550 0x55eb540c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d360a1700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540cf890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d28018040 tx=0x7f4d28015640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe82d95087423e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d197fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d197fa700 1 -- 172.21.15.45:0/309367775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28011040 con 0x55eb540bfd50 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d197fa700 1 -- 172.21.15.45:0/309367775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4d28006e40 con 0x55eb540bfd50 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb54009dc0 con 0x55eb540bfd50 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d197fa700 1 -- 172.21.15.45:0/309367775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d280158d0 con 0x55eb540bfd50 2023-09-23T09:28:44.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d43ceeb80 10 client.15030 fetch_fsmap learned FSMap version 22 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.081+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eb53e3bd90 con 0x55eb540bfd50 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d197fa700 1 -- 172.21.15.45:0/309367775 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d28031080 con 0x55eb540bfd50 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d197fa700 1 -- 172.21.15.45:0/309367775 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d2801c030 con 0x55eb540bfd50 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d43ceeb80 10 client.15030 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d43ceeb80 10 client.15030 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d43ceeb80 20 client.15030 populate_metadata read hostname 'smithi045' 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb540d3e20 0x55eb540d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d43ceeb80 4 client.15030 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb54009ae0 con 0x55eb540d3e20 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.082+0000 7f4d358a0700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb540d3e20 0x55eb540d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.083+0000 7f4d358a0700 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb540d3e20 0x55eb540d2930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea9a824be15a50c0 server_cookie=8a60f9839fb8c3ff in_seq=0 out_seq=0 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.083+0000 7f4d197fa700 10 client.15030 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.083+0000 7f4d197fa700 1 -- 172.21.15.45:0/309367775 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eb54009ae0 con 0x55eb540d3e20 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.083+0000 7f4d197fa700 10 client.15030 handle_command_reply: tid=0 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.083+0000 7f4d43ceeb80 1 client.15030 shutdown 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.083+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 msgr2=0x55eb540cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.083+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540cf890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d28018040 tx=0x7f4d28015640 comp rx=0 tx=0).stop 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 shutdown_connections 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540bfd50 0x55eb540cf890 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540be550 0x55eb540c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb540d3e20 0x55eb540d2930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/309367775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 >> 172.21.15.45:0/309367775 conn(0x55eb54032210 msgr2=0x55eb540b9240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d197fa700 0 client.15030 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:28:44.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 shutdown_connections 2023-09-23T09:28:44.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/309367775 wait complete. 2023-09-23T09:28:44.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.084+0000 7f4d43ceeb80 20 client.15030 trim_cache size 0 max 16384 2023-09-23T09:28:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.101+0000 7f4d43ceeb80 1 Processor -- start 2023-09-23T09:28:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.101+0000 7f4d43ceeb80 1 -- start start 2023-09-23T09:28:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb53e918d0 con 0x55eb540be550 2023-09-23T09:28:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb5400cf00 con 0x55eb540bfd50 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb53e9dde0 con 0x55eb540beeb0 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59634/0 (socket says 172.21.15.45:59634) 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 -- 172.21.15.45:0/1657865147 learned_addr learned my addr 172.21.15.45:0/1657865147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d3509f700 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 -- 172.21.15.45:0/1657865147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 msgr2=0x55eb540cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 -- 172.21.15.45:0/1657865147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 msgr2=0x55eb540c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 -- 172.21.15.45:0/1657865147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb53e24c10 con 0x55eb540be550 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d358a0700 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c2c30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4d200981e0 tx=0x7f4d20086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f69363b0b5e1d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d24d86700 1 -- 172.21.15.45:0/1657865147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2009f8c0 con 0x55eb540be550 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d24d86700 1 -- 172.21.15.45:0/1657865147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d200a3a70 con 0x55eb540be550 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.102+0000 7f4d24d86700 1 -- 172.21.15.45:0/1657865147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2009aec0 con 0x55eb540be550 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/1657865147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 msgr2=0x55eb540c2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c2c30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4d200981e0 tx=0x7f4d20086040 comp rx=0 tx=0).stop 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/1657865147 shutdown_connections 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c2c30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/1657865147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/1657865147 >> 172.21.15.45:0/1657865147 conn(0x55eb54032210 msgr2=0x55eb540b7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/1657865147 shutdown_connections 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/1657865147 wait complete. 2023-09-23T09:28:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 -- start start 2023-09-23T09:28:44.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.103+0000 7f4d43ceeb80 1 Processor -- start 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d43ceeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb540cf4c0 con 0x55eb540be550 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb540cf630 con 0x55eb540bfd50 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d43ceeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb540cf7f0 con 0x55eb540beeb0 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d43ceeb80 10 client.0 _ll_register_callbacks cb 0x7f4d28009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59640/0 (socket says 172.21.15.45:59640) 2023-09-23T09:28:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 -- 172.21.15.45:0/4244236469 learned_addr learned my addr 172.21.15.45:0/4244236469 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d3509f700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d360a1700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 -- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 msgr2=0x55eb540c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 -- 172.21.15.45:0/4244236469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 msgr2=0x55eb540cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 -- 172.21.15.45:0/4244236469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3019f180 con 0x55eb540be550 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d3509f700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d360a1700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d358a0700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c9a40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4d2009b420 tx=0x7f4d20009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82dbe7d25b7197c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.104+0000 7f4d197fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d197fa700 1 -- 172.21.15.45:0/4244236469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2009f8c0 con 0x55eb540be550 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb540b1200 con 0x55eb540be550 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d197fa700 1 -- 172.21.15.45:0/4244236469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4d200cd040 con 0x55eb540be550 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d197fa700 1 -- 172.21.15.45:0/4244236469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d200a35e0 con 0x55eb540be550 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 10 client.15036 fetch_fsmap learned FSMap version 22 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eb53e62200 con 0x55eb540be550 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d197fa700 1 -- 172.21.15.45:0/4244236469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb540b1200 con 0x55eb540be550 2023-09-23T09:28:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d197fa700 1 -- 172.21.15.45:0/4244236469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d2009caa0 con 0x55eb540be550 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 10 client.15036 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 10 client.15036 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 20 client.15036 populate_metadata read hostname 'smithi045' 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb5412bfd0 0x55eb54128470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 4 client.15036 mds_command: new command op to 24496 tid=0[{"prefix": "status"}] 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.105+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eb53e5cf20 con 0x55eb5412bfd0 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d3509f700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb5412bfd0 0x55eb54128470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d197fa700 10 client.15036 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d3509f700 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb5412bfd0 0x55eb54128470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17b3d86ee1bdb70d server_cookie=5bea84139361ceaa in_seq=0 out_seq=0 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d197fa700 1 -- 172.21.15.45:0/4244236469 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55eb53e5cf20 con 0x55eb5412bfd0 2023-09-23T09:28:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d197fa700 10 client.15036 handle_command_reply: tid=0 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 client.15036 shutdown 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 msgr2=0x55eb540c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c9a40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4d2009b420 tx=0x7f4d20009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 shutdown_connections 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb540bfd50 0x55eb540cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb540be550 0x55eb540c9a40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55eb5412bfd0 0x55eb54128470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 --2- 172.21.15.45:0/4244236469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb540beeb0 0x55eb540c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d197fa700 0 client.15036 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 >> 172.21.15.45:0/4244236469 conn(0x55eb54032210 msgr2=0x55eb540c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 shutdown_connections 2023-09-23T09:28:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 1 -- 172.21.15.45:0/4244236469 wait complete. 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.106+0000 7f4d43ceeb80 20 client.15036 trim_cache size 0 max 16384 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24496, 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-09-23T09:28:44.117 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-09-23T09:28:44.118 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 49.415728927000004, 2023-09-23T09:28:44.118 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 22, 2023-09-23T09:28:44.118 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 83, 2023-09-23T09:28:44.118 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-09-23T09:28:44.118 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 66.912727895000003 2023-09-23T09:28:44.118 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4d1c0a1630 msgr2=0x7f4d1c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4d1c0a1630 0x7f4d1c0a3af0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006fd0 tx=0x7f4d2c008040 comp rx=0 tx=0).stop 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 msgr2=0x7f4d30069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d30069ca0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009990 tx=0x7f4d20003ba0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 shutdown_connections 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d30101910 0x7f4d3019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d300ffd80 0x7f4d30069ca0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f4d1c0a1630 0x7f4d1c0a3af0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 --2- 172.21.15.45:0/4240808509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d30100f70 0x7f4d3019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 >> 172.21.15.45:0/4240808509 conn(0x7f4d300fb2f0 msgr2=0x7f4d301044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 shutdown_connections 2023-09-23T09:28:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.107+0000 7f4d37b04700 1 -- 172.21.15.45:0/4240808509 wait complete. 2023-09-23T09:28:44.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 -- 172.21.15.45:0/1096139530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c0fff90 msgr2=0x7fcc1c100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 --2- 172.21.15.45:0/1096139530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c0fff90 0x7fcc1c100410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009990 tx=0x7fcc10009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 -- 172.21.15.45:0/1096139530 shutdown_connections 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 --2- 172.21.15.45:0/1096139530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc1c100950 0x7fcc1c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 --2- 172.21.15.45:0/1096139530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc1c1013d0 0x7fcc1c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 --2- 172.21.15.45:0/1096139530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c0fff90 0x7fcc1c100410 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 -- 172.21.15.45:0/1096139530 >> 172.21.15.45:0/1096139530 conn(0x7fcc1c0faa40 msgr2=0x7fcc1c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 -- 172.21.15.45:0/1096139530 shutdown_connections 2023-09-23T09:28:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 -- 172.21.15.45:0/1096139530 wait complete. 2023-09-23T09:28:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.246+0000 7fcc21949700 1 Processor -- start 2023-09-23T09:28:44.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 -- start start 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc1c0fff90 0x7fcc1c0ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c100950 0x7fcc1c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc1c1013d0 0x7fcc1c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c19e610 con 0x7fcc1c100950 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c19e780 con 0x7fcc1c0fff90 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c19e940 con 0x7fcc1c1013d0 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c100950 0x7fcc1c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c100950 0x7fcc1c19b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59656/0 (socket says 172.21.15.45:59656) 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/912044350 learned_addr learned my addr 172.21.15.45:0/912044350 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1affd700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc1c0fff90 0x7fcc1c0ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc1c1013d0 0x7fcc1c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/912044350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc1c1013d0 msgr2=0x7fcc1c19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc1c1013d0 0x7fcc1c19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/912044350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc1c0fff90 msgr2=0x7fcc1c0ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc1c0fff90 0x7fcc1c0ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/912044350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc100095f0 con 0x7fcc1c100950 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1affd700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc1c0fff90 0x7fcc1c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c100950 0x7fcc1c19b7f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005910 tx=0x7fcc1000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc20947700 1 -- 172.21.15.45:0/912044350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1001d070 con 0x7fcc1c100950 2023-09-23T09:28:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc20947700 1 -- 172.21.15.45:0/912044350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1000fc30 con 0x7fcc1c100950 2023-09-23T09:28:44.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.247+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c19ebc0 con 0x7fcc1c100950 2023-09-23T09:28:44.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.248+0000 7fcc20947700 1 -- 172.21.15.45:0/912044350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10017650 con 0x7fcc1c100950 2023-09-23T09:28:44.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.248+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c19efe0 con 0x7fcc1c100950 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.250+0000 7fcc20947700 1 -- 172.21.15.45:0/912044350 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fcc10022470 con 0x7fcc1c100950 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.250+0000 7fcc20947700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcc040a1630 0x7fcc040a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.250+0000 7fcc1affd700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcc040a1630 0x7fcc040a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.250+0000 7fcc20947700 1 -- 172.21.15.45:0/912044350 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc100ce460 con 0x7fcc1c100950 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc1affd700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcc040a1630 0x7fcc040a3af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005fd0 tx=0x7fcc0c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 Processor -- start 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 -- start start 2023-09-23T09:28:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0eb9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0e21410 con 0x5635e0eb9eb0 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0e12bd0 con 0x5635e0eb9550 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0e104f0 con 0x5635e0ebad50 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.251+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eba770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59664/0 (socket says 172.21.15.45:59664) 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/2242267298 learned_addr learned my addr 172.21.15.45:0/2242267298 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1affd700 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0eb9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/2242267298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 msgr2=0x5635e0ebb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/2242267298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 msgr2=0x5635e0eb9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0eb9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/2242267298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e0e6e880 con 0x5635e0eb9eb0 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1affd700 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0eb9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eba770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcc100cc010 tx=0x7fcc10009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6183b8ba982da7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc01ffb700 1 -- 172.21.15.45:0/2242267298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1009b7f0 con 0x5635e0eb9eb0 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc01ffb700 1 -- 172.21.15.45:0/2242267298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1009f940 con 0x5635e0eb9eb0 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.252+0000 7fcc01ffb700 1 -- 172.21.15.45:0/2242267298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10086310 con 0x5635e0eb9eb0 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 -- 172.21.15.45:0/2242267298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 msgr2=0x5635e0eba770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eba770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcc100cc010 tx=0x7fcc10009310 comp rx=0 tx=0).stop 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 -- 172.21.15.45:0/2242267298 shutdown_connections 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0eb9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/2242267298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eba770 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 -- 172.21.15.45:0/2242267298 >> 172.21.15.45:0/2242267298 conn(0x5635e0e2d210 msgr2=0x5635e0eb4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 -- 172.21.15.45:0/2242267298 shutdown_connections 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 -- 172.21.15.45:0/2242267298 wait complete. 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 -- start start 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.253+0000 7fcc2db33b80 1 Processor -- start 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9550 0x5635e0ec2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0ebad50 0x5635e0eca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0e0bde0 con 0x5635e0eb9550 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0da86c0 con 0x5635e0ebad50 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0da8360 con 0x5635e0eb9eb0 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 10 client.0 _ll_register_callbacks cb 0x5635e0c057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9550 0x5635e0ec2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9550 0x5635e0ec2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59668/0 (socket says 172.21.15.45:59668) 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 -- 172.21.15.45:0/1806072501 learned_addr learned my addr 172.21.15.45:0/1806072501 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 -- 172.21.15.45:0/1806072501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 msgr2=0x5635e0ec3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 -- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0ebad50 msgr2=0x5635e0eca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0ebad50 0x5635e0eca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0ebad50 0x5635e0eca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 -- 172.21.15.45:0/1806072501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c19efe0 con 0x5635e0eb9550 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc1affd700 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9550 0x5635e0ec2fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0109e0 tx=0x7fcc0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5bbbd243a37eb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcbfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcbfaffd700 1 -- 172.21.15.45:0/1806072501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00fa50 con 0x5635e0eb9550 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcbfaffd700 1 -- 172.21.15.45:0/1806072501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc0c00fbb0 con 0x5635e0eb9550 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.254+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e0eac200 con 0x5635e0eb9550 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcbfaffd700 1 -- 172.21.15.45:0/1806072501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc0c00fd70 con 0x5635e0eb9550 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 10 client.15048 fetch_fsmap learned FSMap version 22 2023-09-23T09:28:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5635e0c36d90 con 0x5635e0eb9550 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcbfaffd700 1 -- 172.21.15.45:0/1806072501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635e0eac200 con 0x5635e0eb9550 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcbfaffd700 1 -- 172.21.15.45:0/1806072501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc0c024020 con 0x5635e0eb9550 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 10 client.15048 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 20 client.15048 populate_metadata read hostname 'smithi045' 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0ecec90 0x5635e0ecd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 4 client.15048 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e0e04dc0 con 0x5635e0ecec90 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.255+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0ecec90 0x5635e0ecd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.256+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0ecec90 0x5635e0ecd6c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=606b95276c6dbd6b server_cookie=64e3b4b82f4e14eb in_seq=0 out_seq=0 2023-09-23T09:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.256+0000 7fcbfaffd700 10 client.15048 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.256+0000 7fcbfaffd700 1 -- 172.21.15.45:0/1806072501 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635e0e04dc0 con 0x5635e0ecec90 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.256+0000 7fcbfaffd700 10 client.15048 handle_command_reply: tid=0 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.256+0000 7fcc2db33b80 1 client.15048 shutdown 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9550 msgr2=0x5635e0ec2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9550 0x5635e0ec2fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0109e0 tx=0x7fcc0c004750 comp rx=0 tx=0).stop 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 shutdown_connections 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0ecec90 0x5635e0ecd6c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0ebad50 0x5635e0eca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcbfaffd700 0 client.15048 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9550 0x5635e0ec2fe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1806072501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 >> 172.21.15.45:0/1806072501 conn(0x5635e0e2d210 msgr2=0x5635e0eb4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 shutdown_connections 2023-09-23T09:28:44.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1806072501 wait complete. 2023-09-23T09:28:44.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.257+0000 7fcc2db33b80 20 client.15048 trim_cache size 0 max 16384 2023-09-23T09:28:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 Processor -- start 2023-09-23T09:28:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 -- start start 2023-09-23T09:28:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0ec29f0 con 0x5635e0eb9eb0 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0c8c8d0 con 0x5635e0eb9550 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0c98de0 con 0x5635e0ebad50 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59678/0 (socket says 172.21.15.45:59678) 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/1485231130 learned_addr learned my addr 172.21.15.45:0/1485231130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1affd700 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/1485231130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 msgr2=0x5635e0ebdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/1485231130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 msgr2=0x5635e0ec1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/1485231130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e0c1fc10 con 0x5635e0eb9eb0 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.275+0000 7fcc1affd700 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eca520 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c010 tx=0x7fcc10086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d390c4de3c9a2d52 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc027fc700 1 -- 172.21.15.45:0/1485231130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100a3b70 con 0x5635e0eb9eb0 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc027fc700 1 -- 172.21.15.45:0/1485231130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1009b7f0 con 0x5635e0eb9eb0 2023-09-23T09:28:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc027fc700 1 -- 172.21.15.45:0/1485231130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100b9310 con 0x5635e0eb9eb0 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1485231130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 msgr2=0x5635e0eca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eca520 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c010 tx=0x7fcc10086040 comp rx=0 tx=0).stop 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1485231130 shutdown_connections 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0ebad50 0x5635e0ebdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/1485231130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0eb9eb0 0x5635e0eca520 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1485231130 >> 172.21.15.45:0/1485231130 conn(0x5635e0e2d210 msgr2=0x5635e0eb2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1485231130 shutdown_connections 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.276+0000 7fcc2db33b80 1 -- 172.21.15.45:0/1485231130 wait complete. 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 -- start start 2023-09-23T09:28:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 Processor -- start 2023-09-23T09:28:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0ebad50 0x5635e0ec9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0eca190 con 0x5635e0ebad50 2023-09-23T09:28:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0eca300 con 0x5635e0eb9550 2023-09-23T09:28:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e0eca4c0 con 0x5635e0eb9eb0 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc2db33b80 10 client.0 _ll_register_callbacks cb 0x7fcc0c00ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc1b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0ebad50 0x5635e0ec9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc1b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0ebad50 0x5635e0ec9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59686/0 (socket says 172.21.15.45:59686) 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc1b7fe700 1 -- 172.21.15.45:0/4260846679 learned_addr learned my addr 172.21.15.45:0/4260846679 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc1affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.277+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc1b7fe700 1 -- 172.21.15.45:0/4260846679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 msgr2=0x5635e0ec4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc1b7fe700 1 -- 172.21.15.45:0/4260846679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 msgr2=0x5635e0ec4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc1b7fe700 1 -- 172.21.15.45:0/4260846679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e0e6e880 con 0x5635e0ebad50 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc1b7fe700 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0ebad50 0x5635e0ec9c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc140077b0 tx=0x7fcc14007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4791a424374b719 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcbfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcbfaffd700 1 -- 172.21.15.45:0/4260846679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14018870 con 0x5635e0ebad50 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcbfaffd700 1 -- 172.21.15.45:0/4260846679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc140189d0 con 0x5635e0ebad50 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc1009f940 con 0x5635e0ebad50 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcbfaffd700 1 -- 172.21.15.45:0/4260846679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc14018b90 con 0x5635e0ebad50 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc2db33b80 10 client.15054 fetch_fsmap learned FSMap version 22 2023-09-23T09:28:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5635e0c5d200 con 0x5635e0ebad50 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcbfaffd700 1 -- 172.21.15.45:0/4260846679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1009f940 con 0x5635e0ebad50 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcbfaffd700 1 -- 172.21.15.45:0/4260846679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc140173e0 con 0x5635e0ebad50 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc2db33b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc2db33b80 10 client.15054 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc2db33b80 20 client.15054 populate_metadata read hostname 'smithi045' 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.278+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0f273c0 0x5635e0f23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc2db33b80 4 client.15054 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5635e0e04ae0 con 0x5635e0f273c0 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc1affd700 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0f273c0 0x5635e0f23830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcbfaffd700 10 client.15054 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc1affd700 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0f273c0 0x5635e0f23830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a60472ba440d5e6 server_cookie=affe8d6721f3567d in_seq=0 out_seq=0 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcbfaffd700 1 -- 172.21.15.45:0/4260846679 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5635e0e04ae0 con 0x5635e0f273c0 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcbfaffd700 10 client.15054 handle_command_reply: tid=0 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc2db33b80 1 client.15054 shutdown 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0ebad50 msgr2=0x5635e0ec9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0ebad50 0x5635e0ec9c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc140077b0 tx=0x7fcc14007ac0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 shutdown_connections 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.279+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x5635e0f273c0 0x5635e0f23830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e0ebad50 0x5635e0ec9c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635e0eb9550 0x5635e0ec4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcbfaffd700 0 client.15054 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc2db33b80 1 --2- 172.21.15.45:0/4260846679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635e0eb9eb0 0x5635e0ec4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 >> 172.21.15.45:0/4260846679 conn(0x5635e0e2d210 msgr2=0x5635e0ec1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 shutdown_connections 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc2db33b80 1 -- 172.21.15.45:0/4260846679 wait complete. 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc2db33b80 20 client.15054 trim_cache size 0 max 16384 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:28:44.291 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24500, 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 50.600841318000001, 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 22, 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 83, 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 69.253771564999994 2023-09-23T09:28:44.292 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:28:44.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcc040a1630 msgr2=0x7fcc040a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcc040a1630 0x7fcc040a3af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005fd0 tx=0x7fcc0c005e30 comp rx=0 tx=0).stop 2023-09-23T09:28:44.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c100950 msgr2=0x7fcc1c19b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c100950 0x7fcc1c19b7f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005910 tx=0x7fcc1000f740 comp rx=0 tx=0).stop 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 shutdown_connections 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcc040a1630 0x7fcc040a3af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc1c1013d0 0x7fcc1c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc1c0fff90 0x7fcc1c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 --2- 172.21.15.45:0/912044350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1c100950 0x7fcc1c19b7f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 >> 172.21.15.45:0/912044350 conn(0x7fcc1c0faa40 msgr2=0x7fcc1c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 shutdown_connections 2023-09-23T09:28:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.280+0000 7fcc21949700 1 -- 172.21.15.45:0/912044350 wait complete. 2023-09-23T09:28:44.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-23T09:28:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 -- 172.21.15.45:0/151559491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e4100f00 msgr2=0x7f55e4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 --2- 172.21.15.45:0/151559491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e4100f00 0x7f55e4101360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55e00099c0 tx=0x7f55e0009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 -- 172.21.15.45:0/151559491 shutdown_connections 2023-09-23T09:28:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 --2- 172.21.15.45:0/151559491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e41018a0 0x7f55e4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 --2- 172.21.15.45:0/151559491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e40ffd10 0x7f55e4100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 --2- 172.21.15.45:0/151559491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e4100f00 0x7f55e4101360 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 -- 172.21.15.45:0/151559491 >> 172.21.15.45:0/151559491 conn(0x7f55e40fb280 msgr2=0x7f55e40fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 -- 172.21.15.45:0/151559491 shutdown_connections 2023-09-23T09:28:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.419+0000 7f55eb84f700 1 -- 172.21.15.45:0/151559491 wait complete. 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 Processor -- start 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 -- start start 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e40ffd10 0x7f55e41958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e4100f00 0x7f55e4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e41018a0 0x7f55e419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419d460 con 0x7f55e40ffd10 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419d5a0 con 0x7f55e41018a0 2023-09-23T09:28:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55eb84f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419d730 con 0x7f55e4100f00 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e40ffd10 0x7f55e41958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e40ffd10 0x7f55e41958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59696/0 (socket says 172.21.15.45:59696) 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 -- 172.21.15.45:0/1015152069 learned_addr learned my addr 172.21.15.45:0/1015152069 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e9dec700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e41018a0 0x7f55e419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 -- 172.21.15.45:0/1015152069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e4100f00 msgr2=0x7f55e4195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e4100f00 0x7f55e4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 -- 172.21.15.45:0/1015152069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e41018a0 msgr2=0x7f55e419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e41018a0 0x7f55e419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.420+0000 7f55e95eb700 1 -- 172.21.15.45:0/1015152069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00095f0 con 0x7f55e40ffd10 2023-09-23T09:28:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.421+0000 7f55e95eb700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e40ffd10 0x7f55e41958e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c990 tx=0x7f55d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.421+0000 7f55da7fc700 1 -- 172.21.15.45:0/1015152069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007b00 con 0x7f55e40ffd10 2023-09-23T09:28:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.421+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e419da10 con 0x7f55e40ffd10 2023-09-23T09:28:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.421+0000 7f55da7fc700 1 -- 172.21.15.45:0/1015152069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d4007c60 con 0x7f55e40ffd10 2023-09-23T09:28:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.421+0000 7f55da7fc700 1 -- 172.21.15.45:0/1015152069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4018660 con 0x7f55e40ffd10 2023-09-23T09:28:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.421+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e419df20 con 0x7f55e40ffd10 2023-09-23T09:28:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.423+0000 7f55da7fc700 1 -- 172.21.15.45:0/1015152069 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f55d400f450 con 0x7f55e40ffd10 2023-09-23T09:28:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.423+0000 7f55da7fc700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f55d00a1630 0x7f55d00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.424+0000 7f55e8dea700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f55d00a1630 0x7f55d00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.424+0000 7f55da7fc700 1 -- 172.21.15.45:0/1015152069 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55d40ccec0 con 0x7f55e40ffd10 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.424+0000 7f55e8dea700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f55d00a1630 0x7f55d00a3af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f55e00036f0 tx=0x7f55e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 Processor -- start 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 -- start start 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f24970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f24eb0 0x5557b9f25770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f25d50 0x5557b9f261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9e8c410 con 0x5557b9f25d50 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9e7dbd0 con 0x5557b9f24eb0 2023-09-23T09:28:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9e7b4f0 con 0x5557b9f24550 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f25d50 0x5557b9f261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f25d50 0x5557b9f261d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59708/0 (socket says 172.21.15.45:59708) 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 -- 172.21.15.45:0/4168115398 learned_addr learned my addr 172.21.15.45:0/4168115398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e8dea700 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f24eb0 0x5557b9f25770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e95eb700 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f24970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 -- 172.21.15.45:0/4168115398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 msgr2=0x5557b9f24970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f24970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 -- 172.21.15.45:0/4168115398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f24eb0 msgr2=0x5557b9f25770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f24eb0 0x5557b9f25770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 -- 172.21.15.45:0/4168115398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557b9ed9880 con 0x5557b9f25d50 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.425+0000 7f55e9dec700 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f25d50 0x5557b9f261d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00e190 tx=0x7f55dc00e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13f4e26014cbb93 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55cf7fe700 1 -- 172.21.15.45:0/4168115398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc018070 con 0x5557b9f25d50 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55cf7fe700 1 -- 172.21.15.45:0/4168115398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55dc007c30 con 0x5557b9f25d50 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55cf7fe700 1 -- 172.21.15.45:0/4168115398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc014de0 con 0x5557b9f25d50 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4168115398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f25d50 msgr2=0x5557b9f261d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f25d50 0x5557b9f261d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00e190 tx=0x7f55dc00e4a0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4168115398 shutdown_connections 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f25d50 0x5557b9f261d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f24970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4168115398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f24eb0 0x5557b9f25770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4168115398 >> 172.21.15.45:0/4168115398 conn(0x5557b9e98210 msgr2=0x5557b9f1f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4168115398 shutdown_connections 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.426+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4168115398 wait complete. 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 -- start start 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 Processor -- start 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24550 0x5557b9f2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24eb0 0x5557b9f2e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f35860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9e76de0 con 0x5557b9f24550 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9e136c0 con 0x5557b9f25d50 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9e13360 con 0x5557b9f24eb0 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55f7a39b80 10 client.0 _ll_register_callbacks cb 0x5557b9c707b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:28:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55e95eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24550 0x5557b9f2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55e95eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24550 0x5557b9f2e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59710/0 (socket says 172.21.15.45:59710) 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55e95eb700 1 -- 172.21.15.45:0/4262869261 learned_addr learned my addr 172.21.15.45:0/4262869261 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.427+0000 7f55e8dea700 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24eb0 0x5557b9f2e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55e9dec700 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f35860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55e95eb700 1 -- 172.21.15.45:0/4262869261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24eb0 msgr2=0x5557b9f2e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55e95eb700 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24eb0 0x5557b9f2e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55e95eb700 1 -- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 msgr2=0x5557b9f35860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55e95eb700 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f35860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55e95eb700 1 -- 172.21.15.45:0/4262869261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc00f040 con 0x5557b9f24550 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55e95eb700 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24550 0x5557b9f2e2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55d4096c30 tx=0x7f55d4099c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1d84860f5bf295 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55ccff9700 1 -- 172.21.15.45:0/4262869261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007dd0 con 0x5557b9f24550 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55ccff9700 1 -- 172.21.15.45:0/4262869261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55d409a1c0 con 0x5557b9f24550 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557b9e6fdc0 con 0x5557b9f24550 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55ccff9700 1 -- 172.21.15.45:0/4262869261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d409e310 con 0x5557b9f24550 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55f7a39b80 10 client.15066 fetch_fsmap learned FSMap version 22 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.428+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5557b9ca1d90 con 0x5557b9f24550 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55ccff9700 1 -- 172.21.15.45:0/4262869261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557b9e6fdc0 con 0x5557b9f24550 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55ccff9700 1 -- 172.21.15.45:0/4262869261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55d40996b0 con 0x5557b9f24550 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55f7a39b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55f7a39b80 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T09:28:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55f7a39b80 20 client.15066 populate_metadata read hostname 'smithi045' 2023-09-23T09:28:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f39d70 0x5557b9f38850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55f7a39b80 4 client.15066 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:28:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557b9e6fae0 con 0x5557b9f39d70 2023-09-23T09:28:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.429+0000 7f55e8dea700 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f39d70 0x5557b9f38850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55e8dea700 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f39d70 0x5557b9f38850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704d2b2e73690261 server_cookie=17e4839d5767fe30 in_seq=0 out_seq=0 2023-09-23T09:28:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55ccff9700 10 client.15066 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:28:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55ccff9700 1 -- 172.21.15.45:0/4262869261 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5557b9e6fae0 con 0x5557b9f39d70 2023-09-23T09:28:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55ccff9700 10 client.15066 handle_command_reply: tid=0 2023-09-23T09:28:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 client.15066 shutdown 2023-09-23T09:28:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24550 msgr2=0x5557b9f2e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24550 0x5557b9f2e2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55d4096c30 tx=0x7f55d4099c10 comp rx=0 tx=0).stop 2023-09-23T09:28:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 shutdown_connections 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f39d70 0x5557b9f38850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f35860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55ccff9700 0 client.15066 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24550 0x5557b9f2e2e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.430+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/4262869261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24eb0 0x5557b9f2e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.431+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 >> 172.21.15.45:0/4262869261 conn(0x5557b9e98210 msgr2=0x5557b9f1f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.431+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 shutdown_connections 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.431+0000 7f55f7a39b80 1 -- 172.21.15.45:0/4262869261 wait complete. 2023-09-23T09:28:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.431+0000 7f55f7a39b80 20 client.15066 trim_cache size 0 max 16384 2023-09-23T09:28:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 Processor -- start 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 -- start start 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f28c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9f2dcf0 con 0x5557b9f24eb0 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9cf78d0 con 0x5557b9f25d50 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9e6fae0 con 0x5557b9f24550 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55e8dea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55e8dea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f291d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59720/0 (socket says 172.21.15.45:59720) 2023-09-23T09:28:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55e8dea700 1 -- 172.21.15.45:0/3029641804 learned_addr learned my addr 172.21.15.45:0/3029641804 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.451+0000 7f55e9dec700 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e95eb700 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f28c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e8dea700 1 -- 172.21.15.45:0/3029641804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 msgr2=0x5557b9f28c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e8dea700 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f28c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e8dea700 1 -- 172.21.15.45:0/3029641804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 msgr2=0x5557b9f357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e8dea700 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e8dea700 1 -- 172.21.15.45:0/3029641804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557b9c8ac10 con 0x5557b9f24eb0 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e95eb700 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f28c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e9dec700 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55e8dea700 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f291d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005b40 tx=0x7f55e0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48275f74813b85c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55cffff700 1 -- 172.21.15.45:0/3029641804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e001d070 con 0x5557b9f24eb0 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55cffff700 1 -- 172.21.15.45:0/3029641804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e002a040 con 0x5557b9f24eb0 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.452+0000 7f55cffff700 1 -- 172.21.15.45:0/3029641804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0004880 con 0x5557b9f24eb0 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 -- 172.21.15.45:0/3029641804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 msgr2=0x5557b9f291d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f291d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005b40 tx=0x7f55e0020000 comp rx=0 tx=0).stop 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 -- 172.21.15.45:0/3029641804 shutdown_connections 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f28c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/3029641804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f291d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 -- 172.21.15.45:0/3029641804 >> 172.21.15.45:0/3029641804 conn(0x5557b9e98210 msgr2=0x5557b9f1deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 -- 172.21.15.45:0/3029641804 shutdown_connections 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 -- 172.21.15.45:0/3029641804 wait complete. 2023-09-23T09:28:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 -- start start 2023-09-23T09:28:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 Processor -- start 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f35640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.453+0000 7f55f7a39b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f30610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9f30b50 con 0x5557b9f24eb0 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9f30cc0 con 0x5557b9f25d50 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55f7a39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b9f8d760 con 0x5557b9f24550 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55f7a39b80 10 client.0 _ll_register_callbacks cb 0x7f55d409a060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f35640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f35640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59726/0 (socket says 172.21.15.45:59726) 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 -- 172.21.15.45:0/2009510116 learned_addr learned my addr 172.21.15.45:0/2009510116 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e95eb700 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e9dec700 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f30610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 -- 172.21.15.45:0/2009510116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 msgr2=0x5557b9f300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f300d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 -- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 msgr2=0x5557b9f30610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f30610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 -- 172.21.15.45:0/2009510116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e4108aa0 con 0x5557b9f24eb0 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55e8dea700 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f35640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005850 tx=0x7f55e0032b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a0f79b1d3fa48f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:28:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55ccff9700 1 -- 172.21.15.45:0/2009510116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e001d070 con 0x5557b9f24eb0 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55ccff9700 1 -- 172.21.15.45:0/2009510116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55e0027070 con 0x5557b9f24eb0 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557b9f17200 con 0x5557b9f24eb0 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55ccff9700 1 -- 172.21.15.45:0/2009510116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e000bb70 con 0x5557b9f24eb0 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55f7a39b80 10 client.15072 fetch_fsmap learned FSMap version 22 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.454+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5557b9cc8200 con 0x5557b9f24eb0 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55ccff9700 1 -- 172.21.15.45:0/2009510116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557b9f17200 con 0x5557b9f24eb0 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55ccff9700 1 -- 172.21.15.45:0/2009510116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55e0003b20 con 0x5557b9f24eb0 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55f7a39b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55f7a39b80 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55f7a39b80 20 client.15072 populate_metadata read hostname 'smithi045' 2023-09-23T09:28:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f92490 0x5557b9f8e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55f7a39b80 4 client.15072 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5557b9cc2f20 con 0x5557b9f92490 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55e95eb700 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f92490 0x5557b9f8e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55ccff9700 10 client.15072 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.455+0000 7f55e95eb700 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f92490 0x5557b9f8e970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e6e498fdbbb7a3 server_cookie=f1216494ca231b8c in_seq=0 out_seq=0 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55ccff9700 1 -- 172.21.15.45:0/2009510116 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5557b9cc2f20 con 0x5557b9f92490 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55ccff9700 10 client.15072 handle_command_reply: tid=0 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 client.15072 shutdown 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 msgr2=0x5557b9f35640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f35640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005850 tx=0x7f55e0032b10 comp rx=0 tx=0).stop 2023-09-23T09:28:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 shutdown_connections 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5557b9f92490 0x5557b9f8e970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557b9f25d50 0x5557b9f30610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557b9f24550 0x5557b9f300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 --2- 172.21.15.45:0/2009510116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b9f24eb0 0x5557b9f35640 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55ccff9700 0 client.15072 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 >> 172.21.15.45:0/2009510116 conn(0x5557b9e98210 msgr2=0x5557b9f28c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 shutdown_connections 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 1 -- 172.21.15.45:0/2009510116 wait complete. 2023-09-23T09:28:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.456+0000 7f55f7a39b80 20 client.15072 trim_cache size 0 max 16384 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24530, 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 53.802126993999998, 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 22, 2023-09-23T09:28:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 83, 2023-09-23T09:28:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-09-23T09:28:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 60.527278782000003 2023-09-23T09:28:44.468 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:28:44.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f55d00a1630 msgr2=0x7f55d00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f55d00a1630 0x7f55d00a3af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f55e00036f0 tx=0x7f55e0000bc0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e40ffd10 msgr2=0x7f55e41958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e40ffd10 0x7f55e41958e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c990 tx=0x7f55d400cd50 comp rx=0 tx=0).stop 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 shutdown_connections 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e41018a0 0x7f55e419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55e40ffd10 0x7f55e41958e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f55d00a1630 0x7f55d00a3af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 --2- 172.21.15.45:0/1015152069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e4100f00 0x7f55e4195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 >> 172.21.15.45:0/1015152069 conn(0x7f55e40fb280 msgr2=0x7f55e40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 shutdown_connections 2023-09-23T09:28:44.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:44.457+0000 7f55eb84f700 1 -- 172.21.15.45:0/1015152069 wait complete. 2023-09-23T09:28:44.471 INFO:tasks.cephadm:Setup complete, yielding 2023-09-23T09:28:44.472 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-23T09:28:44.489 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:28:44.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch status' 2023-09-23T09:28:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:44 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2525302267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:44.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:44 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1239062797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:44.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:44 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2525302267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:44.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:44 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1239062797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:44.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:44 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2525302267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:28:44.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:44 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1239062797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:28:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:45.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:45 smithi045 ceph-mon[98627]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:45.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:45 smithi098 ceph-mon[102386]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:45.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:45 smithi123 ceph-mon[103577]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.162+0000 7f05598ad700 1 -- 172.21.15.45:0/1167356140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0554100fb0 msgr2=0x7f0554101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.162+0000 7f05598ad700 1 --2- 172.21.15.45:0/1167356140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0554100fb0 0x7f0554101410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f05440099c0 tx=0x7f0544009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.162+0000 7f05598ad700 1 -- 172.21.15.45:0/1167356140 shutdown_connections 2023-09-23T09:28:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.162+0000 7f05598ad700 1 --2- 172.21.15.45:0/1167356140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0554101950 0x7f0554108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.162+0000 7f05598ad700 1 --2- 172.21.15.45:0/1167356140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05540ffdb0 0x7f05541001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.162+0000 7f05598ad700 1 --2- 172.21.15.45:0/1167356140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0554100fb0 0x7f0554101410 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.162+0000 7f05598ad700 1 -- 172.21.15.45:0/1167356140 >> 172.21.15.45:0/1167356140 conn(0x7f05540fb310 msgr2=0x7f05540fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:46.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.163+0000 7f05598ad700 1 -- 172.21.15.45:0/1167356140 shutdown_connections 2023-09-23T09:28:46.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.163+0000 7f05598ad700 1 -- 172.21.15.45:0/1167356140 wait complete. 2023-09-23T09:28:46.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.163+0000 7f05598ad700 1 Processor -- start 2023-09-23T09:28:46.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.163+0000 7f05598ad700 1 -- start start 2023-09-23T09:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05598ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05540ffdb0 0x7f0554103d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05598ad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0554100fb0 0x7f0554102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05598ad700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0554101950 0x7f05541028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05598ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0554103050 con 0x7f05540ffdb0 2023-09-23T09:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05598ad700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05541031c0 con 0x7f0554101950 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05598ad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05541a0b80 con 0x7f0554100fb0 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05540ffdb0 0x7f0554103d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05540ffdb0 0x7f0554103d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59736/0 (socket says 172.21.15.45:59736) 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 -- 172.21.15.45:0/959957073 learned_addr learned my addr 172.21.15.45:0/959957073 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05527fc700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0554100fb0 0x7f0554102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05537fe700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0554101950 0x7f05541028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 -- 172.21.15.45:0/959957073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0554100fb0 msgr2=0x7f0554102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0554100fb0 0x7f0554102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 -- 172.21.15.45:0/959957073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0554101950 msgr2=0x7f05541028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0554101950 0x7f05541028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 -- 172.21.15.45:0/959957073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440095f0 con 0x7f05540ffdb0 2023-09-23T09:28:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f0552ffd700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05540ffdb0 0x7f0554103d40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f054800d980 tx=0x7f054800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05588ab700 1 -- 172.21.15.45:0/959957073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548004020 con 0x7f05540ffdb0 2023-09-23T09:28:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05541a0e60 con 0x7f05540ffdb0 2023-09-23T09:28:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05588ab700 1 -- 172.21.15.45:0/959957073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054800f690 con 0x7f05540ffdb0 2023-09-23T09:28:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.164+0000 7f05588ab700 1 -- 172.21.15.45:0/959957073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548003a60 con 0x7f05540ffdb0 2023-09-23T09:28:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.165+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05541a1270 con 0x7f05540ffdb0 2023-09-23T09:28:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.166+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055410b5b0 con 0x7f05540ffdb0 2023-09-23T09:28:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.167+0000 7f05588ab700 1 -- 172.21.15.45:0/959957073 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f054800fb00 con 0x7f05540ffdb0 2023-09-23T09:28:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.167+0000 7f05588ab700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f053c0a1630 0x7f053c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.167+0000 7f05527fc700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f053c0a1630 0x7f053c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.168+0000 7f05588ab700 1 -- 172.21.15.45:0/959957073 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05480cca90 con 0x7f05540ffdb0 2023-09-23T09:28:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.168+0000 7f05527fc700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f053c0a1630 0x7f053c0a3af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f054400b580 tx=0x7f0544005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:46.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.172+0000 7f05588ab700 1 -- 172.21.15.45:0/959957073 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f05480960d0 con 0x7f05540ffdb0 2023-09-23T09:28:46.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.394+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0554069a10 con 0x7f053c0a1630 2023-09-23T09:28:46.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.395+0000 7f05588ab700 1 -- 172.21.15.45:0/959957073 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0554069a10 con 0x7f053c0a1630 2023-09-23T09:28:46.397 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2023-09-23T09:28:46.398 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2023-09-23T09:28:46.398 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f053c0a1630 msgr2=0x7f053c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f053c0a1630 0x7f053c0a3af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f054400b580 tx=0x7f0544005bc0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05540ffdb0 msgr2=0x7f0554103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05540ffdb0 0x7f0554103d40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f054800d980 tx=0x7f054800dc90 comp rx=0 tx=0).stop 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 shutdown_connections 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0554101950 0x7f05541028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05540ffdb0 0x7f0554103d40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f053c0a1630 0x7f053c0a3af0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 --2- 172.21.15.45:0/959957073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0554100fb0 0x7f0554102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 >> 172.21.15.45:0/959957073 conn(0x7f05540fb310 msgr2=0x7f0554069300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 shutdown_connections 2023-09-23T09:28:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:46.397+0000 7f05598ad700 1 -- 172.21.15.45:0/959957073 wait complete. 2023-09-23T09:28:46.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-09-23T09:28:47.365 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:47.721 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:47 smithi045 ceph-mon[98627]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:47.721 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:47 smithi045 ceph-mon[98627]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:47.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:47 smithi098 ceph-mon[102386]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:47.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:47 smithi098 ceph-mon[102386]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:47.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:47 smithi123 ceph-mon[103577]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:47.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:47 smithi123 ceph-mon[103577]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 -- 172.21.15.45:0/334130541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8880ffdb0 msgr2=0x7fd8881001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 --2- 172.21.15.45:0/334130541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8880ffdb0 0x7fd8881001d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8780099c0 tx=0x7fd878009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 -- 172.21.15.45:0/334130541 shutdown_connections 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 --2- 172.21.15.45:0/334130541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd888101950 0x7fd888108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 --2- 172.21.15.45:0/334130541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8880ffdb0 0x7fd8881001d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 --2- 172.21.15.45:0/334130541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888100fb0 0x7fd888101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 -- 172.21.15.45:0/334130541 >> 172.21.15.45:0/334130541 conn(0x7fd8880fb330 msgr2=0x7fd8880fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 -- 172.21.15.45:0/334130541 shutdown_connections 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.490+0000 7fd88df13700 1 -- 172.21.15.45:0/334130541 wait complete. 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.491+0000 7fd88df13700 1 Processor -- start 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.491+0000 7fd88df13700 1 -- start start 2023-09-23T09:28:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd88df13700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8880ffdb0 0x7fd888195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd88df13700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd888100fb0 0x7fd888196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd88df13700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101950 0x7fd88819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd88df13700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88819d680 con 0x7fd888100fb0 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd88df13700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88819d7c0 con 0x7fd8880ffdb0 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd88df13700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88819d950 con 0x7fd888101950 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd888100fb0 0x7fd888196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd888100fb0 0x7fd888196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59750/0 (socket says 172.21.15.45:59750) 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 -- 172.21.15.45:0/1141285643 learned_addr learned my addr 172.21.15.45:0/1141285643 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd8877fe700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8880ffdb0 0x7fd888195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd887fff700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101950 0x7fd88819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 -- 172.21.15.45:0/1141285643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101950 msgr2=0x7fd88819d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101950 0x7fd88819d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 -- 172.21.15.45:0/1141285643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8880ffdb0 msgr2=0x7fd888195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8880ffdb0 0x7fd888195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 -- 172.21.15.45:0/1141285643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780095f0 con 0x7fd888100fb0 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd886ffd700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd888100fb0 0x7fd888196030 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c950 tx=0x7fd87c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd884ff9700 1 -- 172.21.15.45:0/1141285643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c007ac0 con 0x7fd888100fb0 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd884ff9700 1 -- 172.21.15.45:0/1141285643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd87c004cf0 con 0x7fd888100fb0 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.492+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88819dc30 con 0x7fd888100fb0 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.493+0000 7fd884ff9700 1 -- 172.21.15.45:0/1141285643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c005580 con 0x7fd888100fb0 2023-09-23T09:28:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.493+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88819e130 con 0x7fd888100fb0 2023-09-23T09:28:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.494+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8881073a0 con 0x7fd888100fb0 2023-09-23T09:28:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.495+0000 7fd884ff9700 1 -- 172.21.15.45:0/1141285643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fd87c007c20 con 0x7fd888100fb0 2023-09-23T09:28:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.495+0000 7fd884ff9700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd8700a1630 0x7fd8700a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.495+0000 7fd8877fe700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd8700a1630 0x7fd8700a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.496+0000 7fd884ff9700 1 -- 172.21.15.45:0/1141285643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd87c0ccb70 con 0x7fd888100fb0 2023-09-23T09:28:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.496+0000 7fd8877fe700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd8700a1630 0x7fd8700a3af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd87800b580 tx=0x7fd878005320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.501+0000 7fd884ff9700 1 -- 172.21.15.45:0/1141285643 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd87c0961b0 con 0x7fd888100fb0 2023-09-23T09:28:48.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.709+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd88802d270 con 0x7fd8700a1630 2023-09-23T09:28:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.729+0000 7fd884ff9700 1 -- 172.21.15.45:0/1141285643 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fd88802d270 con 0x7fd8700a1630 2023-09-23T09:28:48.731 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (67s) - 67s 14.3M - 18.0.0-6336-g86266391 9b6b9c6d0d34 886d984b5a66 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi098 running (65s) - 64s 16.5M - 18.0.0-6336-g86266391 9b6b9c6d0d34 a1718a790ae6 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi123 running (76s) - 75s 13.3M - 18.0.0-6336-g86266391 9b6b9c6d0d34 60bcf51042bb 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (82s) - 82s 15.7M - 18.0.0-6336-g86266391 9b6b9c6d0d34 892fae0cafc4 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi098 running (80s) - 80s 15.6M - 18.0.0-6336-g86266391 9b6b9c6d0d34 0bd5d52b7302 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi123 running (78s) - 78s 16.8M - 18.0.0-6336-g86266391 9b6b9c6d0d34 184e6a3907de 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (69s) - 69s 14.4M - 18.0.0-6336-g86266391 9b6b9c6d0d34 1ca237c49e5e 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi098 running (74s) - 73s 14.4M - 18.0.0-6336-g86266391 9b6b9c6d0d34 86817e815924 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi123 running (71s) - 71s 14.4M - 18.0.0-6336-g86266391 9b6b9c6d0d34 efc8dbc0421c 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (84s) - 84s 14.4M - 18.0.0-6336-g86266391 9b6b9c6d0d34 f67b3af3e858 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi098 running (86s) - 86s 15.7M - 18.0.0-6336-g86266391 9b6b9c6d0d34 0e5bf4c0319d 2023-09-23T09:28:48.732 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi123 running (89s) - 88s 14.3M - 18.0.0-6336-g86266391 9b6b9c6d0d34 34e967a23182 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (7m) - 7m 515M - 18.0.0-6336-g86266391 9b6b9c6d0d34 246ececf6b18 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi098 *:8443,8765 running (5m) - 5m 450M - 18.0.0-6336-g86266391 9b6b9c6d0d34 642e98be66eb 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi123 *:8443,8765 running (5m) - 5m 450M - 18.0.0-6336-g86266391 9b6b9c6d0d34 8df500528090 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (7m) - 7m 84.1M 2048M 18.0.0-6336-g86266391 9b6b9c6d0d34 300e349c0765 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi098 running (6m) - 6m 57.3M 2048M 18.0.0-6336-g86266391 9b6b9c6d0d34 9be13296a478 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi123 running (6m) - 6m 59.0M 2048M 18.0.0-6336-g86266391 9b6b9c6d0d34 fdc165c42e56 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (5m) - 5m 57.1M 4293M 18.0.0-6336-g86266391 9b6b9c6d0d34 2793d1234bf0 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi098 running (5m) - 5m 65.5M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 7212383f92dc 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi123 running (4m) - 4m 58.5M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 d8c966206462 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (4m) - 4m 50.4M 4293M 18.0.0-6336-g86266391 9b6b9c6d0d34 0561fb4fd1d8 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi098 running (4m) - 4m 50.0M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 2b57de03bad2 2023-09-23T09:28:48.733 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi123 running (3m) - 3m 51.1M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 adcf81d04126 2023-09-23T09:28:48.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (3m) - 3m 50.3M 4293M 18.0.0-6336-g86266391 9b6b9c6d0d34 92bdcd30afed 2023-09-23T09:28:48.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi098 running (3m) - 3m 51.8M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 04807e11f2ed 2023-09-23T09:28:48.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi123 running (2m) - 2m 48.7M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 f09bc252e6a1 2023-09-23T09:28:48.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (2m) - 2m 44.3M 4293M 18.0.0-6336-g86266391 9b6b9c6d0d34 a1adf4005e03 2023-09-23T09:28:48.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi098 running (2m) - 2m 47.2M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 ebe2823e8a04 2023-09-23T09:28:48.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi123 running (106s) - 106s 42.7M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 5d0d852357fb 2023-09-23T09:28:48.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd8700a1630 msgr2=0x7fd8700a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd8700a1630 0x7fd8700a3af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd87800b580 tx=0x7fd878005320 comp rx=0 tx=0).stop 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd888100fb0 msgr2=0x7fd888196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd888100fb0 0x7fd888196030 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c950 tx=0x7fd87c00cd10 comp rx=0 tx=0).stop 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 shutdown_connections 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fd8700a1630 0x7fd8700a3af0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101950 0x7fd88819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8880ffdb0 0x7fd888195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 --2- 172.21.15.45:0/1141285643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd888100fb0 0x7fd888196030 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 >> 172.21.15.45:0/1141285643 conn(0x7fd8880fb330 msgr2=0x7fd8881044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 shutdown_connections 2023-09-23T09:28:48.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:48.731+0000 7fd88df13700 1 -- 172.21.15.45:0/1141285643 wait complete. 2023-09-23T09:28:49.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-09-23T09:28:49.689 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:49.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:49 smithi098 ceph-mon[102386]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-23T09:28:49.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:49 smithi098 ceph-mon[102386]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:49.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:49 smithi045 ceph-mon[98627]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-23T09:28:49.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:49 smithi045 ceph-mon[98627]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:49.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:49 smithi123 ceph-mon[103577]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-23T09:28:49.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:49 smithi123 ceph-mon[103577]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:50.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 -- 172.21.15.45:0/1671309578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c0ffdb0 msgr2=0x7f878c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:50.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 --2- 172.21.15.45:0/1671309578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c0ffdb0 0x7f878c1001d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f877c0099c0 tx=0x7f877c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:28:50.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 -- 172.21.15.45:0/1671309578 shutdown_connections 2023-09-23T09:28:50.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 --2- 172.21.15.45:0/1671309578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f878c101950 0x7f878c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:50.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 --2- 172.21.15.45:0/1671309578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c0ffdb0 0x7f878c1001d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:50.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 --2- 172.21.15.45:0/1671309578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f878c100fb0 0x7f878c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:50.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 -- 172.21.15.45:0/1671309578 >> 172.21.15.45:0/1671309578 conn(0x7f878c0fb330 msgr2=0x7f878c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:50.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 -- 172.21.15.45:0/1671309578 shutdown_connections 2023-09-23T09:28:50.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 -- 172.21.15.45:0/1671309578 wait complete. 2023-09-23T09:28:50.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.876+0000 7f8791b46700 1 Processor -- start 2023-09-23T09:28:50.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f8791b46700 1 -- start start 2023-09-23T09:28:50.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f8791b46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f878c0ffdb0 0x7f878c195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:50.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f8791b46700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c100fb0 0x7f878c196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f8791b46700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f878c101950 0x7f878c19d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f8791b46700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c19d760 con 0x7f878c100fb0 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f8791b46700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c19d8a0 con 0x7f878c101950 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f8791b46700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c19da60 con 0x7f878c0ffdb0 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c100fb0 0x7f878c196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c100fb0 0x7f878c196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59762/0 (socket says 172.21.15.45:59762) 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 -- 172.21.15.45:0/30022123 learned_addr learned my addr 172.21.15.45:0/30022123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878b7fe700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f878c0ffdb0 0x7f878c195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 -- 172.21.15.45:0/30022123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f878c0ffdb0 msgr2=0x7f878c195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f878c0ffdb0 0x7f878c195c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 -- 172.21.15.45:0/30022123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f878c101950 msgr2=0x7f878c19d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878bfff700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f878c101950 0x7f878c19d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f878c101950 0x7f878c19d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 -- 172.21.15.45:0/30022123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c0095f0 con 0x7f878c100fb0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878bfff700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f878c101950 0x7f878c19d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.877+0000 7f878affd700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c100fb0 0x7f878c196150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f877400b530 tx=0x7f877400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.878+0000 7f8788ff9700 1 -- 172.21.15.45:0/30022123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774011830 con 0x7f878c100fb0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.878+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c19dd40 con 0x7f878c100fb0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.878+0000 7f8788ff9700 1 -- 172.21.15.45:0/30022123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8774011e50 con 0x7f878c100fb0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.878+0000 7f8788ff9700 1 -- 172.21.15.45:0/30022123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877400f2e0 con 0x7f878c100fb0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.878+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c19e190 con 0x7f878c100fb0 2023-09-23T09:28:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.879+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c107400 con 0x7f878c100fb0 2023-09-23T09:28:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.880+0000 7f8788ff9700 1 -- 172.21.15.45:0/30022123 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f8774010580 con 0x7f878c100fb0 2023-09-23T09:28:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.880+0000 7f8788ff9700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f87780a1630 0x7f87780a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.881+0000 7f878b7fe700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f87780a1630 0x7f87780a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.881+0000 7f8788ff9700 1 -- 172.21.15.45:0/30022123 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f87740cce90 con 0x7f878c100fb0 2023-09-23T09:28:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.881+0000 7f878b7fe700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f87780a1630 0x7f87780a3af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f877c0098e0 tx=0x7f877c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:50.884+0000 7f8788ff9700 1 -- 172.21.15.45:0/30022123 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f87740965e0 con 0x7f878c100fb0 2023-09-23T09:28:51.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.091+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f878c10b6b0 con 0x7f87780a1630 2023-09-23T09:28:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.098+0000 7f8788ff9700 1 -- 172.21.15.45:0/30022123 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f878c10b6b0 con 0x7f87780a1630 2023-09-23T09:28:51.099 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-23T09:28:51.099 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 - 93s smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-23T09:28:51.100 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 - 5m smithi045=x;smithi098=y;smithi123=z;count:3 2023-09-23T09:28:51.100 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 - 6m smithi045:172.21.15.45=a;smithi098:172.21.15.98=b;smithi123:172.21.15.123=c;count:3 2023-09-23T09:28:51.100 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 - - 2023-09-23T09:28:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.099+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f87780a1630 msgr2=0x7f87780a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.099+0000 7f8791b46700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f87780a1630 0x7f87780a3af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f877c0098e0 tx=0x7f877c000bc0 comp rx=0 tx=0).stop 2023-09-23T09:28:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.099+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c100fb0 msgr2=0x7f878c196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.099+0000 7f8791b46700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c100fb0 0x7f878c196150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f877400b530 tx=0x7f877400b8f0 comp rx=0 tx=0).stop 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 shutdown_connections 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f878c101950 0x7f878c19d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f87780a1630 0x7f87780a3af0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f878c0ffdb0 0x7f878c195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 --2- 172.21.15.45:0/30022123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c100fb0 0x7f878c196150 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 >> 172.21.15.45:0/30022123 conn(0x7f878c0fb330 msgr2=0x7f878c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 shutdown_connections 2023-09-23T09:28:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:51.100+0000 7f8791b46700 1 -- 172.21.15.45:0/30022123 wait complete. 2023-09-23T09:28:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:51 smithi045 ceph-mon[98627]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:51 smithi045 ceph-mon[98627]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:51 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:28:51.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:51 smithi098 ceph-mon[102386]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:51.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:51 smithi098 ceph-mon[102386]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:51.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:51 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:28:51.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-09-23T09:28:51.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:51 smithi123 ceph-mon[103577]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:51.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:51 smithi123 ceph-mon[103577]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:51.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:51 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:28:52.120 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:53.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.539+0000 7f986f5dd700 1 -- 172.21.15.45:0/1508215749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868100e70 msgr2=0x7f9868101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:53.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.539+0000 7f986f5dd700 1 --2- 172.21.15.45:0/1508215749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868100e70 0x7f9868101290 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f986400b0a0 tx=0x7f986400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 -- 172.21.15.45:0/1508215749 shutdown_connections 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 --2- 172.21.15.45:0/1508215749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9868102a20 0x7f9868109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 --2- 172.21.15.45:0/1508215749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868100e70 0x7f9868101290 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 --2- 172.21.15.45:0/1508215749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9868102060 0x7f98681024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 -- 172.21.15.45:0/1508215749 >> 172.21.15.45:0/1508215749 conn(0x7f98680fc440 msgr2=0x7f98680fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 -- 172.21.15.45:0/1508215749 shutdown_connections 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 -- 172.21.15.45:0/1508215749 wait complete. 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.540+0000 7f986f5dd700 1 Processor -- start 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986f5dd700 1 -- start start 2023-09-23T09:28:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986f5dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9868100e70 0x7f9868074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986f5dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868102060 0x7f9868073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986f5dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9868102a20 0x7f9868073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986f5dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868073ed0 con 0x7f9868102060 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986f5dd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868074040 con 0x7f9868100e70 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986f5dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98681a0d70 con 0x7f9868102a20 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868102060 0x7f9868073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868102060 0x7f9868073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59774/0 (socket says 172.21.15.45:59774) 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 -- 172.21.15.45:0/3117196471 learned_addr learned my addr 172.21.15.45:0/3117196471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986d379700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9868100e70 0x7f9868074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 -- 172.21.15.45:0/3117196471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9868102a20 msgr2=0x7f9868073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9868102a20 0x7f9868073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 -- 172.21.15.45:0/3117196471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9868100e70 msgr2=0x7f9868074bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9868100e70 0x7f9868074bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 -- 172.21.15.45:0/3117196471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864009d00 con 0x7f9868102060 2023-09-23T09:28:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f986cb78700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868102060 0x7f9868073210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f985800d980 tx=0x7f985800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.541+0000 7f985e7fc700 1 -- 172.21.15.45:0/3117196471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858004020 con 0x7f9868102060 2023-09-23T09:28:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.542+0000 7f985e7fc700 1 -- 172.21.15.45:0/3117196471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9858009730 con 0x7f9868102060 2023-09-23T09:28:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.542+0000 7f985e7fc700 1 -- 172.21.15.45:0/3117196471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858003a80 con 0x7f9868102060 2023-09-23T09:28:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.542+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98681a0ff0 con 0x7f9868102060 2023-09-23T09:28:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.542+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98681a1470 con 0x7f9868102060 2023-09-23T09:28:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.543+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986810c6f0 con 0x7f9868102060 2023-09-23T09:28:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.544+0000 7f985e7fc700 1 -- 172.21.15.45:0/3117196471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f98580098a0 con 0x7f9868102060 2023-09-23T09:28:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.544+0000 7f985e7fc700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98540a1630 0x7f98540a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.544+0000 7f985e7fc700 1 -- 172.21.15.45:0/3117196471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f98580ccbf0 con 0x7f9868102060 2023-09-23T09:28:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.544+0000 7f986d379700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98540a1630 0x7f98540a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.545+0000 7f986d379700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98540a1630 0x7f98540a3af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98640079f0 tx=0x7f986400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:53.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.549+0000 7f985e7fc700 1 -- 172.21.15.45:0/3117196471 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9858096230 con 0x7f9868102060 2023-09-23T09:28:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.761+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9868061260 con 0x7f98540a1630 2023-09-23T09:28:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.763+0000 7f985e7fc700 1 -- 172.21.15.45:0/3117196471 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f9868061260 con 0x7f98540a1630 2023-09-23T09:28:53.765 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2023-09-23T09:28:53.765 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2023-09-23T09:28:53.765 INFO:teuthology.orchestra.run.smithi045.stdout:smithi098 172.21.15.98 2023-09-23T09:28:53.765 INFO:teuthology.orchestra.run.smithi045.stdout:smithi123 172.21.15.123 2023-09-23T09:28:53.765 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98540a1630 msgr2=0x7f98540a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98540a1630 0x7f98540a3af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98640079f0 tx=0x7f986400b700 comp rx=0 tx=0).stop 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868102060 msgr2=0x7f9868073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868102060 0x7f9868073210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f985800d980 tx=0x7f985800dc90 comp rx=0 tx=0).stop 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 shutdown_connections 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98540a1630 0x7f98540a3af0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9868102a20 0x7f9868073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9868100e70 0x7f9868074bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 --2- 172.21.15.45:0/3117196471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868102060 0x7f9868073210 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 >> 172.21.15.45:0/3117196471 conn(0x7f98680fc440 msgr2=0x7f98680fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 shutdown_connections 2023-09-23T09:28:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:53.767+0000 7f986f5dd700 1 -- 172.21.15.45:0/3117196471 wait complete. 2023-09-23T09:28:53.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:53 smithi098 ceph-mon[102386]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:53.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:53 smithi045 ceph-mon[98627]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:53.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:53 smithi123 ceph-mon[103577]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-23T09:28:54.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-09-23T09:28:54.702 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:55.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-09-23T09:28:55.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-23T09:28:55.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-09-23T09:28:55.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-09-23T09:28:55.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:54 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.115 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-09-23T09:28:55.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-23T09:28:55.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-09-23T09:28:55.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-09-23T09:28:55.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:54 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:55.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-23T09:28:55.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-09-23T09:28:55.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-09-23T09:28:55.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:54 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/344222817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102a20 msgr2=0x7f3db8109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/344222817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102a20 0x7f3db8109340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b0a0 tx=0x7f3db000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/344222817 shutdown_connections 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/344222817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102a20 0x7f3db8109340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/344222817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db8100e70 0x7f3db8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/344222817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3db8102060 0x7f3db81024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/344222817 >> 172.21.15.45:0/344222817 conn(0x7f3db80fc440 msgr2=0x7f3db80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/344222817 shutdown_connections 2023-09-23T09:28:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.866+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/344222817 wait complete. 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 Processor -- start 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 -- start start 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db8100e70 0x7f3db8074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102060 0x7f3db8073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3db8102a20 0x7f3db8073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8073ed0 con 0x7f3db8102060 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8074040 con 0x7f3db8102a20 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3dbd5ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81a0bb0 con 0x7f3db8100e70 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3db67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102060 0x7f3db8073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3db67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102060 0x7f3db8073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59792/0 (socket says 172.21.15.45:59792) 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3db67fc700 1 -- 172.21.15.45:0/2222551776 learned_addr learned my addr 172.21.15.45:0/2222551776 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3db6ffd700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db8100e70 0x7f3db8074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.867+0000 7f3db77fe700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3db8102a20 0x7f3db8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3db67fc700 1 -- 172.21.15.45:0/2222551776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db8100e70 msgr2=0x7f3db8074bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3db67fc700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db8100e70 0x7f3db8074bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3db67fc700 1 -- 172.21.15.45:0/2222551776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3db8102a20 msgr2=0x7f3db8073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3db67fc700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3db8102a20 0x7f3db8073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3db67fc700 1 -- 172.21.15.45:0/2222551776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0009d00 con 0x7f3db8102060 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3db67fc700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102060 0x7f3db8073210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00e970 tx=0x7f3dac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3d9ffff700 1 -- 172.21.15.45:0/2222551776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac009790 con 0x7f3db8102060 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3d9ffff700 1 -- 172.21.15.45:0/2222551776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dac00c7e0 con 0x7f3db8102060 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3d9ffff700 1 -- 172.21.15.45:0/2222551776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac003c70 con 0x7f3db8102060 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db81a0e90 con 0x7f3db8102060 2023-09-23T09:28:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.868+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db81a1300 con 0x7f3db8102060 2023-09-23T09:28:55.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.869+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db810c740 con 0x7f3db8102060 2023-09-23T09:28:55.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.870+0000 7f3d9ffff700 1 -- 172.21.15.45:0/2222551776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f3dac00cc50 con 0x7f3db8102060 2023-09-23T09:28:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.871+0000 7f3d9ffff700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3da00a16b0 0x7f3da00a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.871+0000 7f3db6ffd700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3da00a16b0 0x7f3da00a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.871+0000 7f3d9ffff700 1 -- 172.21.15.45:0/2222551776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3dac014070 con 0x7f3db8102060 2023-09-23T09:28:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.871+0000 7f3db6ffd700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3da00a16b0 0x7f3da00a3b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3db8101bd0 tx=0x7f3da800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:55.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:55.875+0000 7f3d9ffff700 1 -- 172.21.15.45:0/2222551776 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3dac096670 con 0x7f3db8102060 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-09-23T09:28:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-23T09:28:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-09-23T09:28:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-09-23T09:28:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: osdmap e84: 12 total, 12 up, 12 in 2023-09-23T09:28:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:28:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:28:56.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:55 smithi045 ceph-mon[98627]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.086+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3db8061260 con 0x7f3da00a16b0 2023-09-23T09:28:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.088+0000 7f3d9ffff700 1 -- 172.21.15.45:0/2222551776 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f3db8061260 con 0x7f3da00a16b0 2023-09-23T09:28:56.090 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-23T09:28:56.090 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-09-23T09:28:56.090 INFO:teuthology.orchestra.run.smithi045.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-23T09:28:56.090 INFO:teuthology.orchestra.run.smithi045.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 107s ago Has a FileSystem, LVM detected 2023-09-23T09:28:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3da00a16b0 msgr2=0x7f3da00a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3da00a16b0 0x7f3da00a3b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3db8101bd0 tx=0x7f3da800a380 comp rx=0 tx=0).stop 2023-09-23T09:28:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102060 msgr2=0x7f3db8073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102060 0x7f3db8073210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00e970 tx=0x7f3dac00ec80 comp rx=0 tx=0).stop 2023-09-23T09:28:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 shutdown_connections 2023-09-23T09:28:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3db8102a20 0x7f3db8073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3da00a16b0 0x7f3da00a3b70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db8100e70 0x7f3db8074bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 --2- 172.21.15.45:0/2222551776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db8102060 0x7f3db8073210 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.090+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 >> 172.21.15.45:0/2222551776 conn(0x7f3db80fc440 msgr2=0x7f3db80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.091+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 shutdown_connections 2023-09-23T09:28:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:56.091+0000 7f3dbd5ec700 1 -- 172.21.15.45:0/2222551776 wait complete. 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-23T09:28:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-09-23T09:28:56.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-09-23T09:28:56.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: osdmap e84: 12 total, 12 up, 12 in 2023-09-23T09:28:56.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:28:56.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:28:56.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:55 smithi098 ceph-mon[102386]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:56.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-23T09:28:56.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: osdmap e84: 12 total, 12 up, 12 in 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:28:56.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:55 smithi123 ceph-mon[103577]: from='mgr.14152 172.21.15.45:0/3766377852' entity='mgr.x' 2023-09-23T09:28:56.447 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-23T09:28:56.456 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:28:56.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-09-23T09:28:57.018 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:57.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:56 smithi045 ceph-mon[98627]: osdmap e85: 12 total, 12 up, 12 in 2023-09-23T09:28:57.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:56 smithi045 ceph-mon[98627]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:57.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:56 smithi098 ceph-mon[102386]: osdmap e85: 12 total, 12 up, 12 in 2023-09-23T09:28:57.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:56 smithi098 ceph-mon[102386]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:57.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:56 smithi123 ceph-mon[103577]: osdmap e85: 12 total, 12 up, 12 in 2023-09-23T09:28:57.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:56 smithi123 ceph-mon[103577]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:28:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 -- 172.21.15.45:0/2757157613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750102080 msgr2=0x7fe750102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 --2- 172.21.15.45:0/2757157613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750102080 0x7fe750102500 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe744009990 tx=0x7fe744009ca0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 -- 172.21.15.45:0/2757157613 shutdown_connections 2023-09-23T09:28:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 --2- 172.21.15.45:0/2757157613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe750102a40 0x7fe750109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 --2- 172.21.15.45:0/2757157613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe750100e90 0x7fe7501012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 --2- 172.21.15.45:0/2757157613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750102080 0x7fe750102500 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 -- 172.21.15.45:0/2757157613 >> 172.21.15.45:0/2757157613 conn(0x7fe7500fc4a0 msgr2=0x7fe7500fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 -- 172.21.15.45:0/2757157613 shutdown_connections 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.187+0000 7fe757431700 1 -- 172.21.15.45:0/2757157613 wait complete. 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 Processor -- start 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 -- start start 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750100e90 0x7fe750196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe750102080 0x7fe7501970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe750102a40 0x7fe75019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75019e730 con 0x7fe750100e90 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75019e870 con 0x7fe750102080 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe757431700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75019ea00 con 0x7fe750102a40 2023-09-23T09:28:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe7551cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750100e90 0x7fe750196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe7551cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750100e90 0x7fe750196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59804/0 (socket says 172.21.15.45:59804) 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe7551cd700 1 -- 172.21.15.45:0/2532471418 learned_addr learned my addr 172.21.15.45:0/2532471418 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe7549cc700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe750102080 0x7fe7501970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.188+0000 7fe7559ce700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe750102a40 0x7fe75019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7551cd700 1 -- 172.21.15.45:0/2532471418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe750102a40 msgr2=0x7fe75019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7551cd700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe750102a40 0x7fe75019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7551cd700 1 -- 172.21.15.45:0/2532471418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe750102080 msgr2=0x7fe7501970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7551cd700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe750102080 0x7fe7501970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7551cd700 1 -- 172.21.15.45:0/2532471418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440095f0 con 0x7fe750100e90 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7551cd700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750100e90 0x7fe750196ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c9a0 tx=0x7fe74c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7427fc700 1 -- 172.21.15.45:0/2532471418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c004220 con 0x7fe750100e90 2023-09-23T09:28:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7427fc700 1 -- 172.21.15.45:0/2532471418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe74c007d20 con 0x7fe750100e90 2023-09-23T09:28:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75019ece0 con 0x7fe750100e90 2023-09-23T09:28:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe7427fc700 1 -- 172.21.15.45:0/2532471418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c003c00 con 0x7fe750100e90 2023-09-23T09:28:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.189+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75019f190 con 0x7fe750100e90 2023-09-23T09:28:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.190+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750108510 con 0x7fe750100e90 2023-09-23T09:28:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.191+0000 7fe7427fc700 1 -- 172.21.15.45:0/2532471418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fe74c007860 con 0x7fe750100e90 2023-09-23T09:28:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.192+0000 7fe7427fc700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe73c0a1630 0x7fe73c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:28:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.192+0000 7fe7549cc700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe73c0a1630 0x7fe73c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:28:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.192+0000 7fe7427fc700 1 -- 172.21.15.45:0/2532471418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe74c0cd080 con 0x7fe750100e90 2023-09-23T09:28:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.192+0000 7fe7549cc700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe73c0a1630 0x7fe73c0a3af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe744006010 tx=0x7fe744005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:28:58.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.195+0000 7fe7427fc700 1 -- 172.21.15.45:0/2532471418 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe74c0964d0 con 0x7fe750100e90 2023-09-23T09:28:58.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:57 smithi098 ceph-mon[102386]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-23T09:28:58.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:57 smithi098 ceph-mon[102386]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-23T09:28:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:57 smithi045 ceph-mon[98627]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-23T09:28:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:57 smithi045 ceph-mon[98627]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-23T09:28:58.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:57 smithi123 ceph-mon[103577]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-23T09:28:58.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:57 smithi123 ceph-mon[103577]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-23T09:28:58.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.439+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe75010c9c0 con 0x7fe750100e90 2023-09-23T09:28:58.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.439+0000 7fe7427fc700 1 -- 172.21.15.45:0/2532471418 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2933 (secure 0 0 0) 0x7fe74c01d070 con 0x7fe750100e90 2023-09-23T09:28:58.441 INFO:teuthology.orchestra.run.smithi045.stdout:e22 2023-09-23T09:28:58.441 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T09:28:58.441 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T09:28:58.441 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 22 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:created 2023-09-23T09:27:50.073600+0000 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2023-09-23T09:27:55.252601+0000 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2023-09-23T09:28:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 3 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:in 0,1,2 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24530,1=24500,2=24496} 2023-09-23T09:28:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3] 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24530} state up:active seq 3 addr [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.123:6836/2272642085,v1:172.21.15.123:6837/2272642085] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{1:24500} state up:active seq 6 addr [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.444 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{1:24466} state up:standby-replay seq 1 addr [v2:172.21.15.123:6834/3236511886,v1:172.21.15.123:6835/3236511886] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{2:24496} state up:active seq 6 addr [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{2:24464} state up:standby-replay seq 1 addr [v2:172.21.15.98:6834/3506316120,v1:172.21.15.98:6835/3506316120] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.45:6834/427947541,v1:172.21.15.45:6835/427947541] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.45:6836/1158485570,v1:172.21.15.45:6837/1158485570] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{-1:14607} state up:standby seq 1 addr [v2:172.21.15.45:6838/581280967,v1:172.21.15.45:6839/581280967] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{-1:14619} state up:standby seq 1 addr [v2:172.21.15.45:6840/4027335319,v1:172.21.15.45:6841/4027335319] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{-1:24433} state up:standby seq 1 addr [v2:172.21.15.123:6832/782227022,v1:172.21.15.123:6833/782227022] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{-1:24440} state up:standby seq 1 addr [v2:172.21.15.98:6832/3376968618,v1:172.21.15.98:6833/3376968618] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T09:28:58.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe73c0a1630 msgr2=0x7fe73c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:58.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe73c0a1630 0x7fe73c0a3af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe744006010 tx=0x7fe744005960 comp rx=0 tx=0).stop 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750100e90 msgr2=0x7fe750196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750100e90 0x7fe750196ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c9a0 tx=0x7fe74c00ccb0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 shutdown_connections 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fe73c0a1630 0x7fe73c0a3af0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe750102a40 0x7fe75019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe750100e90 0x7fe750196ba0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 --2- 172.21.15.45:0/2532471418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe750102080 0x7fe7501970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 >> 172.21.15.45:0/2532471418 conn(0x7fe7500fc4a0 msgr2=0x7fe750105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:28:58.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 shutdown_connections 2023-09-23T09:28:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:28:58.441+0000 7fe757431700 1 -- 172.21.15.45:0/2532471418 wait complete. 2023-09-23T09:28:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:28:58.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-09-23T09:28:59.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:58 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2532471418' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-23T09:28:59.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:58 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2532471418' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-23T09:28:59.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:58 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2532471418' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-23T09:28:59.433 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:28:59.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:28:59 smithi045 ceph-mon[98627]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:00.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:28:59 smithi098 ceph-mon[102386]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:28:59 smithi123 ceph-mon[103577]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 -- 172.21.15.45:0/3529555583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 msgr2=0x7fb4381001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 --2- 172.21.15.45:0/3529555583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb4381001d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0099c0 tx=0x7fb42c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 -- 172.21.15.45:0/3529555583 shutdown_connections 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 --2- 172.21.15.45:0/3529555583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb438101950 0x7fb438108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 --2- 172.21.15.45:0/3529555583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb4381001d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 --2- 172.21.15.45:0/3529555583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438100fb0 0x7fb438101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 -- 172.21.15.45:0/3529555583 >> 172.21.15.45:0/3529555583 conn(0x7fb4380fb330 msgr2=0x7fb4380fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 -- 172.21.15.45:0/3529555583 shutdown_connections 2023-09-23T09:29:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 -- 172.21.15.45:0/3529555583 wait complete. 2023-09-23T09:29:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.525+0000 7fb43d623700 1 Processor -- start 2023-09-23T09:29:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb43d623700 1 -- start start 2023-09-23T09:29:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb43d623700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb438069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb43d623700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438100fb0 0x7fb43819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb43d623700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb438101950 0x7fb43819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb43d623700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43819e6d0 con 0x7fb4380ffdb0 2023-09-23T09:29:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb43d623700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43819e840 con 0x7fb438100fb0 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb43d623700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43819ea00 con 0x7fb438101950 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb438069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb438069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59818/0 (socket says 172.21.15.45:59818) 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 -- 172.21.15.45:0/2472413400 learned_addr learned my addr 172.21.15.45:0/2472413400 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb4367fc700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438100fb0 0x7fb43819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb4377fe700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb438101950 0x7fb43819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 -- 172.21.15.45:0/2472413400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb438101950 msgr2=0x7fb43819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb438101950 0x7fb43819df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 -- 172.21.15.45:0/2472413400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438100fb0 msgr2=0x7fb43819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438100fb0 0x7fb43819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 -- 172.21.15.45:0/2472413400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c0095f0 con 0x7fb4380ffdb0 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb4367fc700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438100fb0 0x7fb43819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb4377fe700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb438101950 0x7fb43819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.526+0000 7fb436ffd700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb438069460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009990 tx=0x7fb42c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.527+0000 7fb41ffff700 1 -- 172.21.15.45:0/2472413400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c01d070 con 0x7fb4380ffdb0 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.527+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43819ec80 con 0x7fb4380ffdb0 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.527+0000 7fb41ffff700 1 -- 172.21.15.45:0/2472413400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42c004200 con 0x7fb4380ffdb0 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.527+0000 7fb41ffff700 1 -- 172.21.15.45:0/2472413400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c022e30 con 0x7fb4380ffdb0 2023-09-23T09:29:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.527+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43819f120 con 0x7fb4380ffdb0 2023-09-23T09:29:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.528+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb438107420 con 0x7fb4380ffdb0 2023-09-23T09:29:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.529+0000 7fb41ffff700 1 -- 172.21.15.45:0/2472413400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fb42c003d40 con 0x7fb4380ffdb0 2023-09-23T09:29:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.529+0000 7fb41ffff700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb4200a1630 0x7fb4200a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.529+0000 7fb4367fc700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb4200a1630 0x7fb4200a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.530+0000 7fb41ffff700 1 -- 172.21.15.45:0/2472413400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb42c0ce470 con 0x7fb4380ffdb0 2023-09-23T09:29:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.530+0000 7fb4367fc700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb4200a1630 0x7fb4200a3af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb428005fd0 tx=0x7fb428005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:00.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.534+0000 7fb41ffff700 1 -- 172.21.15.45:0/2472413400 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb42c097950 con 0x7fb4380ffdb0 2023-09-23T09:29:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.736+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fb43810b7d0 con 0x7fb4380ffdb0 2023-09-23T09:29:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.737+0000 7fb41ffff700 1 -- 172.21.15.45:0/2472413400 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5286 (secure 0 0 0) 0x7fb42c09af70 con 0x7fb4380ffdb0 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 85 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:created 2023-09-23T09:20:53.563973+0000 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2023-09-23T09:28:55.849323+0000 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 26 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2023-09-23T09:29:00.739 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release reef 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/1500595998,v1:172.21.15.45:6803/1500595998] [v2:172.21.15.45:6804/1500595998,v1:172.21.15.45:6805/1500595998] exists,up 8136dd58-335a-4231-97c9-99ae3b5e6f8f 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/2863958609,v1:172.21.15.98:6801/2863958609] [v2:172.21.15.98:6802/2863958609,v1:172.21.15.98:6803/2863958609] exists,up 934fb488-aa5c-4a75-acf1-4f1a42c9a428 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/976297925,v1:172.21.15.123:6801/976297925] [v2:172.21.15.123:6802/976297925,v1:172.21.15.123:6803/976297925] exists,up 752d7ddf-4714-4f48-b2ce-5ad5e04e6426 2023-09-23T09:29:00.740 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/3800235078,v1:172.21.15.45:6811/3800235078] [v2:172.21.15.45:6812/3800235078,v1:172.21.15.45:6813/3800235078] exists,up 20e42b6b-8c74-4414-a64d-e76bb61de1aa 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/4058003666,v1:172.21.15.98:6809/4058003666] [v2:172.21.15.98:6810/4058003666,v1:172.21.15.98:6811/4058003666] exists,up ecfe4f49-dd41-46e6-b948-c396c14fe25a 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/3175137003,v1:172.21.15.123:6809/3175137003] [v2:172.21.15.123:6810/3175137003,v1:172.21.15.123:6811/3175137003] exists,up 1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 45 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/668839971,v1:172.21.15.45:6819/668839971] [v2:172.21.15.45:6820/668839971,v1:172.21.15.45:6821/668839971] exists,up ce5cc090-fa1f-4813-8950-970817a59b7d 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 50 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/3609791505,v1:172.21.15.98:6817/3609791505] [v2:172.21.15.98:6818/3609791505,v1:172.21.15.98:6819/3609791505] exists,up f045102a-f2d2-4e94-a7ac-ff19e15c4126 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 58 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/101690512,v1:172.21.15.123:6817/101690512] [v2:172.21.15.123:6818/101690512,v1:172.21.15.123:6819/101690512] exists,up 7fc7b9f6-15f6-4025-8728-353e53602172 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/2888091166,v1:172.21.15.45:6827/2888091166] [v2:172.21.15.45:6828/2888091166,v1:172.21.15.45:6829/2888091166] exists,up 3029adaf-0e5e-4ab8-8b6d-d5376029092c 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 71 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/2174091835,v1:172.21.15.98:6825/2174091835] [v2:172.21.15.98:6826/2174091835,v1:172.21.15.98:6827/2174091835] exists,up 30007304-e7de-46d9-8b01-c03dfc7c22a7 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 78 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/523003313,v1:172.21.15.123:6825/523003313] [v2:172.21.15.123:6826/523003313,v1:172.21.15.123:6827/523003313] exists,up fbebaf92-0e82-4758-b581-d15dc738abec 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.4 [2,11] 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.6 [8,11] 2023-09-23T09:29:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.e [2,4] 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1a [8,1] 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1 [4,9] 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.c [3,9] 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.13 [4,9] 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1a [5,6] 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.23 [4,8] 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/690218910 expires 2023-09-24T09:21:54.248141+0000 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/1791488577 expires 2023-09-24T09:21:54.248141+0000 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/1791488577 expires 2023-09-24T09:21:54.248141+0000 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3832887419 expires 2023-09-24T09:21:28.141138+0000 2023-09-23T09:29:00.742 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/4072483061 expires 2023-09-24T09:21:28.141138+0000 2023-09-23T09:29:00.743 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/4072483061 expires 2023-09-24T09:21:28.141138+0000 2023-09-23T09:29:00.743 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3656954721 expires 2023-09-24T09:21:54.248141+0000 2023-09-23T09:29:00.743 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/258118400 expires 2023-09-24T09:21:28.141138+0000 2023-09-23T09:29:00.743 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3011609987 expires 2023-09-24T09:21:28.141138+0000 2023-09-23T09:29:00.743 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2302872888 expires 2023-09-24T09:21:54.248141+0000 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb4200a1630 msgr2=0x7fb4200a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb4200a1630 0x7fb4200a3af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb428005fd0 tx=0x7fb428005dc0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 msgr2=0x7fb438069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb438069460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009990 tx=0x7fb42c005e80 comp rx=0 tx=0).stop 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 shutdown_connections 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb4200a1630 0x7fb4200a3af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb438101950 0x7fb43819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4380ffdb0 0x7fb438069460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 --2- 172.21.15.45:0/2472413400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438100fb0 0x7fb43819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 >> 172.21.15.45:0/2472413400 conn(0x7fb4380fb330 msgr2=0x7fb438104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 shutdown_connections 2023-09-23T09:29:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:00.739+0000 7fb43d623700 1 -- 172.21.15.45:0/2472413400 wait complete. 2023-09-23T09:29:01.092 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-23T09:29:01.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:29:01.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 -- 172.21.15.45:0/4063435251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0140ffd00 msgr2=0x7fd014100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 --2- 172.21.15.45:0/4063435251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0140ffd00 0x7fd014100120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd0040099c0 tx=0x7fd004009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 -- 172.21.15.45:0/4063435251 shutdown_connections 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 --2- 172.21.15.45:0/4063435251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd014101890 0x7fd014108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 --2- 172.21.15.45:0/4063435251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0140ffd00 0x7fd014100120 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 --2- 172.21.15.45:0/4063435251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd014100ef0 0x7fd014101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 -- 172.21.15.45:0/4063435251 >> 172.21.15.45:0/4063435251 conn(0x7fd0140fb250 msgr2=0x7fd0140fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 -- 172.21.15.45:0/4063435251 shutdown_connections 2023-09-23T09:29:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 -- 172.21.15.45:0/4063435251 wait complete. 2023-09-23T09:29:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 Processor -- start 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.231+0000 7fd018e0b700 1 -- start start 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd018e0b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0140ffd00 0x7fd014068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd018e0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd014100ef0 0x7fd01419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd018e0b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd014101890 0x7fd01419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd018e0b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01419e630 con 0x7fd014100ef0 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd018e0b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01419e7a0 con 0x7fd014101890 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd018e0b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01419e960 con 0x7fd0140ffd00 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd014100ef0 0x7fd01419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd014100ef0 0x7fd01419b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59832/0 (socket says 172.21.15.45:59832) 2023-09-23T09:29:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 -- 172.21.15.45:0/3782305955 learned_addr learned my addr 172.21.15.45:0/3782305955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd01259c700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0140ffd00 0x7fd014068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd012d9d700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd014101890 0x7fd01419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 -- 172.21.15.45:0/3782305955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0140ffd00 msgr2=0x7fd014068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0140ffd00 0x7fd014068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 -- 172.21.15.45:0/3782305955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd014101890 msgr2=0x7fd01419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd014101890 0x7fd01419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 -- 172.21.15.45:0/3782305955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040095f0 con 0x7fd014100ef0 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd011d9b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd014100ef0 0x7fd01419b850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d980 tx=0x7fd00800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.232+0000 7fd0037fe700 1 -- 172.21.15.45:0/3782305955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd008004220 con 0x7fd014100ef0 2023-09-23T09:29:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.233+0000 7fd0037fe700 1 -- 172.21.15.45:0/3782305955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd008007640 con 0x7fd014100ef0 2023-09-23T09:29:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.233+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01419ec40 con 0x7fd014100ef0 2023-09-23T09:29:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.233+0000 7fd0037fe700 1 -- 172.21.15.45:0/3782305955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd008005690 con 0x7fd014100ef0 2023-09-23T09:29:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.233+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01419f150 con 0x7fd014100ef0 2023-09-23T09:29:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.233+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd014106c80 con 0x7fd014100ef0 2023-09-23T09:29:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.235+0000 7fd0037fe700 1 -- 172.21.15.45:0/3782305955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fd008004750 con 0x7fd014100ef0 2023-09-23T09:29:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.236+0000 7fd0037fe700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcffc0a1670 0x7fcffc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.236+0000 7fd01259c700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcffc0a1670 0x7fcffc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.236+0000 7fd0037fe700 1 -- 172.21.15.45:0/3782305955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0080cd3a0 con 0x7fd014100ef0 2023-09-23T09:29:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.236+0000 7fd01259c700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcffc0a1670 0x7fcffc0a3b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd004009f50 tx=0x7fd00400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.239+0000 7fd0037fe700 1 -- 172.21.15.45:0/3782305955 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd0080967f0 con 0x7fd014100ef0 2023-09-23T09:29:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.443+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0140008d0 con 0x7fd014100ef0 2023-09-23T09:29:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.444+0000 7fd0037fe700 1 -- 172.21.15.45:0/3782305955 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22481 (secure 0 0 0) 0x7fd008099e10 con 0x7fd014100ef0 2023-09-23T09:29:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:01.447 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:28:55.849323+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:29:01.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.446+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcffc0a1670 msgr2=0x7fcffc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.446+0000 7fd018e0b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcffc0a1670 0x7fcffc0a3b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd004009f50 tx=0x7fd00400b540 comp rx=0 tx=0).stop 2023-09-23T09:29:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.446+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd014100ef0 msgr2=0x7fd01419b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.446+0000 7fd018e0b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd014100ef0 0x7fd01419b850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d980 tx=0x7fd00800dc90 comp rx=0 tx=0).stop 2023-09-23T09:29:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 shutdown_connections 2023-09-23T09:29:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd014101890 0x7fd01419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fcffc0a1670 0x7fcffc0a3b30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0140ffd00 0x7fd014068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 --2- 172.21.15.45:0/3782305955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd014100ef0 0x7fd01419b850 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 >> 172.21.15.45:0/3782305955 conn(0x7fd0140fb250 msgr2=0x7fd0140fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 shutdown_connections 2023-09-23T09:29:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.447+0000 7fd018e0b700 1 -- 172.21.15.45:0/3782305955 wait complete. 2023-09-23T09:29:01.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:01 smithi045 ceph-mon[98627]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:01.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:01 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2472413400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-23T09:29:01.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:29:01.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:29:01.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:01 smithi098 ceph-mon[102386]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:01 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2472413400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-23T09:29:01.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:01 smithi123 ceph-mon[103577]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:01.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:01 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2472413400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-23T09:29:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 -- 172.21.15.45:0/2962647131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f581018f0 msgr2=0x7f6f58103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 --2- 172.21.15.45:0/2962647131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f581018f0 0x7f6f58103ce0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009990 tx=0x7f6f48009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 -- 172.21.15.45:0/2962647131 shutdown_connections 2023-09-23T09:29:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 --2- 172.21.15.45:0/2962647131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f58106b50 0x7f6f58108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 --2- 172.21.15.45:0/2962647131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f581018f0 0x7f6f58103ce0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 --2- 172.21.15.45:0/2962647131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f58104220 0x7f6f58106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 -- 172.21.15.45:0/2962647131 >> 172.21.15.45:0/2962647131 conn(0x7f6f580fb280 msgr2=0x7f6f580fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 -- 172.21.15.45:0/2962647131 shutdown_connections 2023-09-23T09:29:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.589+0000 7f6f6072a700 1 -- 172.21.15.45:0/2962647131 wait complete. 2023-09-23T09:29:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 Processor -- start 2023-09-23T09:29:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 -- start start 2023-09-23T09:29:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f581018f0 0x7f6f58196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f58104220 0x7f6f58196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f58106b50 0x7f6f5819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5819e620 con 0x7f6f58104220 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5819e760 con 0x7f6f58106b50 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f6072a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5819e8f0 con 0x7f6f581018f0 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f5dcc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f58104220 0x7f6f58196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f5dcc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f58104220 0x7f6f58196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59844/0 (socket says 172.21.15.45:59844) 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f5dcc5700 1 -- 172.21.15.45:0/935903194 learned_addr learned my addr 172.21.15.45:0/935903194 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f5ecc7700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f58106b50 0x7f6f5819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.590+0000 7f6f5e4c6700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f581018f0 0x7f6f58196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f5dcc5700 1 -- 172.21.15.45:0/935903194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f581018f0 msgr2=0x7f6f58196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f5dcc5700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f581018f0 0x7f6f58196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f5dcc5700 1 -- 172.21.15.45:0/935903194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f58106b50 msgr2=0x7f6f5819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f5dcc5700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f58106b50 0x7f6f5819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f5dcc5700 1 -- 172.21.15.45:0/935903194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480095f0 con 0x7f6f58104220 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f5ecc7700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f58106b50 0x7f6f5819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f5dcc5700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f58104220 0x7f6f58196fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e9c0 tx=0x7f6f5400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f4f7fe700 1 -- 172.21.15.45:0/935903194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540097b0 con 0x7f6f58104220 2023-09-23T09:29:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5819ebd0 con 0x7f6f58104220 2023-09-23T09:29:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f4f7fe700 1 -- 172.21.15.45:0/935903194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f54004cf0 con 0x7f6f58104220 2023-09-23T09:29:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f4f7fe700 1 -- 172.21.15.45:0/935903194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54005600 con 0x7f6f58104220 2023-09-23T09:29:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.591+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5819f090 con 0x7f6f58104220 2023-09-23T09:29:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.592+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f580fce00 con 0x7f6f58104220 2023-09-23T09:29:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.593+0000 7f6f4f7fe700 1 -- 172.21.15.45:0/935903194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f6f5400cc60 con 0x7f6f58104220 2023-09-23T09:29:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.594+0000 7f6f4f7fe700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6f440a1630 0x7f6f440a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.594+0000 7f6f5e4c6700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6f440a1630 0x7f6f440a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.594+0000 7f6f4f7fe700 1 -- 172.21.15.45:0/935903194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6f54014070 con 0x7f6f58104220 2023-09-23T09:29:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.594+0000 7f6f5e4c6700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6f440a1630 0x7f6f440a3af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800b2e0 tx=0x7f6f48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:01.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.597+0000 7f6f4f7fe700 1 -- 172.21.15.45:0/935903194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6f540969e0 con 0x7f6f58104220 2023-09-23T09:29:01.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.800+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f5810c7a0 con 0x7f6f58104220 2023-09-23T09:29:01.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.801+0000 7f6f4f7fe700 1 -- 172.21.15.45:0/935903194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f5409a000 con 0x7f6f58104220 2023-09-23T09:29:01.803 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:29:01.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6f440a1630 msgr2=0x7f6f440a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6f440a1630 0x7f6f440a3af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800b2e0 tx=0x7f6f48005fb0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f58104220 msgr2=0x7f6f58196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f58104220 0x7f6f58196fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e9c0 tx=0x7f6f5400ecd0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 shutdown_connections 2023-09-23T09:29:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f58106b50 0x7f6f5819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6f440a1630 0x7f6f440a3af0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f581018f0 0x7f6f58196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 --2- 172.21.15.45:0/935903194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f58104220 0x7f6f58196fe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 >> 172.21.15.45:0/935903194 conn(0x7f6f580fb280 msgr2=0x7f6f58102580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 shutdown_connections 2023-09-23T09:29:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.803+0000 7f6f6072a700 1 -- 172.21.15.45:0/935903194 wait complete. 2023-09-23T09:29:01.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:29:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/2058801436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4101f60 msgr2=0x7fc1a41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/2058801436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4101f60 0x7fc1a41023e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009990 tx=0x7fc18c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/2058801436 shutdown_connections 2023-09-23T09:29:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/2058801436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a4102920 0x7fc1a4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/2058801436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a4100e10 0x7fc1a4101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/2058801436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4101f60 0x7fc1a41023e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/2058801436 >> 172.21.15.45:0/2058801436 conn(0x7fc1a40fc3a0 msgr2=0x7fc1a40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/2058801436 shutdown_connections 2023-09-23T09:29:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.944+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/2058801436 wait complete. 2023-09-23T09:29:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 Processor -- start 2023-09-23T09:29:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 -- start start 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a4100e10 0x7fc1a4074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a4101f60 0x7fc1a4073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4102920 0x7fc1a4073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a4073ef0 con 0x7fc1a4102920 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a4074060 con 0x7fc1a4101f60 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a8d1f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a41a0be0 con 0x7fc1a4100e10 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4102920 0x7fc1a4073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4102920 0x7fc1a4073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59860/0 (socket says 172.21.15.45:59860) 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 -- 172.21.15.45:0/3880230032 learned_addr learned my addr 172.21.15.45:0/3880230032 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a1d9b700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a4101f60 0x7fc1a4073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a259c700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a4100e10 0x7fc1a4074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 -- 172.21.15.45:0/3880230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a4100e10 msgr2=0x7fc1a4074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a4100e10 0x7fc1a4074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 -- 172.21.15.45:0/3880230032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a4101f60 msgr2=0x7fc1a4073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a4101f60 0x7fc1a4073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a2d9d700 1 -- 172.21.15.45:0/3880230032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0095f0 con 0x7fc1a4102920 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a1d9b700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a4101f60 0x7fc1a4073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.945+0000 7fc1a259c700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a4100e10 0x7fc1a4074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.946+0000 7fc1a2d9d700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4102920 0x7fc1a4073750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b970 tx=0x7fc19c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.946+0000 7fc19b7fe700 1 -- 172.21.15.45:0/3880230032 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00c9b0 con 0x7fc1a4102920 2023-09-23T09:29:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.946+0000 7fc19b7fe700 1 -- 172.21.15.45:0/3880230032 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c014440 con 0x7fc1a4102920 2023-09-23T09:29:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.946+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a41a0ec0 con 0x7fc1a4102920 2023-09-23T09:29:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.946+0000 7fc19b7fe700 1 -- 172.21.15.45:0/3880230032 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0075d0 con 0x7fc1a4102920 2023-09-23T09:29:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.946+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a41a1370 con 0x7fc1a4102920 2023-09-23T09:29:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.947+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a410c6e0 con 0x7fc1a4102920 2023-09-23T09:29:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.948+0000 7fc19b7fe700 1 -- 172.21.15.45:0/3880230032 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fc19c0145b0 con 0x7fc1a4102920 2023-09-23T09:29:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.949+0000 7fc19b7fe700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc1900a1630 0x7fc1900a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.949+0000 7fc1a259c700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc1900a1630 0x7fc1900a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.949+0000 7fc19b7fe700 1 -- 172.21.15.45:0/3880230032 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc19c006e40 con 0x7fc1a4102920 2023-09-23T09:29:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.949+0000 7fc1a259c700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc1900a1630 0x7fc1900a3af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc194005fd0 tx=0x7fc194005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:01.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:01.953+0000 7fc19b7fe700 1 -- 172.21.15.45:0/3880230032 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc19c04eec0 con 0x7fc1a4102920 2023-09-23T09:29:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.156+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1a4107d40 con 0x7fc1a4102920 2023-09-23T09:29:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.157+0000 7fc19b7fe700 1 -- 172.21.15.45:0/3880230032 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc19c095ca0 con 0x7fc1a4102920 2023-09-23T09:29:02.158 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:29:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc1900a1630 msgr2=0x7fc1900a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc1900a1630 0x7fc1900a3af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc194005fd0 tx=0x7fc194005ee0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4102920 msgr2=0x7fc1a4073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4102920 0x7fc1a4073750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b970 tx=0x7fc19c00bd30 comp rx=0 tx=0).stop 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 shutdown_connections 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a4102920 0x7fc1a4073750 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc1900a1630 0x7fc1900a3af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a4100e10 0x7fc1a4074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 --2- 172.21.15.45:0/3880230032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a4101f60 0x7fc1a4073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 >> 172.21.15.45:0/3880230032 conn(0x7fc1a40fc3a0 msgr2=0x7fc1a40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 shutdown_connections 2023-09-23T09:29:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.159+0000 7fc1a8d1f700 1 -- 172.21.15.45:0/3880230032 wait complete. 2023-09-23T09:29:02.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:29:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 -- 172.21.15.45:0/4014426955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102090 msgr2=0x7ff4241024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 --2- 172.21.15.45:0/4014426955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102090 0x7ff4241024f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff414009990 tx=0x7ff414009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 -- 172.21.15.45:0/4014426955 shutdown_connections 2023-09-23T09:29:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 --2- 172.21.15.45:0/4014426955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff424102a30 0x7ff4241092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 --2- 172.21.15.45:0/4014426955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4240ff0c0 0x7ff4240ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 --2- 172.21.15.45:0/4014426955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102090 0x7ff4241024f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 -- 172.21.15.45:0/4014426955 >> 172.21.15.45:0/4014426955 conn(0x7ff4240fa9f0 msgr2=0x7ff4240fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 -- 172.21.15.45:0/4014426955 shutdown_connections 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.302+0000 7ff42b6de700 1 -- 172.21.15.45:0/4014426955 wait complete. 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 Processor -- start 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 -- start start 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4240ff0c0 0x7ff424196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff424102090 0x7ff4241970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102a30 0x7ff42419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419e6e0 con 0x7ff424102a30 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419e820 con 0x7ff4240ff0c0 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42b6de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419e9b0 con 0x7ff424102090 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff429c7b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102a30 0x7ff42419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff429c7b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102a30 0x7ff42419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59872/0 (socket says 172.21.15.45:59872) 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff429c7b700 1 -- 172.21.15.45:0/1490218991 learned_addr learned my addr 172.21.15.45:0/1490218991 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff42947a700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4240ff0c0 0x7ff424196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.303+0000 7ff428c79700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff424102090 0x7ff4241970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff429c7b700 1 -- 172.21.15.45:0/1490218991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff424102090 msgr2=0x7ff4241970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff429c7b700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff424102090 0x7ff4241970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff429c7b700 1 -- 172.21.15.45:0/1490218991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4240ff0c0 msgr2=0x7ff424196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff429c7b700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4240ff0c0 0x7ff424196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff429c7b700 1 -- 172.21.15.45:0/1490218991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4140095f0 con 0x7ff424102a30 2023-09-23T09:29:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff429c7b700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102a30 0x7ff42419e110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00b970 tx=0x7ff41c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff41a7fc700 1 -- 172.21.15.45:0/1490218991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00c9b0 con 0x7ff424102a30 2023-09-23T09:29:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42419ec90 con 0x7ff424102a30 2023-09-23T09:29:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff41a7fc700 1 -- 172.21.15.45:0/1490218991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff41c014440 con 0x7ff424102a30 2023-09-23T09:29:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff41a7fc700 1 -- 172.21.15.45:0/1490218991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0075d0 con 0x7ff424102a30 2023-09-23T09:29:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.304+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42419f150 con 0x7ff424102a30 2023-09-23T09:29:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.305+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff424107e20 con 0x7ff424102a30 2023-09-23T09:29:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.306+0000 7ff41a7fc700 1 -- 172.21.15.45:0/1490218991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7ff41c0145b0 con 0x7ff424102a30 2023-09-23T09:29:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.307+0000 7ff41a7fc700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff4100a1630 0x7ff4100a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.307+0000 7ff42947a700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff4100a1630 0x7ff4100a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.307+0000 7ff41a7fc700 1 -- 172.21.15.45:0/1490218991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff41c006e40 con 0x7ff424102a30 2023-09-23T09:29:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.307+0000 7ff42947a700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff4100a1630 0x7ff4100a3af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff420005fd0 tx=0x7ff420005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.311+0000 7ff41a7fc700 1 -- 172.21.15.45:0/1490218991 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff41c0960e0 con 0x7ff424102a30 2023-09-23T09:29:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.511+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff42410c890 con 0x7ff424102a30 2023-09-23T09:29:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.511+0000 7ff41a7fc700 1 -- 172.21.15.45:0/1490218991 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff41c023020 con 0x7ff424102a30 2023-09-23T09:29:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff4100a1630 msgr2=0x7ff4100a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff4100a1630 0x7ff4100a3af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff420005fd0 tx=0x7ff420005dd0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102a30 msgr2=0x7ff42419e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102a30 0x7ff42419e110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00b970 tx=0x7ff41c00bd30 comp rx=0 tx=0).stop 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 shutdown_connections 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff424102a30 0x7ff42419e110 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4240ff0c0 0x7ff424196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff4100a1630 0x7ff4100a3af0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 --2- 172.21.15.45:0/1490218991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff424102090 0x7ff4241970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 >> 172.21.15.45:0/1490218991 conn(0x7ff4240fa9f0 msgr2=0x7ff4240fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 shutdown_connections 2023-09-23T09:29:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.514+0000 7ff42b6de700 1 -- 172.21.15.45:0/1490218991 wait complete. 2023-09-23T09:29:02.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:02 smithi045 ceph-mon[98627]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-09-23T09:29:02.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:02 smithi045 ceph-mon[98627]: Cluster is now healthy 2023-09-23T09:29:02.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:02 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3782305955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:02.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:02 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/935903194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:02.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:02 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3880230032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:02.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:29:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:02 smithi098 ceph-mon[102386]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-09-23T09:29:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:02 smithi098 ceph-mon[102386]: Cluster is now healthy 2023-09-23T09:29:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:02 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3782305955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:02 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/935903194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:02.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:02 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3880230032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:02 smithi123 ceph-mon[103577]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-09-23T09:29:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:02 smithi123 ceph-mon[103577]: Cluster is now healthy 2023-09-23T09:29:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:02 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3782305955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:02 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/935903194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:02.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:02 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3880230032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:02.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 -- 172.21.15.45:0/83895916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 msgr2=0x7ffacc101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 --2- 172.21.15.45:0/83895916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc101350 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffac00099c0 tx=0x7ffac0009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 -- 172.21.15.45:0/83895916 shutdown_connections 2023-09-23T09:29:02.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 --2- 172.21.15.45:0/83895916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffacc101890 0x7ffacc108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 --2- 172.21.15.45:0/83895916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffacc0ffd00 0x7ffacc100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 --2- 172.21.15.45:0/83895916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc101350 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 -- 172.21.15.45:0/83895916 >> 172.21.15.45:0/83895916 conn(0x7ffacc0fb250 msgr2=0x7ffacc0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:02.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 -- 172.21.15.45:0/83895916 shutdown_connections 2023-09-23T09:29:02.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.652+0000 7ffad3959700 1 -- 172.21.15.45:0/83895916 wait complete. 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.653+0000 7ffad3959700 1 Processor -- start 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.653+0000 7ffad3959700 1 -- start start 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffacc0ffd00 0x7ffacc1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffacc101890 0x7ffacc19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc19d570 con 0x7ffacc100ef0 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc19d6b0 con 0x7ffacc101890 2023-09-23T09:29:02.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc19d840 con 0x7ffacc0ffd00 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59886/0 (socket says 172.21.15.45:59886) 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 -- 172.21.15.45:0/2279667569 learned_addr learned my addr 172.21.15.45:0/2279667569 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad16f5700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffacc0ffd00 0x7ffacc1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad1ef6700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffacc101890 0x7ffacc19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 -- 172.21.15.45:0/2279667569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffacc0ffd00 msgr2=0x7ffacc1959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffacc0ffd00 0x7ffacc1959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 -- 172.21.15.45:0/2279667569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffacc101890 msgr2=0x7ffacc19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffacc101890 0x7ffacc19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 -- 172.21.15.45:0/2279667569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00095f0 con 0x7ffacc100ef0 2023-09-23T09:29:02.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad0ef4700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc195f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009990 tx=0x7ffac000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:02.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffabe7fc700 1 -- 172.21.15.45:0/2279667569 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac001d070 con 0x7ffacc100ef0 2023-09-23T09:29:02.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffabe7fc700 1 -- 172.21.15.45:0/2279667569 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac00054f0 con 0x7ffacc100ef0 2023-09-23T09:29:02.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffabe7fc700 1 -- 172.21.15.45:0/2279667569 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004bd0 con 0x7ffacc100ef0 2023-09-23T09:29:02.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc19dac0 con 0x7ffacc100ef0 2023-09-23T09:29:02.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.654+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc19de90 con 0x7ffacc100ef0 2023-09-23T09:29:02.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.655+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc1072e0 con 0x7ffacc100ef0 2023-09-23T09:29:02.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.656+0000 7ffabe7fc700 1 -- 172.21.15.45:0/2279667569 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7ffac000fb80 con 0x7ffacc100ef0 2023-09-23T09:29:02.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.657+0000 7ffabe7fc700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffab80a1670 0x7ffab80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:02.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.657+0000 7ffad16f5700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffab80a1670 0x7ffab80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:02.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.657+0000 7ffabe7fc700 1 -- 172.21.15.45:0/2279667569 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffac00ce370 con 0x7ffacc100ef0 2023-09-23T09:29:02.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.657+0000 7ffad16f5700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffab80a1670 0x7ffab80a3b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffacc100d50 tx=0x7ffac800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:02.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.660+0000 7ffabe7fc700 1 -- 172.21.15.45:0/2279667569 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ffac0097890 con 0x7ffacc100ef0 2023-09-23T09:29:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.907+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffacc10b7b0 con 0x7ffacc100ef0 2023-09-23T09:29:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.907+0000 7ffabe7fc700 1 -- 172.21.15.45:0/2279667569 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7ffac009aeb0 con 0x7ffacc100ef0 2023-09-23T09:29:02.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:02.909 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:29:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffab80a1670 msgr2=0x7ffab80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffab80a1670 0x7ffab80a3b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffacc100d50 tx=0x7ffac800b410 comp rx=0 tx=0).stop 2023-09-23T09:29:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 msgr2=0x7ffacc195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc195f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009990 tx=0x7ffac000f690 comp rx=0 tx=0).stop 2023-09-23T09:29:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 shutdown_connections 2023-09-23T09:29:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffacc101890 0x7ffacc19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ffab80a1670 0x7ffab80a3b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffacc0ffd00 0x7ffacc1959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 --2- 172.21.15.45:0/2279667569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffacc100ef0 0x7ffacc195f30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 >> 172.21.15.45:0/2279667569 conn(0x7ffacc0fb250 msgr2=0x7ffacc104420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 shutdown_connections 2023-09-23T09:29:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:02.910+0000 7ffad3959700 1 -- 172.21.15.45:0/2279667569 wait complete. 2023-09-23T09:29:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:29:02.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:29:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 -- 172.21.15.45:0/72384342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20102930 msgr2=0x7f3d20109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 --2- 172.21.15.45:0/72384342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20102930 0x7f3d20109250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b0a0 tx=0x7f3d1800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 -- 172.21.15.45:0/72384342 shutdown_connections 2023-09-23T09:29:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 --2- 172.21.15.45:0/72384342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20102930 0x7f3d20109250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 --2- 172.21.15.45:0/72384342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d20100e20 0x7f3d20101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 --2- 172.21.15.45:0/72384342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d20101f70 0x7f3d201023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 -- 172.21.15.45:0/72384342 >> 172.21.15.45:0/72384342 conn(0x7f3d200fc370 msgr2=0x7f3d200fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 -- 172.21.15.45:0/72384342 shutdown_connections 2023-09-23T09:29:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.047+0000 7f3d27615700 1 -- 172.21.15.45:0/72384342 wait complete. 2023-09-23T09:29:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 Processor -- start 2023-09-23T09:29:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 -- start start 2023-09-23T09:29:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d20100e20 0x7f3d20196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20101f70 0x7f3d201970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d20102930 0x7f3d2019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2019e720 con 0x7f3d20101f70 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2019e860 con 0x7f3d20100e20 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.048+0000 7f3d27615700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2019e9f0 con 0x7f3d20102930 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20101f70 0x7f3d201970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d25bb2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d20102930 0x7f3d2019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d253b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d20100e20 0x7f3d20196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d25bb2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d20102930 0x7f3d2019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:33776/0 (socket says 172.21.15.45:33776) 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d25bb2700 1 -- 172.21.15.45:0/651074423 learned_addr learned my addr 172.21.15.45:0/651074423 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d253b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d20100e20 0x7f3d20196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:53016/0 (socket says 172.21.15.45:53016) 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20101f70 0x7f3d201970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59900/0 (socket says 172.21.15.45:59900) 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 -- 172.21.15.45:0/651074423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d20102930 msgr2=0x7f3d2019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d20102930 0x7f3d2019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 -- 172.21.15.45:0/651074423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d20100e20 msgr2=0x7f3d20196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d20100e20 0x7f3d20196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 -- 172.21.15.45:0/651074423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d18009d00 con 0x7f3d20101f70 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d24bb0700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20101f70 0x7f3d201970e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c970 tx=0x7f3d1400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d127fc700 1 -- 172.21.15.45:0/651074423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14004a30 con 0x7f3d20101f70 2023-09-23T09:29:03.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d127fc700 1 -- 172.21.15.45:0/651074423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d140054e0 con 0x7f3d20101f70 2023-09-23T09:29:03.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d127fc700 1 -- 172.21.15.45:0/651074423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1400eb40 con 0x7f3d20101f70 2023-09-23T09:29:03.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2019ecd0 con 0x7f3d20101f70 2023-09-23T09:29:03.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.049+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2019f190 con 0x7f3d20101f70 2023-09-23T09:29:03.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.050+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2010c620 con 0x7f3d20101f70 2023-09-23T09:29:03.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.051+0000 7f3d127fc700 1 -- 172.21.15.45:0/651074423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f3d1400eca0 con 0x7f3d20101f70 2023-09-23T09:29:03.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.052+0000 7f3d127fc700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3d0c0a16f0 0x7f3d0c0a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.052+0000 7f3d253b1700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3d0c0a16f0 0x7f3d0c0a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.052+0000 7f3d127fc700 1 -- 172.21.15.45:0/651074423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3d140ccd80 con 0x7f3d20101f70 2023-09-23T09:29:03.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.052+0000 7f3d253b1700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3d0c0a16f0 0x7f3d0c0a3bb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c005950 tx=0x7f3d1c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.055+0000 7f3d127fc700 1 -- 172.21.15.45:0/651074423 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3d14096290 con 0x7f3d20101f70 2023-09-23T09:29:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.260+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d2019f3e0 con 0x7f3d20101f70 2023-09-23T09:29:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.262+0000 7f3d127fc700 1 -- 172.21.15.45:0/651074423 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f3d1401d070 con 0x7f3d20101f70 2023-09-23T09:29:03.264 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:03.264 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:28:55.849323+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:29:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3d0c0a16f0 msgr2=0x7f3d0c0a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3d0c0a16f0 0x7f3d0c0a3bb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c005950 tx=0x7f3d1c00a300 comp rx=0 tx=0).stop 2023-09-23T09:29:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20101f70 msgr2=0x7f3d201970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20101f70 0x7f3d201970e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c970 tx=0x7f3d1400cc80 comp rx=0 tx=0).stop 2023-09-23T09:29:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 shutdown_connections 2023-09-23T09:29:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3d0c0a16f0 0x7f3d0c0a3bb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d20102930 0x7f3d2019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d20100e20 0x7f3d20196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 --2- 172.21.15.45:0/651074423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d20101f70 0x7f3d201970e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 >> 172.21.15.45:0/651074423 conn(0x7f3d200fc370 msgr2=0x7f3d200fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 shutdown_connections 2023-09-23T09:29:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.264+0000 7f3d27615700 1 -- 172.21.15.45:0/651074423 wait complete. 2023-09-23T09:29:03.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:29:03.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:29:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 -- 172.21.15.45:0/3249945403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 msgr2=0x7f5b9c1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 --2- 172.21.15.45:0/3249945403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c1078c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400c240 tx=0x7f5b9400c550 comp rx=0 tx=0).stop 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 -- 172.21.15.45:0/3249945403 shutdown_connections 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 --2- 172.21.15.45:0/3249945403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c1078c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 --2- 172.21.15.45:0/3249945403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c0ff480 0x7f5b9c0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 --2- 172.21.15.45:0/3249945403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b9c100670 0x7f5b9c100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 -- 172.21.15.45:0/3249945403 >> 172.21.15.45:0/3249945403 conn(0x7f5b9c0fa9f0 msgr2=0x7f5b9c0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 -- 172.21.15.45:0/3249945403 shutdown_connections 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 -- 172.21.15.45:0/3249945403 wait complete. 2023-09-23T09:29:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 Processor -- start 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 -- start start 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b9c0ff480 0x7f5b9c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c100670 0x7f5b9c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.407+0000 7f5ba2418700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba2418700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c19e720 con 0x7f5b9c101010 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba2418700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c19e860 con 0x7f5b9c0ff480 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba2418700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c19e9f0 con 0x7f5b9c100670 2023-09-23T09:29:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c19e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59916/0 (socket says 172.21.15.45:59916) 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 -- 172.21.15.45:0/2326296326 learned_addr learned my addr 172.21.15.45:0/2326296326 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5b93fff700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c100670 0x7f5b9c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5b9bfff700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b9c0ff480 0x7f5b9c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 -- 172.21.15.45:0/2326296326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c100670 msgr2=0x7f5b9c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c100670 0x7f5b9c1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 -- 172.21.15.45:0/2326296326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b9c0ff480 msgr2=0x7f5b9c196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b9c0ff480 0x7f5b9c196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 -- 172.21.15.45:0/2326296326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9400ae10 con 0x7f5b9c101010 2023-09-23T09:29:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba09b5700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c19e150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b940062a0 tx=0x7f5b94008340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5b99ffb700 1 -- 172.21.15.45:0/2326296326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400f040 con 0x7f5b9c101010 2023-09-23T09:29:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c19ec70 con 0x7f5b9c101010 2023-09-23T09:29:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5b99ffb700 1 -- 172.21.15.45:0/2326296326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9400a560 con 0x7f5b9c101010 2023-09-23T09:29:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5b99ffb700 1 -- 172.21.15.45:0/2326296326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9401e870 con 0x7f5b9c101010 2023-09-23T09:29:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.408+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c19f0a0 con 0x7f5b9c101010 2023-09-23T09:29:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.409+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c106380 con 0x7f5b9c101010 2023-09-23T09:29:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.411+0000 7f5b99ffb700 1 -- 172.21.15.45:0/2326296326 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f5b94007af0 con 0x7f5b9c101010 2023-09-23T09:29:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.411+0000 7f5b99ffb700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5b880a1630 0x7f5b880a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.412+0000 7f5b9bfff700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5b880a1630 0x7f5b880a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.412+0000 7f5b99ffb700 1 -- 172.21.15.45:0/2326296326 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5b940cee50 con 0x7f5b9c101010 2023-09-23T09:29:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.412+0000 7f5b9bfff700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5b880a1630 0x7f5b880a3af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005950 tx=0x7f5b8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.415+0000 7f5b99ffb700 1 -- 172.21.15.45:0/2326296326 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5b94098370 con 0x7f5b9c101010 2023-09-23T09:29:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:03 smithi098 ceph-mon[102386]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:03 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1490218991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:03 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2279667569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:03.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:03 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/651074423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:03 smithi045 ceph-mon[98627]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:03 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1490218991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:03 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2279667569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:03 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/651074423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:03 smithi123 ceph-mon[103577]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:03 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1490218991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:03 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2279667569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:03.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:03 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/651074423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:03.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.619+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b9c10a630 con 0x7f5b9c101010 2023-09-23T09:29:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.619+0000 7f5b99ffb700 1 -- 172.21.15.45:0/2326296326 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b9409b990 con 0x7f5b9c101010 2023-09-23T09:29:03.621 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:29:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5b880a1630 msgr2=0x7f5b880a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5b880a1630 0x7f5b880a3af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005950 tx=0x7f5b8c0058e0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 msgr2=0x7f5b9c19e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c19e150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b940062a0 tx=0x7f5b94008340 comp rx=0 tx=0).stop 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 shutdown_connections 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c101010 0x7f5b9c19e150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b9c0ff480 0x7f5b9c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5b880a1630 0x7f5b880a3af0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 --2- 172.21.15.45:0/2326296326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c100670 0x7f5b9c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.621+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 >> 172.21.15.45:0/2326296326 conn(0x7f5b9c0fa9f0 msgr2=0x7f5b9c0fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.622+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 shutdown_connections 2023-09-23T09:29:03.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.622+0000 7f5ba2418700 1 -- 172.21.15.45:0/2326296326 wait complete. 2023-09-23T09:29:03.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:29:03.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 -- 172.21.15.45:0/4031996727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c106b70 msgr2=0x7f070c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 --2- 172.21.15.45:0/4031996727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c106b70 0x7f070c108f60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f070400b0a0 tx=0x7f070400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 -- 172.21.15.45:0/4031996727 shutdown_connections 2023-09-23T09:29:03.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 --2- 172.21.15.45:0/4031996727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c106b70 0x7f070c108f60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 --2- 172.21.15.45:0/4031996727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f070c101910 0x7f070c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 --2- 172.21.15.45:0/4031996727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070c104240 0x7f070c106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 -- 172.21.15.45:0/4031996727 >> 172.21.15.45:0/4031996727 conn(0x7f070c0fb280 msgr2=0x7f070c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:03.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 -- 172.21.15.45:0/4031996727 shutdown_connections 2023-09-23T09:29:03.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.762+0000 7f0711a87700 1 -- 172.21.15.45:0/4031996727 wait complete. 2023-09-23T09:29:03.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 Processor -- start 2023-09-23T09:29:03.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 -- start start 2023-09-23T09:29:03.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c101910 0x7f070c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f070c104240 0x7f070c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070c106b70 0x7f070c19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c19e790 con 0x7f070c101910 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c19e900 con 0x7f070c106b70 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f0711a87700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c19eac0 con 0x7f070c104240 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c101910 0x7f070c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c101910 0x7f070c196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59926/0 (socket says 172.21.15.45:59926) 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 -- 172.21.15.45:0/2038631681 learned_addr learned my addr 172.21.15.45:0/2038631681 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070affd700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f070c104240 0x7f070c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070bfff700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070c106b70 0x7f070c19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 -- 172.21.15.45:0/2038631681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f070c104240 msgr2=0x7f070c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f070c104240 0x7f070c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 -- 172.21.15.45:0/2038631681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070c106b70 msgr2=0x7f070c19e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070c106b70 0x7f070c19e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.763+0000 7f070b7fe700 1 -- 172.21.15.45:0/2038631681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0704009d00 con 0x7f070c101910 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.764+0000 7f070b7fe700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c101910 0x7f070c196c50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c920 tx=0x7f06fc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.764+0000 7f0708ff9700 1 -- 172.21.15.45:0/2038631681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc004a30 con 0x7f070c101910 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.764+0000 7f0708ff9700 1 -- 172.21.15.45:0/2038631681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06fc0054e0 con 0x7f070c101910 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.764+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c19eda0 con 0x7f070c101910 2023-09-23T09:29:03.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.764+0000 7f0708ff9700 1 -- 172.21.15.45:0/2038631681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc007bd0 con 0x7f070c101910 2023-09-23T09:29:03.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.764+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c19f1f0 con 0x7f070c101910 2023-09-23T09:29:03.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.765+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c0fce00 con 0x7f070c101910 2023-09-23T09:29:03.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.767+0000 7f0708ff9700 1 -- 172.21.15.45:0/2038631681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f06fc005000 con 0x7f070c101910 2023-09-23T09:29:03.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.767+0000 7f0708ff9700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06f80a1630 0x7f06f80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:03.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.767+0000 7f0708ff9700 1 -- 172.21.15.45:0/2038631681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06fc0ccfe0 con 0x7f070c101910 2023-09-23T09:29:03.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.767+0000 7f070affd700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06f80a1630 0x7f06f80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:03.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.768+0000 7f070affd700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06f80a1630 0x7f06f80a3af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f06f4005950 tx=0x7f06f400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:03.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.771+0000 7f0708ff9700 1 -- 172.21.15.45:0/2038631681 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f06fc096470 con 0x7f070c101910 2023-09-23T09:29:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.972+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f070c10c8b0 con 0x7f070c101910 2023-09-23T09:29:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.972+0000 7f0708ff9700 1 -- 172.21.15.45:0/2038631681 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06fc01d0a0 con 0x7f070c101910 2023-09-23T09:29:03.974 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:29:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06f80a1630 msgr2=0x7f06f80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06f80a1630 0x7f06f80a3af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f06f4005950 tx=0x7f06f400a300 comp rx=0 tx=0).stop 2023-09-23T09:29:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c101910 msgr2=0x7f070c196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c101910 0x7f070c196c50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c920 tx=0x7f06fc00cc30 comp rx=0 tx=0).stop 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 shutdown_connections 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070c106b70 0x7f070c19e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f070c101910 0x7f070c196c50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06f80a1630 0x7f06f80a3af0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 --2- 172.21.15.45:0/2038631681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f070c104240 0x7f070c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 >> 172.21.15.45:0/2038631681 conn(0x7f070c0fb280 msgr2=0x7f070c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 shutdown_connections 2023-09-23T09:29:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:03.974+0000 7f0711a87700 1 -- 172.21.15.45:0/2038631681 wait complete. 2023-09-23T09:29:03.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:29:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 -- 172.21.15.45:0/2806867968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 msgr2=0x7f8944106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 --2- 172.21.15.45:0/2806867968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944106630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f892c009990 tx=0x7f892c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 -- 172.21.15.45:0/2806867968 shutdown_connections 2023-09-23T09:29:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 --2- 172.21.15.45:0/2806867968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8944106b70 0x7f8944108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 --2- 172.21.15.45:0/2806867968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8944101910 0x7f8944103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 --2- 172.21.15.45:0/2806867968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944106630 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 -- 172.21.15.45:0/2806867968 >> 172.21.15.45:0/2806867968 conn(0x7f89440fb2e0 msgr2=0x7f89440fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 -- 172.21.15.45:0/2806867968 shutdown_connections 2023-09-23T09:29:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.115+0000 7f8949e62700 1 -- 172.21.15.45:0/2806867968 wait complete. 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 Processor -- start 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 -- start start 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8944101910 0x7f8944196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8944106b70 0x7f894419e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894419e660 con 0x7f8944104240 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894419e7a0 con 0x7f8944106b70 2023-09-23T09:29:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8949e62700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894419e930 con 0x7f8944101910 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8942ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8942ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59942/0 (socket says 172.21.15.45:59942) 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8942ffd700 1 -- 172.21.15.45:0/589947338 learned_addr learned my addr 172.21.15.45:0/589947338 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f89437fe700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8944101910 0x7f8944196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.116+0000 7f8943fff700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8944106b70 0x7f894419e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8942ffd700 1 -- 172.21.15.45:0/589947338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8944101910 msgr2=0x7f8944196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8942ffd700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8944101910 0x7f8944196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8942ffd700 1 -- 172.21.15.45:0/589947338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8944106b70 msgr2=0x7f894419e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8942ffd700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8944106b70 0x7f894419e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8942ffd700 1 -- 172.21.15.45:0/589947338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0095f0 con 0x7f8944104240 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8942ffd700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944197020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f892c000c00 tx=0x7f892c00f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8940ff9700 1 -- 172.21.15.45:0/589947338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c01d070 con 0x7f8944104240 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8940ff9700 1 -- 172.21.15.45:0/589947338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892c00fb80 con 0x7f8944104240 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8940ff9700 1 -- 172.21.15.45:0/589947338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c017650 con 0x7f8944104240 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894419ebb0 con 0x7f8944104240 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.117+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894419f010 con 0x7f8944104240 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.118+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89440fce60 con 0x7f8944104240 2023-09-23T09:29:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.119+0000 7f8940ff9700 1 -- 172.21.15.45:0/589947338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f892c022470 con 0x7f8944104240 2023-09-23T09:29:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.120+0000 7f8940ff9700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89300a1630 0x7f89300a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.120+0000 7f8940ff9700 1 -- 172.21.15.45:0/589947338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f892c0ce640 con 0x7f8944104240 2023-09-23T09:29:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.120+0000 7f89437fe700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89300a1630 0x7f89300a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.120+0000 7f89437fe700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89300a1630 0x7f89300a3af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8934005950 tx=0x7f893400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:04.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.124+0000 7f8940ff9700 1 -- 172.21.15.45:0/589947338 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f892c097c40 con 0x7f8944104240 2023-09-23T09:29:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.325+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f894410c830 con 0x7f8944104240 2023-09-23T09:29:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.325+0000 7f8940ff9700 1 -- 172.21.15.45:0/589947338 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f892c0270a0 con 0x7f8944104240 2023-09-23T09:29:04.327 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:29:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89300a1630 msgr2=0x7f89300a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89300a1630 0x7f89300a3af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8934005950 tx=0x7f893400a300 comp rx=0 tx=0).stop 2023-09-23T09:29:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 msgr2=0x7f8944197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944197020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f892c000c00 tx=0x7f892c00f780 comp rx=0 tx=0).stop 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 shutdown_connections 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8944106b70 0x7f894419e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f89300a1630 0x7f89300a3af0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8944101910 0x7f8944196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 --2- 172.21.15.45:0/589947338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8944104240 0x7f8944197020 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 >> 172.21.15.45:0/589947338 conn(0x7f89440fb2e0 msgr2=0x7f8944102580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 shutdown_connections 2023-09-23T09:29:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.327+0000 7f8949e62700 1 -- 172.21.15.45:0/589947338 wait complete. 2023-09-23T09:29:04.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 -- 172.21.15.45:0/2072182657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78980ffd00 msgr2=0x7f7898100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 --2- 172.21.15.45:0/2072182657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78980ffd00 0x7f7898100120 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f78880099c0 tx=0x7f7888009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 -- 172.21.15.45:0/2072182657 shutdown_connections 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 --2- 172.21.15.45:0/2072182657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898101890 0x7f7898108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 --2- 172.21.15.45:0/2072182657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78980ffd00 0x7f7898100120 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 --2- 172.21.15.45:0/2072182657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7898100ef0 0x7f7898101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 -- 172.21.15.45:0/2072182657 >> 172.21.15.45:0/2072182657 conn(0x7f78980fb250 msgr2=0x7f78980fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 -- 172.21.15.45:0/2072182657 shutdown_connections 2023-09-23T09:29:04.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.466+0000 7f789cec9700 1 -- 172.21.15.45:0/2072182657 wait complete. 2023-09-23T09:29:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 Processor -- start 2023-09-23T09:29:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 -- start start 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980ffd00 0x7f7898068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7898100ef0 0x7f789819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7898101890 0x7f789819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789819e610 con 0x7f7898101890 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789819e780 con 0x7f78980ffd00 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789cec9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789819e940 con 0x7f7898100ef0 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980ffd00 0x7f7898068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f7896d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7898101890 0x7f789819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980ffd00 0x7f7898068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:53072/0 (socket says 172.21.15.45:53072) 2023-09-23T09:29:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f789659c700 1 -- 172.21.15.45:0/1456497247 learned_addr learned my addr 172.21.15.45:0/1456497247 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f7895d9b700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7898100ef0 0x7f789819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f7896d9d700 1 -- 172.21.15.45:0/1456497247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7898100ef0 msgr2=0x7f789819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f7896d9d700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7898100ef0 0x7f789819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f7896d9d700 1 -- 172.21.15.45:0/1456497247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980ffd00 msgr2=0x7f7898068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f7896d9d700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980ffd00 0x7f7898068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.467+0000 7f7896d9d700 1 -- 172.21.15.45:0/1456497247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880095f0 con 0x7f7898101890 2023-09-23T09:29:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.468+0000 7f7896d9d700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7898101890 0x7f789819de40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f789000ab40 tx=0x7f789000ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.468+0000 7f78877fe700 1 -- 172.21.15.45:0/1456497247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000f070 con 0x7f7898101890 2023-09-23T09:29:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.468+0000 7f78877fe700 1 -- 172.21.15.45:0/1456497247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7890007a70 con 0x7f7898101890 2023-09-23T09:29:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.468+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789819ec20 con 0x7f7898101890 2023-09-23T09:29:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.468+0000 7f78877fe700 1 -- 172.21.15.45:0/1456497247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890005280 con 0x7f7898101890 2023-09-23T09:29:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.468+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789819f130 con 0x7f7898101890 2023-09-23T09:29:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.469+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78981072b0 con 0x7f7898101890 2023-09-23T09:29:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.470+0000 7f78877fe700 1 -- 172.21.15.45:0/1456497247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f7890004ad0 con 0x7f7898101890 2023-09-23T09:29:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.470+0000 7f78877fe700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f78800a1670 0x7f78800a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.471+0000 7f789659c700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f78800a1670 0x7f78800a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.471+0000 7f78877fe700 1 -- 172.21.15.45:0/1456497247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78900cc4d0 con 0x7f7898101890 2023-09-23T09:29:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.471+0000 7f789659c700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f78800a1670 0x7f78800a3b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7888000c00 tx=0x7f7888005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.475+0000 7f78877fe700 1 -- 172.21.15.45:0/1456497247 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f78900959f0 con 0x7f7898101890 2023-09-23T09:29:04.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:04 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2326296326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:04.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:04 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2038631681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:04 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2326296326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:04.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:04 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2038631681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:04.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:04 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2326296326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:04.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:04 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2038631681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.723+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f789802cf30 con 0x7f7898101890 2023-09-23T09:29:04.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.723+0000 7f78877fe700 1 -- 172.21.15.45:0/1456497247 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7f7890099010 con 0x7f7898101890 2023-09-23T09:29:04.725 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:04.725 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:29:04.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f78800a1670 msgr2=0x7f78800a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f78800a1670 0x7f78800a3b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7888000c00 tx=0x7f7888005bc0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7898101890 msgr2=0x7f789819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7898101890 0x7f789819de40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f789000ab40 tx=0x7f789000ae50 comp rx=0 tx=0).stop 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 shutdown_connections 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7898101890 0x7f789819de40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980ffd00 0x7f7898068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f78800a1670 0x7f78800a3b30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 --2- 172.21.15.45:0/1456497247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7898100ef0 0x7f789819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.725+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 >> 172.21.15.45:0/1456497247 conn(0x7f78980fb250 msgr2=0x7f78981043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.726+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 shutdown_connections 2023-09-23T09:29:04.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.726+0000 7f789cec9700 1 -- 172.21.15.45:0/1456497247 wait complete. 2023-09-23T09:29:04.729 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:29:04.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:29:04.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 -- 172.21.15.45:0/1397478886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434100ef0 msgr2=0x7f6434101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 --2- 172.21.15.45:0/1397478886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434100ef0 0x7f6434101350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f64280099c0 tx=0x7f6428009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 -- 172.21.15.45:0/1397478886 shutdown_connections 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 --2- 172.21.15.45:0/1397478886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434101890 0x7f6434108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 --2- 172.21.15.45:0/1397478886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64340ffd00 0x7f6434100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 --2- 172.21.15.45:0/1397478886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434100ef0 0x7f6434101350 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 -- 172.21.15.45:0/1397478886 >> 172.21.15.45:0/1397478886 conn(0x7f64340fb250 msgr2=0x7f64340fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.863+0000 7f643a8af700 1 -- 172.21.15.45:0/1397478886 shutdown_connections 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.864+0000 7f643a8af700 1 -- 172.21.15.45:0/1397478886 wait complete. 2023-09-23T09:29:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.864+0000 7f643a8af700 1 Processor -- start 2023-09-23T09:29:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.864+0000 7f643a8af700 1 -- start start 2023-09-23T09:29:04.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.864+0000 7f643a8af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64340ffd00 0x7f6434196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.864+0000 7f643a8af700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434100ef0 0x7f6434196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.864+0000 7f643a8af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434101890 0x7f643419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.864+0000 7f643a8af700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419e5c0 con 0x7f6434101890 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f643a8af700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419e700 con 0x7f6434100ef0 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f643a8af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419e890 con 0x7f64340ffd00 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434101890 0x7f643419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6433fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64340ffd00 0x7f6434196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434101890 0x7f643419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59970/0 (socket says 172.21.15.45:59970) 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 -- 172.21.15.45:0/2922151638 learned_addr learned my addr 172.21.15.45:0/2922151638 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f64337fe700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434100ef0 0x7f6434196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 -- 172.21.15.45:0/2922151638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64340ffd00 msgr2=0x7f6434196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64340ffd00 0x7f6434196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 -- 172.21.15.45:0/2922151638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434100ef0 msgr2=0x7f6434196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434100ef0 0x7f6434196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 -- 172.21.15.45:0/2922151638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280095f0 con 0x7f6434101890 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f6438e4c700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434101890 0x7f643419dff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f642c007d70 tx=0x7f642c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f64317fa700 1 -- 172.21.15.45:0/2922151638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c00ad40 con 0x7f6434101890 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643419eb70 con 0x7f6434101890 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f64317fa700 1 -- 172.21.15.45:0/2922151638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f642c0128e0 con 0x7f6434101890 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f64317fa700 1 -- 172.21.15.45:0/2922151638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0116b0 con 0x7f6434101890 2023-09-23T09:29:04.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.865+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643419f030 con 0x7f6434101890 2023-09-23T09:29:04.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.866+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6434107d90 con 0x7f6434101890 2023-09-23T09:29:04.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.868+0000 7f64317fa700 1 -- 172.21.15.45:0/2922151638 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f642c012420 con 0x7f6434101890 2023-09-23T09:29:04.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.869+0000 7f64317fa700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f641c0a1670 0x7f641c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:04.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.869+0000 7f6433fff700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f641c0a1670 0x7f641c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:04.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.869+0000 7f64317fa700 1 -- 172.21.15.45:0/2922151638 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f642c0cd620 con 0x7f6434101890 2023-09-23T09:29:04.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.869+0000 7f6433fff700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f641c0a1670 0x7f641c0a3b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6434100d50 tx=0x7f6424009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:04.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:04.872+0000 7f64317fa700 1 -- 172.21.15.45:0/2922151638 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f642c096b80 con 0x7f6434101890 2023-09-23T09:29:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.073+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f643410b530 con 0x7f6434101890 2023-09-23T09:29:05.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.074+0000 7f64317fa700 1 -- 172.21.15.45:0/2922151638 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f642c09a1a0 con 0x7f6434101890 2023-09-23T09:29:05.076 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:05.077 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:28:55.849323+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:29:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.076+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f641c0a1670 msgr2=0x7f641c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.076+0000 7f643a8af700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f641c0a1670 0x7f641c0a3b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6434100d50 tx=0x7f6424009260 comp rx=0 tx=0).stop 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.076+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434101890 msgr2=0x7f643419dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.076+0000 7f643a8af700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434101890 0x7f643419dff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f642c007d70 tx=0x7f642c00a340 comp rx=0 tx=0).stop 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 shutdown_connections 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434101890 0x7f643419dff0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f641c0a1670 0x7f641c0a3b30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64340ffd00 0x7f6434196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 --2- 172.21.15.45:0/2922151638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434100ef0 0x7f6434196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 >> 172.21.15.45:0/2922151638 conn(0x7f64340fb250 msgr2=0x7f64340fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 shutdown_connections 2023-09-23T09:29:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.077+0000 7f643a8af700 1 -- 172.21.15.45:0/2922151638 wait complete. 2023-09-23T09:29:05.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-23T09:27:50.073600+0000', 'modified': '2023-09-23T09:27:55.252601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24530, 'mds_1': 24500, 'mds_2': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/3506316120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 3506316120}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 3506316120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/3236511886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3236511886}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3236511886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/2272642085', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2272642085}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2272642085}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/4054887711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 4054887711}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 4054887711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.98:6837/767375327', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 767375327}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 767375327}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/2348516337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 2348516337}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 2348516337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T09:29:05.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-23T09:29:05.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-23T09:29:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 -- 172.21.15.45:0/3340941845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 msgr2=0x7fc4cc101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/3340941845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc101360 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40099c0 tx=0x7fc4b4009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 -- 172.21.15.45:0/3340941845 shutdown_connections 2023-09-23T09:29:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/3340941845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/3340941845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc0ffd10 0x7fc4cc100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/3340941845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc101360 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 -- 172.21.15.45:0/3340941845 >> 172.21.15.45:0/3340941845 conn(0x7fc4cc0fb280 msgr2=0x7fc4cc0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 -- 172.21.15.45:0/3340941845 shutdown_connections 2023-09-23T09:29:05.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 -- 172.21.15.45:0/3340941845 wait complete. 2023-09-23T09:29:05.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.217+0000 7fc4d16eb700 1 Processor -- start 2023-09-23T09:29:05.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4d16eb700 1 -- start start 2023-09-23T09:29:05.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4d16eb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0ffd10 0x7fc4cc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4d16eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4d16eb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4d16eb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc19e600 con 0x7fc4cc100f00 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4d16eb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc19e770 con 0x7fc4cc0ffd10 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4d16eb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc19e930 con 0x7fc4cc1018a0 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59982/0 (socket says 172.21.15.45:59982) 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 -- 172.21.15.45:0/2584866955 learned_addr learned my addr 172.21.15.45:0/2584866955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4caffd700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0ffd10 0x7fc4cc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 -- 172.21.15.45:0/2584866955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc1018a0 msgr2=0x7fc4cc19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 -- 172.21.15.45:0/2584866955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0ffd10 msgr2=0x7fc4cc068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0ffd10 0x7fc4cc068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 -- 172.21.15.45:0/2584866955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b40095f0 con 0x7fc4cc100f00 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4caffd700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0ffd10 0x7fc4cc068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.218+0000 7fc4ca7fc700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc19b7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009990 tx=0x7fc4b4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.219+0000 7fc4c3fff700 1 -- 172.21.15.45:0/2584866955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b401d070 con 0x7fc4cc100f00 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.219+0000 7fc4c3fff700 1 -- 172.21.15.45:0/2584866955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4b4022470 con 0x7fc4cc100f00 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.219+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc19ebb0 con 0x7fc4cc100f00 2023-09-23T09:29:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.219+0000 7fc4c3fff700 1 -- 172.21.15.45:0/2584866955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4004230 con 0x7fc4cc100f00 2023-09-23T09:29:05.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.219+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc19f060 con 0x7fc4cc100f00 2023-09-23T09:29:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.221+0000 7fc4c3fff700 1 -- 172.21.15.45:0/2584866955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fc4b40225e0 con 0x7fc4cc100f00 2023-09-23T09:29:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.222+0000 7fc4c3fff700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc4b80a1630 0x7fc4b80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.222+0000 7fc4c3fff700 1 -- 172.21.15.45:0/2584866955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4b40ce2f0 con 0x7fc4cc100f00 2023-09-23T09:29:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.222+0000 7fc4caffd700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc4b80a1630 0x7fc4b80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.222+0000 7fc4caffd700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc4b80a1630 0x7fc4b80a3af0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc005fd0 tx=0x7fc4bc005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.222+0000 7fc4dd8d5b80 1 Processor -- start 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4dd8d5b80 1 -- start start 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c524970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c525770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c48c410 con 0x564b4c525d50 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c47dbd0 con 0x564b4c524eb0 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c47b4f0 con 0x564b4c524550 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5261d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59990/0 (socket says 172.21.15.45:59990) 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2407893263 learned_addr learned my addr 172.21.15.45:0/2407893263 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4caffd700 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c524970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4ca7fc700 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c525770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2407893263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 msgr2=0x564b4c524970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c524970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2407893263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 msgr2=0x564b4c525770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c525770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.223+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2407893263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b4c4d9880 con 0x564b4c525d50 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5261d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e260 tx=0x7fc4c400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a47074d114531ec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4c17fa700 1 -- 172.21.15.45:0/2407893263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4018070 con 0x564b4c525d50 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4c17fa700 1 -- 172.21.15.45:0/2407893263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4c4007d00 con 0x564b4c525d50 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4c17fa700 1 -- 172.21.15.45:0/2407893263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4014d70 con 0x564b4c525d50 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2407893263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 msgr2=0x564b4c5261d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5261d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e260 tx=0x7fc4c400e570 comp rx=0 tx=0).stop 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2407893263 shutdown_connections 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5261d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c524970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.224+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2407893263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c525770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2407893263 >> 172.21.15.45:0/2407893263 conn(0x564b4c498210 msgr2=0x564b4c51f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2407893263 shutdown_connections 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2407893263 wait complete. 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 -- start start 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 Processor -- start 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c52e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c52e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5358a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c476de0 con 0x564b4c525d50 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c4136c0 con 0x564b4c524eb0 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c413360 con 0x564b4c524550 2023-09-23T09:29:05.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4dd8d5b80 10 client.0 _ll_register_callbacks cb 0x564b4c2707b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5358a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59998/0 (socket says 172.21.15.45:59998) 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/3535508299 learned_addr learned my addr 172.21.15.45:0/3535508299 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4ca7fc700 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c52e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.225+0000 7fc4caffd700 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c52e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 msgr2=0x564b4c52e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c52e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/3535508299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 msgr2=0x564b4c52e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c52e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/3535508299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c400f040 con 0x564b4c525d50 2023-09-23T09:29:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5358a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e8c0 tx=0x7fc4c400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cbee1a15cd77c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4b27fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4b27fc700 1 -- 172.21.15.45:0/3535508299 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4018040 con 0x564b4c525d50 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b4c517200 con 0x564b4c525d50 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4b27fc700 1 -- 172.21.15.45:0/3535508299 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4c4004d60 con 0x564b4c525d50 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4b27fc700 1 -- 172.21.15.45:0/3535508299 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c4014480 con 0x564b4c525d50 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4dd8d5b80 10 client.15192 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x564b4c2a1d90 con 0x564b4c525d50 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.226+0000 7fc4b27fc700 1 -- 172.21.15.45:0/3535508299 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b4c517200 con 0x564b4c525d50 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4b27fc700 1 -- 172.21.15.45:0/3535508299 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc4c4004360 con 0x564b4c525d50 2023-09-23T09:29:05.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4dd8d5b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4dd8d5b80 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4dd8d5b80 20 client.15192 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c539d80 0x564b4c538890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4dd8d5b80 4 client.15192 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b4c46fdc0 con 0x564b4c539d80 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4caffd700 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c539d80 0x564b4c538890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4caffd700 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c539d80 0x564b4c538890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9ca7d16528d77c5 server_cookie=2bb6871d89e7d72a in_seq=0 out_seq=0 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.227+0000 7fc4b27fc700 10 client.15192 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4b27fc700 1 -- 172.21.15.45:0/3535508299 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b4c46fdc0 con 0x564b4c539d80 2023-09-23T09:29:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4b27fc700 10 client.15192 handle_command_reply: tid=0 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 client.15192 shutdown 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 msgr2=0x564b4c5358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5358a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e8c0 tx=0x7fc4c400efd0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 shutdown_connections 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c5358a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c539d80 0x564b4c538890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524550 0x564b4c52e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/3535508299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524eb0 0x564b4c52e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4b27fc700 0 client.15192 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 >> 172.21.15.45:0/3535508299 conn(0x564b4c498210 msgr2=0x564b4c51f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 shutdown_connections 2023-09-23T09:29:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.228+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/3535508299 wait complete. 2023-09-23T09:29:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.229+0000 7fc4dd8d5b80 20 client.15192 trim_cache size 0 max 16384 2023-09-23T09:29:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 Processor -- start 2023-09-23T09:29:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 -- start start 2023-09-23T09:29:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c51f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c535860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c52cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c2f78d0 con 0x564b4c525d50 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c303de0 con 0x564b4c524550 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.246+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c435590 con 0x564b4c524eb0 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c52cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c52cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60006/0 (socket says 172.21.15.45:60006) 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2663938628 learned_addr learned my addr 172.21.15.45:0/2663938628 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4ca7fc700 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c535860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4caffd700 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c51f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2663938628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 msgr2=0x564b4c535860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c535860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2663938628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 msgr2=0x564b4c51f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c51f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/2663938628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b4c28ac10 con 0x564b4c525d50 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c52cff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4007c50 tx=0x7fc4c4004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab095d61bb809f50 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4c1ffb700 1 -- 172.21.15.45:0/2663938628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4004d60 con 0x564b4c525d50 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4c1ffb700 1 -- 172.21.15.45:0/2663938628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4c401db80 con 0x564b4c525d50 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.247+0000 7fc4c1ffb700 1 -- 172.21.15.45:0/2663938628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4014880 con 0x564b4c525d50 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2663938628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 msgr2=0x564b4c52cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c52cff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4007c50 tx=0x7fc4c4004aa0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2663938628 shutdown_connections 2023-09-23T09:29:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c52cff0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c51f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/2663938628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c535860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2663938628 >> 172.21.15.45:0/2663938628 conn(0x564b4c498210 msgr2=0x564b4c51deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2663938628 shutdown_connections 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/2663938628 wait complete. 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- start start 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 Processor -- start 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c52fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c52ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c534fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c535510 con 0x564b4c525d50 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c535680 con 0x564b4c524550 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4c535840 con 0x564b4c524eb0 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4dd8d5b80 10 client.0 _ll_register_callbacks cb 0x7fc4bc009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.248+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c534fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c534fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60012/0 (socket says 172.21.15.45:60012) 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/61197934 learned_addr learned my addr 172.21.15.45:0/61197934 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4caffd700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c52fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4ca7fc700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c52ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 msgr2=0x564b4c52ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c52ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/61197934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 msgr2=0x564b4c52fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c52fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 -- 172.21.15.45:0/61197934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b4c2a1d90 con 0x564b4c525d50 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4caffd700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c52fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4cb7fe700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c534fd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e8c0 tx=0x7fc4c401da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaa8c143afeceaf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4b27fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4b27fc700 1 -- 172.21.15.45:0/61197934 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4014a10 con 0x564b4c525d50 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4c4007d00 con 0x564b4c525d50 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4b27fc700 1 -- 172.21.15.45:0/61197934 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4c4014b70 con 0x564b4c525d50 2023-09-23T09:29:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4b27fc700 1 -- 172.21.15.45:0/61197934 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c4014d30 con 0x564b4c525d50 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4dd8d5b80 10 client.15198 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.249+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x564b4c2c8200 con 0x564b4c525d50 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4b27fc700 1 -- 172.21.15.45:0/61197934 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4c4007d00 con 0x564b4c525d50 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4b27fc700 1 -- 172.21.15.45:0/61197934 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc4c4039070 con 0x564b4c525d50 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4dd8d5b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4dd8d5b80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4dd8d5b80 20 client.15198 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c591ec0 0x564b4c58e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4dd8d5b80 4 client.15198 mds_command: new command op to 24496 tid=0[{"prefix": "status"}] 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b4c46fae0 con 0x564b4c591ec0 2023-09-23T09:29:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4caffd700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c591ec0 0x564b4c58e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4b27fc700 10 client.15198 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.250+0000 7fc4caffd700 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c591ec0 0x564b4c58e400 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b305d1a7b9fa5763 server_cookie=985626299e70ac4c in_seq=0 out_seq=0 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4b27fc700 1 -- 172.21.15.45:0/61197934 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564b4c46fae0 con 0x564b4c591ec0 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4b27fc700 10 client.15198 handle_command_reply: tid=0 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 client.15198 shutdown 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 msgr2=0x564b4c534fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c534fd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e8c0 tx=0x7fc4c401da50 comp rx=0 tx=0).stop 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 shutdown_connections 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b4c525d50 0x564b4c534fd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b4c524550 0x564b4c52fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x564b4c591ec0 0x564b4c58e400 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 --2- 172.21.15.45:0/61197934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b4c524eb0 0x564b4c52ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 >> 172.21.15.45:0/61197934 conn(0x564b4c498210 msgr2=0x564b4c52c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4b27fc700 0 client.15198 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 shutdown_connections 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 1 -- 172.21.15.45:0/61197934 wait complete. 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4dd8d5b80 20 client.15198 trim_cache size 0 max 16384 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:05.261 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24496, 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 70.559212479999999, 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 22, 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-09-23T09:29:05.262 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 88.056211911999995 2023-09-23T09:29:05.263 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc4b80a1630 msgr2=0x7fc4b80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc4b80a1630 0x7fc4b80a3af0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc005fd0 tx=0x7fc4bc005f00 comp rx=0 tx=0).stop 2023-09-23T09:29:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 msgr2=0x7fc4cc19b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc19b7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009990 tx=0x7fc4b4005e80 comp rx=0 tx=0).stop 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 shutdown_connections 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc4b80a1630 0x7fc4b80a3af0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0ffd10 0x7fc4cc068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 --2- 172.21.15.45:0/2584866955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4cc100f00 0x7fc4cc19b7e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.251+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 >> 172.21.15.45:0/2584866955 conn(0x7fc4cc0fb280 msgr2=0x7fc4cc1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.252+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 shutdown_connections 2023-09-23T09:29:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.252+0000 7fc4d16eb700 1 -- 172.21.15.45:0/2584866955 wait complete. 2023-09-23T09:29:05.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-23T09:29:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 -- 172.21.15.45:0/1319899186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c1018d0 msgr2=0x7f791c108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 --2- 172.21.15.45:0/1319899186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c1018d0 0x7f791c108180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f791400b0d0 tx=0x7f791400b3e0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 -- 172.21.15.45:0/1319899186 shutdown_connections 2023-09-23T09:29:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 --2- 172.21.15.45:0/1319899186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c1018d0 0x7f791c108180 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 --2- 172.21.15.45:0/1319899186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f791c0ffd40 0x7f791c100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 --2- 172.21.15.45:0/1319899186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c100f30 0x7f791c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 -- 172.21.15.45:0/1319899186 >> 172.21.15.45:0/1319899186 conn(0x7f791c0fb2b0 msgr2=0x7f791c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 -- 172.21.15.45:0/1319899186 shutdown_connections 2023-09-23T09:29:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.391+0000 7f7922e7e700 1 -- 172.21.15.45:0/1319899186 wait complete. 2023-09-23T09:29:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 Processor -- start 2023-09-23T09:29:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 -- start start 2023-09-23T09:29:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c0ffd40 0x7f791c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c100f30 0x7f791c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f791c1018d0 0x7f791c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c19e700 con 0x7f791c100f30 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c19e840 con 0x7f791c1018d0 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7922e7e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c19e9d0 con 0x7f791c0ffd40 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f791bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c100f30 0x7f791c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f791bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c100f30 0x7f791c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60024/0 (socket says 172.21.15.45:60024) 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f791bfff700 1 -- 172.21.15.45:0/935607352 learned_addr learned my addr 172.21.15.45:0/935607352 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f7920c1a700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c0ffd40 0x7f791c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.392+0000 7f792141b700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f791c1018d0 0x7f791c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f791bfff700 1 -- 172.21.15.45:0/935607352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c0ffd40 msgr2=0x7f791c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f791bfff700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c0ffd40 0x7f791c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f791bfff700 1 -- 172.21.15.45:0/935607352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f791c1018d0 msgr2=0x7f791c19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f791bfff700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f791c1018d0 0x7f791c19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f791bfff700 1 -- 172.21.15.45:0/935607352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7914009d00 con 0x7f791c100f30 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f792141b700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f791c1018d0 0x7f791c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f791bfff700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c100f30 0x7f791c1970c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f790800c9b0 tx=0x7f790800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f7919ffb700 1 -- 172.21.15.45:0/935607352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908007b00 con 0x7f791c100f30 2023-09-23T09:29:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f7919ffb700 1 -- 172.21.15.45:0/935607352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7908007c60 con 0x7f791c100f30 2023-09-23T09:29:05.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c19ecb0 con 0x7f791c100f30 2023-09-23T09:29:05.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f7919ffb700 1 -- 172.21.15.45:0/935607352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908018660 con 0x7f791c100f30 2023-09-23T09:29:05.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.393+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c19f1c0 con 0x7f791c100f30 2023-09-23T09:29:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.395+0000 7f7919ffb700 1 -- 172.21.15.45:0/935607352 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f790800f450 con 0x7f791c100f30 2023-09-23T09:29:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.395+0000 7f7919ffb700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f790c0a16b0 0x7f790c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.396+0000 7f7919ffb700 1 -- 172.21.15.45:0/935607352 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f79080ccff0 con 0x7f791c100f30 2023-09-23T09:29:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.396+0000 7f7920c1a700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f790c0a16b0 0x7f790c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.396+0000 7f7920c1a700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f790c0a16b0 0x7f790c0a3b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f791c100d90 tx=0x7f791000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 Processor -- start 2023-09-23T09:29:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 -- start start 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c49d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4a2c0 0x55b040c4ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4b160 0x55b040c4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040bad410 con 0x55b040c49960 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040b9ebd0 con 0x55b040c4b160 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792f068b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040b9c4f0 con 0x55b040c4a2c0 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c49d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c49d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60030/0 (socket says 172.21.15.45:60030) 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 -- 172.21.15.45:0/863522534 learned_addr learned my addr 172.21.15.45:0/863522534 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792141b700 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4b160 0x55b040c4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 -- 172.21.15.45:0/863522534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4a2c0 msgr2=0x55b040c4ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4a2c0 0x55b040c4ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 -- 172.21.15.45:0/863522534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4b160 msgr2=0x55b040c4b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4b160 0x55b040c4b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f7920c1a700 1 -- 172.21.15.45:0/863522534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b040bfa880 con 0x55b040c49960 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.397+0000 7f792141b700 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4b160 0x55b040c4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f7920c1a700 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c49d80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f791000fd80 tx=0x7f7910012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd3b9e22b562977 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f7906ffd700 1 -- 172.21.15.45:0/863522534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910010ac0 con 0x55b040c49960 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f7906ffd700 1 -- 172.21.15.45:0/863522534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7910020070 con 0x55b040c49960 2023-09-23T09:29:05.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f7906ffd700 1 -- 172.21.15.45:0/863522534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791001be10 con 0x55b040c49960 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 -- 172.21.15.45:0/863522534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 msgr2=0x55b040c49d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c49d80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f791000fd80 tx=0x7f7910012000 comp rx=0 tx=0).stop 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 -- 172.21.15.45:0/863522534 shutdown_connections 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4b160 0x55b040c4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c49d80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 --2- 172.21.15.45:0/863522534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4a2c0 0x55b040c4ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 -- 172.21.15.45:0/863522534 >> 172.21.15.45:0/863522534 conn(0x55b040bb9210 msgr2=0x55b040c40430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 -- 172.21.15.45:0/863522534 shutdown_connections 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.398+0000 7f792f068b80 1 -- 172.21.15.45:0/863522534 wait complete. 2023-09-23T09:29:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 -- start start 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 Processor -- start 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c53810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c53d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040b97de0 con 0x55b040c4a2c0 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040b346c0 con 0x55b040c49960 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040b34360 con 0x55b040c4b160 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f792f068b80 10 client.0 _ll_register_callbacks cb 0x55b0409917b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:05.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f791bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c53d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f791bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c53d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60038/0 (socket says 172.21.15.45:60038) 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f791bfff700 1 -- 172.21.15.45:0/2130539155 learned_addr learned my addr 172.21.15.45:0/2130539155 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.399+0000 7f7920c1a700 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c53810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f792141b700 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f791bfff700 1 -- 172.21.15.45:0/2130539155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 msgr2=0x55b040c5ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f791bfff700 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f791bfff700 1 -- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 msgr2=0x55b040c53810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f791bfff700 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c53810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f791bfff700 1 -- 172.21.15.45:0/2130539155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791000fa60 con 0x55b040c4a2c0 2023-09-23T09:29:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f791bfff700 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c53d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7908096640 tx=0x7f790809a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915d20052c503118 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f78fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f78fbfff700 1 -- 172.21.15.45:0/2130539155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790809e290 con 0x55b040c4a2c0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f78fbfff700 1 -- 172.21.15.45:0/2130539155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f79080a24d0 con 0x55b040c4a2c0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b040c38200 con 0x55b040c4a2c0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f78fbfff700 1 -- 172.21.15.45:0/2130539155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f790809a3a0 con 0x55b040c4a2c0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f792f068b80 10 client.15210 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b0409c2d90 con 0x55b040c4a2c0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.400+0000 7f78fbfff700 1 -- 172.21.15.45:0/2130539155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b040c38200 con 0x55b040c4a2c0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f78fbfff700 1 -- 172.21.15.45:0/2130539155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f79080a2040 con 0x55b040c4a2c0 2023-09-23T09:29:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f792f068b80 10 client.15210 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f792f068b80 10 client.15210 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f792f068b80 20 client.15210 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f792f068b80 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040c5f260 0x55b040c5de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f792f068b80 4 client.15210 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b040b90dc0 con 0x55b040c5f260 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f7920c1a700 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040c5f260 0x55b040c5de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f78fbfff700 10 client.15210 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.401+0000 7f7920c1a700 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040c5f260 0x55b040c5de60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99807b981e652c95 server_cookie=724239576a7e173d in_seq=0 out_seq=0 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f78fbfff700 1 -- 172.21.15.45:0/2130539155 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b040b90dc0 con 0x55b040c5f260 2023-09-23T09:29:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f78fbfff700 10 client.15210 handle_command_reply: tid=0 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 client.15210 shutdown 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 msgr2=0x55b040c53d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c53d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7908096640 tx=0x7f790809a840 comp rx=0 tx=0).stop 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 shutdown_connections 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040c5f260 0x55b040c5de60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f78fbfff700 0 client.15210 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c53810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 --2- 172.21.15.45:0/2130539155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c53d50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 >> 172.21.15.45:0/2130539155 conn(0x55b040bb9210 msgr2=0x55b040c40240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 shutdown_connections 2023-09-23T09:29:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.402+0000 7f792f068b80 1 -- 172.21.15.45:0/2130539155 wait complete. 2023-09-23T09:29:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.403+0000 7f792f068b80 20 client.15210 trim_cache size 0 max 16384 2023-09-23T09:29:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 Processor -- start 2023-09-23T09:29:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 -- start start 2023-09-23T09:29:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c40240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c5ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040a188d0 con 0x55b040c4a2c0 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040c54a00 con 0x55b040c49960 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.420+0000 7f792f068b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040c51cc0 con 0x55b040c4b160 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c5ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c5ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60046/0 (socket says 172.21.15.45:60046) 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 -- 172.21.15.45:0/1031082118 learned_addr learned my addr 172.21.15.45:0/1031082118 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f7920c1a700 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c40240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f792141b700 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 -- 172.21.15.45:0/1031082118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 msgr2=0x55b040c5b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 -- 172.21.15.45:0/1031082118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 msgr2=0x55b040c40240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c40240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 -- 172.21.15.45:0/1031082118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0409abc10 con 0x55b040c4a2c0 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f791bfff700 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c5ad50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f790809a0d0 tx=0x7f79080986f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0f6b7620742932 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f79077fe700 1 -- 172.21.15.45:0/1031082118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79080ca8e0 con 0x55b040c4a2c0 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f79077fe700 1 -- 172.21.15.45:0/1031082118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79080a24d0 con 0x55b040c4a2c0 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.421+0000 7f79077fe700 1 -- 172.21.15.45:0/1031082118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908099630 con 0x55b040c4a2c0 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 -- 172.21.15.45:0/1031082118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 msgr2=0x55b040c5ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c5ad50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f790809a0d0 tx=0x7f79080986f0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 -- 172.21.15.45:0/1031082118 shutdown_connections 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c5b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c49960 0x55b040c40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 --2- 172.21.15.45:0/1031082118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c4a2c0 0x55b040c5ad50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 -- 172.21.15.45:0/1031082118 >> 172.21.15.45:0/1031082118 conn(0x55b040bb9210 msgr2=0x55b040c515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 -- 172.21.15.45:0/1031082118 shutdown_connections 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 -- 172.21.15.45:0/1031082118 wait complete. 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 -- start start 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 Processor -- start 2023-09-23T09:29:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.422+0000 7f792f068b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4a2c0 0x55b040c574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c57a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040c57f40 con 0x55b040c49960 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040c580b0 con 0x55b040c4a2c0 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040cb2c50 con 0x55b040c4b160 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 10 client.0 _ll_register_callbacks cb 0x55b0409917b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c5aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60050/0 (socket says 172.21.15.45:60050) 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 -- 172.21.15.45:0/3108043070 learned_addr learned my addr 172.21.15.45:0/3108043070 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792141b700 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c57a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f791bfff700 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4a2c0 0x55b040c574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 -- 172.21.15.45:0/3108043070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 msgr2=0x55b040c57a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c57a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 -- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4a2c0 msgr2=0x55b040c574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4a2c0 0x55b040c574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 -- 172.21.15.45:0/3108043070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0409c2d90 con 0x55b040c49960 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f7920c1a700 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c5aa20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f791000fd80 tx=0x7f791000f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f44cff68087aab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f78fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f78fbfff700 1 -- 172.21.15.45:0/3108043070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791002a040 con 0x55b040c49960 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f78fbfff700 1 -- 172.21.15.45:0/3108043070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7910032bf0 con 0x55b040c49960 2023-09-23T09:29:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7910020070 con 0x55b040c49960 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f78fbfff700 1 -- 172.21.15.45:0/3108043070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7910032db0 con 0x55b040c49960 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 10 client.15216 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.423+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b0409e9200 con 0x55b040c49960 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f78fbfff700 1 -- 172.21.15.45:0/3108043070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7910020070 con 0x55b040c49960 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f78fbfff700 1 -- 172.21.15.45:0/3108043070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7910038020 con 0x55b040c49960 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f792f068b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f792f068b80 10 client.15216 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f792f068b80 20 client.15216 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f792f068b80 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040cb7e90 0x55b040cb42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f792f068b80 4 client.15216 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b040b90ae0 con 0x55b040cb7e90 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f791bfff700 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040cb7e90 0x55b040cb42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f78fbfff700 10 client.15216 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.424+0000 7f791bfff700 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040cb7e90 0x55b040cb42f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e93a1643243279bc server_cookie=ee56b53fc3d3a977 in_seq=0 out_seq=0 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f78fbfff700 1 -- 172.21.15.45:0/3108043070 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b040b90ae0 con 0x55b040cb7e90 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f78fbfff700 10 client.15216 handle_command_reply: tid=0 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 client.15216 shutdown 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 msgr2=0x55b040c5aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c5aa20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f791000fd80 tx=0x7f791000f800 comp rx=0 tx=0).stop 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 shutdown_connections 2023-09-23T09:29:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x55b040cb7e90 0x55b040cb42f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b040c4b160 0x55b040c57a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f78fbfff700 0 client.15216 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b040c49960 0x55b040c5aa20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 --2- 172.21.15.45:0/3108043070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b040c4a2c0 0x55b040c574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 >> 172.21.15.45:0/3108043070 conn(0x55b040bb9210 msgr2=0x55b040c40200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 shutdown_connections 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 1 -- 172.21.15.45:0/3108043070 wait complete. 2023-09-23T09:29:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.425+0000 7f792f068b80 20 client.15216 trim_cache size 0 max 16384 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24500, 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 71.745290518999994, 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 22, 2023-09-23T09:29:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-09-23T09:29:05.437 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-09-23T09:29:05.437 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 90.398221082999996 2023-09-23T09:29:05.437 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f790c0a16b0 msgr2=0x7f790c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f790c0a16b0 0x7f790c0a3b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f791c100d90 tx=0x7f791000a380 comp rx=0 tx=0).stop 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c100f30 msgr2=0x7f791c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c100f30 0x7f791c1970c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f790800c9b0 tx=0x7f790800ccc0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 shutdown_connections 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f791c1018d0 0x7f791c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f790c0a16b0 0x7f790c0a3b70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c0ffd40 0x7f791c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 --2- 172.21.15.45:0/935607352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791c100f30 0x7f791c1970c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 >> 172.21.15.45:0/935607352 conn(0x7f791c0fb2b0 msgr2=0x7f791c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 shutdown_connections 2023-09-23T09:29:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.426+0000 7f7922e7e700 1 -- 172.21.15.45:0/935607352 wait complete. 2023-09-23T09:29:05.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:05 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/589947338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:05.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:05 smithi045 ceph-mon[98627]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:05.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:05 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1456497247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:05.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:05 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2922151638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:05.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-23T09:29:05.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 -- 172.21.15.45:0/2588704446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c101380 msgr2=0x7f305c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 --2- 172.21.15.45:0/2588704446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c101380 0x7f305c1017a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3050009990 tx=0x7f3050009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 -- 172.21.15.45:0/2588704446 shutdown_connections 2023-09-23T09:29:05.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 --2- 172.21.15.45:0/2588704446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c100900 0x7f305c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 --2- 172.21.15.45:0/2588704446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c101380 0x7f305c1017a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 --2- 172.21.15.45:0/2588704446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f305c0fff40 0x7f305c1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 -- 172.21.15.45:0/2588704446 >> 172.21.15.45:0/2588704446 conn(0x7f305c0fa9f0 msgr2=0x7f305c0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 -- 172.21.15.45:0/2588704446 shutdown_connections 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.564+0000 7f30630a3700 1 -- 172.21.15.45:0/2588704446 wait complete. 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 Processor -- start 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 -- start start 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c0fff40 0x7f305c0ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c100900 0x7f305c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f305c101380 0x7f305c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c19e6d0 con 0x7f305c0fff40 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c19e840 con 0x7f305c100900 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f30630a3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c19ea00 con 0x7f305c101380 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c0fff40 0x7f305c0ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c0fff40 0x7f305c0ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60064/0 (socket says 172.21.15.45:60064) 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f3060e3f700 1 -- 172.21.15.45:0/3098065864 learned_addr learned my addr 172.21.15.45:0/3098065864 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f3061640700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f305c101380 0x7f305c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.565+0000 7f305bfff700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c100900 0x7f305c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3060e3f700 1 -- 172.21.15.45:0/3098065864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f305c101380 msgr2=0x7f305c19df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3060e3f700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f305c101380 0x7f305c19df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3060e3f700 1 -- 172.21.15.45:0/3098065864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c100900 msgr2=0x7f305c19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3060e3f700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c100900 0x7f305c19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3060e3f700 1 -- 172.21.15.45:0/3098065864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30500095f0 con 0x7f305c0fff40 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f305bfff700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c100900 0x7f305c19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3060e3f700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c0fff40 0x7f305c0ff9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3050000c00 tx=0x7f305000f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3059ffb700 1 -- 172.21.15.45:0/3098065864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305001d070 con 0x7f305c0fff40 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3059ffb700 1 -- 172.21.15.45:0/3098065864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305000fc30 con 0x7f305c0fff40 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c19ec80 con 0x7f305c0fff40 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f3059ffb700 1 -- 172.21.15.45:0/3098065864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050017650 con 0x7f305c0fff40 2023-09-23T09:29:05.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.566+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c19f180 con 0x7f305c0fff40 2023-09-23T09:29:05.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.568+0000 7f3059ffb700 1 -- 172.21.15.45:0/3098065864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f3050022470 con 0x7f305c0fff40 2023-09-23T09:29:05.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.569+0000 7f3059ffb700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f304c0a16b0 0x7f304c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.569+0000 7f3059ffb700 1 -- 172.21.15.45:0/3098065864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f30500ce3d0 con 0x7f305c0fff40 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.569+0000 7f305bfff700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f304c0a16b0 0x7f304c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.569+0000 7f305bfff700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f304c0a16b0 0x7f304c0a3b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f305c0ff730 tx=0x7f3048008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 Processor -- start 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 -- start start 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a285960 0x559a5a285d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a2862c0 0x559a5a286b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a287160 0x559a5a2875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a1e9410 con 0x559a5a2862c0 2023-09-23T09:29:05.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a1dabd0 con 0x559a5a285960 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f306f28db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a1d84f0 con 0x559a5a287160 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f305bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a2862c0 0x559a5a286b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f305bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a2862c0 0x559a5a286b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60074/0 (socket says 172.21.15.45:60074) 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f305bfff700 1 -- 172.21.15.45:0/901118096 learned_addr learned my addr 172.21.15.45:0/901118096 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f3060e3f700 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a285960 0x559a5a285d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f3061640700 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a287160 0x559a5a2875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f305bfff700 1 -- 172.21.15.45:0/901118096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a287160 msgr2=0x559a5a2875e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f305bfff700 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a287160 0x559a5a2875e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f305bfff700 1 -- 172.21.15.45:0/901118096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a285960 msgr2=0x559a5a285d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.570+0000 7f305bfff700 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a285960 0x559a5a285d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f305bfff700 1 -- 172.21.15.45:0/901118096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a5a236880 con 0x559a5a2862c0 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f305bfff700 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a2862c0 0x559a5a286b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f304800d780 tx=0x7f3048010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b01dfe0644076e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f3046ffd700 1 -- 172.21.15.45:0/901118096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048014640 con 0x559a5a2862c0 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f3046ffd700 1 -- 172.21.15.45:0/901118096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3048014c60 con 0x559a5a2862c0 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f3046ffd700 1 -- 172.21.15.45:0/901118096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304801fc00 con 0x559a5a2862c0 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f306f28db80 1 -- 172.21.15.45:0/901118096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a2862c0 msgr2=0x559a5a286b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f306f28db80 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a2862c0 0x559a5a286b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f304800d780 tx=0x7f3048010000 comp rx=0 tx=0).stop 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.571+0000 7f306f28db80 1 -- 172.21.15.45:0/901118096 shutdown_connections 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a287160 0x559a5a2875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a285960 0x559a5a285d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 --2- 172.21.15.45:0/901118096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a2862c0 0x559a5a286b80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 -- 172.21.15.45:0/901118096 >> 172.21.15.45:0/901118096 conn(0x559a5a1f5210 msgr2=0x559a5a27c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 -- 172.21.15.45:0/901118096 shutdown_connections 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 -- 172.21.15.45:0/901118096 wait complete. 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 -- start start 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.572+0000 7f306f28db80 1 Processor -- start 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f306f28db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f306f28db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a28fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f306f28db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a296d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f306f28db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a1d3de0 con 0x559a5a285960 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f306f28db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a1706c0 con 0x559a5a287160 2023-09-23T09:29:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f306f28db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a170360 con 0x559a5a2862c0 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f306f28db80 10 client.0 _ll_register_callbacks cb 0x559a59fcd7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60078/0 (socket says 172.21.15.45:60078) 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 -- 172.21.15.45:0/947009571 learned_addr learned my addr 172.21.15.45:0/947009571 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3061640700 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a296d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f305bfff700 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a28fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 -- 172.21.15.45:0/947009571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 msgr2=0x559a5a28fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a28fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 -- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 msgr2=0x559a5a296d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a296d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 -- 172.21.15.45:0/947009571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304800e040 con 0x559a5a285960 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f3060e3f700 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28f7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3050006010 tx=0x7f305009baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ab4c44778bc130 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f303bfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f303bfff700 1 -- 172.21.15.45:0/947009571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050086070 con 0x559a5a285960 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.573+0000 7f303bfff700 1 -- 172.21.15.45:0/947009571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f305009bcb0 con 0x559a5a285960 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a5a274200 con 0x559a5a285960 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f303bfff700 1 -- 172.21.15.45:0/947009571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f305009b5b0 con 0x559a5a285960 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 10 client.15228 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559a59ffed90 con 0x559a5a285960 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f303bfff700 1 -- 172.21.15.45:0/947009571 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30500b9020 con 0x559a5a285960 2023-09-23T09:29:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f303bfff700 1 -- 172.21.15.45:0/947009571 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f305007b080 con 0x559a5a285960 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 10 client.15228 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 20 client.15228 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a29afd0 0x559a5a299c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 4 client.15228 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a5a1ccdc0 con 0x559a5a29afd0 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.574+0000 7f305bfff700 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a29afd0 0x559a5a299c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.575+0000 7f303bfff700 10 client.15228 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.575+0000 7f305bfff700 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a29afd0 0x559a5a299c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b821aa5476c46d server_cookie=e58aa3c04be25316 in_seq=0 out_seq=0 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.575+0000 7f303bfff700 1 -- 172.21.15.45:0/947009571 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559a5a1ccdc0 con 0x559a5a29afd0 2023-09-23T09:29:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.575+0000 7f303bfff700 10 client.15228 handle_command_reply: tid=0 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.575+0000 7f306f28db80 1 client.15228 shutdown 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 msgr2=0x559a5a28f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28f7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3050006010 tx=0x7f305009baf0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 shutdown_connections 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a29afd0 0x559a5a299c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a296d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28f7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 --2- 172.21.15.45:0/947009571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a28fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f303bfff700 0 client.15228 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 >> 172.21.15.45:0/947009571 conn(0x559a5a1f5210 msgr2=0x559a5a27c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 shutdown_connections 2023-09-23T09:29:05.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 1 -- 172.21.15.45:0/947009571 wait complete. 2023-09-23T09:29:05.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.576+0000 7f306f28db80 20 client.15228 trim_cache size 0 max 16384 2023-09-23T09:29:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:05 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/589947338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:05 smithi123 ceph-mon[103577]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:05 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1456497247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:05.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:05 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2922151638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.593+0000 7f306f28db80 1 Processor -- start 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f306f28db80 1 -- start start 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f306f28db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f306f28db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a296ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f306f28db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a28a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f306f28db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a0548d0 con 0x559a5a285960 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f306f28db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a2765f0 con 0x559a5a287160 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f306f28db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a1ccdc0 con 0x559a5a2862c0 2023-09-23T09:29:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60086/0 (socket says 172.21.15.45:60086) 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 -- 172.21.15.45:0/513696535 learned_addr learned my addr 172.21.15.45:0/513696535 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3061640700 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a28a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 -- 172.21.15.45:0/513696535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 msgr2=0x559a5a296ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a296ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 -- 172.21.15.45:0/513696535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 msgr2=0x559a5a28a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a28a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 -- 172.21.15.45:0/513696535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a59fe7c10 con 0x559a5a285960 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3061640700 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a28a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f3060e3f700 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28a0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f305009b2b0 tx=0x7f3050099b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83726f358e35b92 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f30477fe700 1 -- 172.21.15.45:0/513696535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050086070 con 0x559a5a285960 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f30477fe700 1 -- 172.21.15.45:0/513696535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30500b9070 con 0x559a5a285960 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.594+0000 7f30477fe700 1 -- 172.21.15.45:0/513696535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305009f700 con 0x559a5a285960 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 -- 172.21.15.45:0/513696535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 msgr2=0x559a5a28a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28a0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f305009b2b0 tx=0x7f3050099b60 comp rx=0 tx=0).stop 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 -- 172.21.15.45:0/513696535 shutdown_connections 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a28a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a28a0d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 --2- 172.21.15.45:0/513696535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a296ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 -- 172.21.15.45:0/513696535 >> 172.21.15.45:0/513696535 conn(0x559a5a1f5210 msgr2=0x559a5a28e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 -- 172.21.15.45:0/513696535 shutdown_connections 2023-09-23T09:29:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 -- 172.21.15.45:0/513696535 wait complete. 2023-09-23T09:29:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 -- start start 2023-09-23T09:29:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.595+0000 7f306f28db80 1 Processor -- start 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a2966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a296c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a293210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a293970 con 0x559a5a285960 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a293ae0 con 0x559a5a287160 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5a293ca0 con 0x559a5a2862c0 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 10 client.0 _ll_register_callbacks cb 0x7f305009b0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a2966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a2966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60092/0 (socket says 172.21.15.45:60092) 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 -- 172.21.15.45:0/34956693 learned_addr learned my addr 172.21.15.45:0/34956693 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f305bfff700 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a296c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 -- 172.21.15.45:0/34956693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 msgr2=0x559a5a296c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a296c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 -- 172.21.15.45:0/34956693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 msgr2=0x559a5a293210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a293210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 -- 172.21.15.45:0/34956693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a59ffed90 con 0x559a5a285960 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f3060e3f700 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a2966f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f30500cc300 tx=0x7f30500bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf18f281a158af6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f303bfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f303bfff700 1 -- 172.21.15.45:0/34956693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050084df0 con 0x559a5a285960 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f303bfff700 1 -- 172.21.15.45:0/34956693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f30500b3e40 con 0x559a5a285960 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.596+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3048014c60 con 0x559a5a285960 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f303bfff700 1 -- 172.21.15.45:0/34956693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30500860b0 con 0x559a5a285960 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 10 client.15234 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559a5a025200 con 0x559a5a285960 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f303bfff700 1 -- 172.21.15.45:0/34956693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3048014c60 con 0x559a5a285960 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f303bfff700 1 -- 172.21.15.45:0/34956693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f305007b070 con 0x559a5a285960 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 10 client.15234 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 20 client.15234 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a2f3160 0x559a5a2ef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 4 client.15234 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559a5a1ccae0 con 0x559a5a2f3160 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.597+0000 7f305bfff700 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a2f3160 0x559a5a2ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f303bfff700 10 client.15234 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f305bfff700 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a2f3160 0x559a5a2ef870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4045058365c6e963 server_cookie=882d616815f8b74a in_seq=0 out_seq=0 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f303bfff700 1 -- 172.21.15.45:0/34956693 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559a5a1ccae0 con 0x559a5a2f3160 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f303bfff700 10 client.15234 handle_command_reply: tid=0 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 client.15234 shutdown 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 msgr2=0x559a5a2966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a2966f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f30500cc300 tx=0x7f30500bb040 comp rx=0 tx=0).stop 2023-09-23T09:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 shutdown_connections 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559a5a2f3160 0x559a5a2ef870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a5a287160 0x559a5a293210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5a285960 0x559a5a2966f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 --2- 172.21.15.45:0/34956693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a5a2862c0 0x559a5a296c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f303bfff700 0 client.15234 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 >> 172.21.15.45:0/34956693 conn(0x559a5a1f5210 msgr2=0x559a5a28a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 shutdown_connections 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 1 -- 172.21.15.45:0/34956693 wait complete. 2023-09-23T09:29:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.598+0000 7f306f28db80 20 client.15234 trim_cache size 0 max 16384 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "654e5bfa-59f2-11ee-8db4-212e2dc638e7", 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24530, 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 74.943217064999999, 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 22, 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-09-23T09:29:05.608 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-09-23T09:29:05.609 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 81.668366065000001 2023-09-23T09:29:05.609 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f304c0a16b0 msgr2=0x7f304c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f304c0a16b0 0x7f304c0a3b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f305c0ff730 tx=0x7f3048008040 comp rx=0 tx=0).stop 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c0fff40 msgr2=0x7f305c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c0fff40 0x7f305c0ff9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3050000c00 tx=0x7f305000f780 comp rx=0 tx=0).stop 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 shutdown_connections 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f304c0a16b0 0x7f304c0a3b70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f305c101380 0x7f305c19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c0fff40 0x7f305c0ff9c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 --2- 172.21.15.45:0/3098065864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c100900 0x7f305c19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 >> 172.21.15.45:0/3098065864 conn(0x7f305c0fa9f0 msgr2=0x7f305c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 shutdown_connections 2023-09-23T09:29:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:05.599+0000 7f30630a3700 1 -- 172.21.15.45:0/3098065864 wait complete. 2023-09-23T09:29:05.613 INFO:teuthology.run_tasks:Running task kclient... 2023-09-23T09:29:05.634 INFO:tasks.kclient:Mounting kernel clients... 2023-09-23T09:29:05.634 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-23T09:29:05.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:05.635 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-09-23T09:29:05.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:05.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-09-23T09:29:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2023-09-23T09:29:05.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:29:05.723 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-09-23T09:29:05.724 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T09:29:05.724 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-09-23T09:29:05.724 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-09-23T09:29:05.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph fs ls 2023-09-23T09:29:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:05 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/589947338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:05 smithi098 ceph-mon[102386]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:05 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1456497247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:05.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:05 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2922151638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:29:07.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 -- 172.21.15.45:0/1570173012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 msgr2=0x7f987c107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:07.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 --2- 172.21.15.45:0/1570173012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c107390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9878009990 tx=0x7f9878009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 -- 172.21.15.45:0/1570173012 shutdown_connections 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 --2- 172.21.15.45:0/1570173012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f987c069300 0x7f987c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 --2- 172.21.15.45:0/1570173012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c107390 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 --2- 172.21.15.45:0/1570173012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c068940 0x7f987c068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 -- 172.21.15.45:0/1570173012 >> 172.21.15.45:0/1570173012 conn(0x7f987c076e20 msgr2=0x7f987c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 -- 172.21.15.45:0/1570173012 shutdown_connections 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.498+0000 7f9883afe700 1 -- 172.21.15.45:0/1570173012 wait complete. 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 Processor -- start 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 -- start start 2023-09-23T09:29:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c068940 0x7f987c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f987c069300 0x7f987c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c19bf50 con 0x7f987c106fb0 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c19c0c0 con 0x7f987c069300 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9883afe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c19c280 con 0x7f987c068940 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c19b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60108/0 (socket says 172.21.15.45:60108) 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 -- 172.21.15.45:0/2936447253 learned_addr learned my addr 172.21.15.45:0/2936447253 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988189a700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c068940 0x7f987c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f9881099700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f987c069300 0x7f987c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:07.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 -- 172.21.15.45:0/2936447253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c068940 msgr2=0x7f987c196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c068940 0x7f987c196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 -- 172.21.15.45:0/2936447253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f987c069300 msgr2=0x7f987c1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f987c069300 0x7f987c1974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988209b700 1 -- 172.21.15.45:0/2936447253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780095f0 con 0x7f987c106fb0 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.499+0000 7f988189a700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c068940 0x7f987c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.500+0000 7f988209b700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c19b940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f987400b9e0 tx=0x7f987400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.500+0000 7f9872ffd700 1 -- 172.21.15.45:0/2936447253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400c950 con 0x7f987c106fb0 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.500+0000 7f9872ffd700 1 -- 172.21.15.45:0/2936447253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9874014440 con 0x7f987c106fb0 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.500+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987c19c560 con 0x7f987c106fb0 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.500+0000 7f9872ffd700 1 -- 172.21.15.45:0/2936447253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740075d0 con 0x7f987c106fb0 2023-09-23T09:29:07.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.500+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c1a3dc0 con 0x7f987c106fb0 2023-09-23T09:29:07.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.501+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c04eb20 con 0x7f987c106fb0 2023-09-23T09:29:07.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.502+0000 7f9872ffd700 1 -- 172.21.15.45:0/2936447253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f98740145b0 con 0x7f987c106fb0 2023-09-23T09:29:07.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.502+0000 7f9872ffd700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98680a1630 0x7f98680a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:07.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.503+0000 7f9872ffd700 1 -- 172.21.15.45:0/2936447253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9874006e40 con 0x7f987c106fb0 2023-09-23T09:29:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.503+0000 7f988189a700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98680a1630 0x7f98680a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.503+0000 7f988189a700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98680a1630 0x7f98680a3af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9878006010 tx=0x7f987800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:07.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.507+0000 7f9872ffd700 1 -- 172.21.15.45:0/2936447253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9874095d40 con 0x7f987c106fb0 2023-09-23T09:29:07.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:07 smithi045 ceph-mon[98627]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:07.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.753+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f987c110dc0 con 0x7f987c106fb0 2023-09-23T09:29:07.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.753+0000 7f9872ffd700 1 -- 172.21.15.45:0/2936447253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9874099360 con 0x7f987c106fb0 2023-09-23T09:29:07.755 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T09:29:07.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.755+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98680a1630 msgr2=0x7f98680a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:07.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.755+0000 7f9883afe700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98680a1630 0x7f98680a3af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9878006010 tx=0x7f987800b540 comp rx=0 tx=0).stop 2023-09-23T09:29:07.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.755+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 msgr2=0x7f987c19b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.755+0000 7f9883afe700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c19b940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f987400b9e0 tx=0x7f987400bda0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.755+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 shutdown_connections 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.756+0000 7f9883afe700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c106fb0 0x7f987c19b940 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.756+0000 7f9883afe700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f98680a1630 0x7f98680a3af0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.756+0000 7f9883afe700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c068940 0x7f987c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.756+0000 7f9883afe700 1 --2- 172.21.15.45:0/2936447253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f987c069300 0x7f987c1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.756+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 >> 172.21.15.45:0/2936447253 conn(0x7f987c076e20 msgr2=0x7f987c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.756+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 shutdown_connections 2023-09-23T09:29:07.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:07.756+0000 7f9883afe700 1 -- 172.21.15.45:0/2936447253 wait complete. 2023-09-23T09:29:07.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:07 smithi098 ceph-mon[102386]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:07.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:07 smithi123 ceph-mon[103577]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:08.082 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T09:29:08.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T09:29:08.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-09-23T09:29:08.082 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T09:29:08.083 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-09-23T09:29:08.083 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T09:29:08.083 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T09:29:08.083 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T09:29:08.083 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-09-23T09:29:08.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.083 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-09-23T09:29:08.102 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T09:29:08.102 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T09:29:08.102 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T09:29:08.102 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-09-23T09:29:08.102 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41905sec preferred_lft 41905sec 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-09-23T09:29:08.103 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-09-23T09:29:08.104 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T09:29:08.104 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-09-23T09:29:08.104 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-23T09:29:08.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.104 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-09-23T09:29:08.105 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-09-23T09:29:08.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2023-09-23T09:29:08.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2023-09-23T09:29:08.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2023-09-23T09:29:08.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-23T09:29:08.105 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-09-23T09:29:08.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.276 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T09:29:08.306 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-09-23T09:29:08.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.307 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-09-23T09:29:08.388 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-09-23T09:29:08.388 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T09:29:08.388 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-23T09:29:08.388 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-23T09:29:08.388 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T09:29:08.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.389 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-09-23T09:29:08.389 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-09-23T09:29:08.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-23T09:29:08.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-23T09:29:08.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-23T09:29:08.389 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-09-23T09:29:08.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.643 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-09-23T09:29:08.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.660 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-09-23T09:29:08.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.718 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-09-23T09:29:08.718 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-09-23T09:29:08.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--tmp-cephfs 2023-09-23T09:29:08.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-09-23T09:29:08.718 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-09-23T09:29:08.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:08 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2936447253' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T09:29:08.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:08 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2936447253' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T09:29:08.814 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-09-23T09:29:08.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-09-23T09:29:08.815 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-09-23T09:29:08.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:08 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2936447253' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T09:29:09.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:09.070 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-09-23T09:29:09.070 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-09-23T09:29:09.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-09-23T09:29:09.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2023-09-23T09:29:09.071 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-09-23T09:29:09.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T09:29:09.170 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /tmp/cephfs 2023-09-23T09:29:09.187 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/tmp/cephfs' 2023-09-23T09:29:09.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T09:29:09.187 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /tmp/cephfs 2023-09-23T09:29:09.245 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-09-23T09:29:09.245 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-09-23T09:29:09.245 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/03909e3e-9314-4e2a-91e9-07669c01496d 2023-09-23T09:29:09.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:09.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/03909e3e-9314-4e2a-91e9-07669c01496d /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context=system_u:object_r:postgresql_db_t:s0 2023-09-23T09:29:09.346 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2023-09-23T09:29:09.346 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-23T09:29:09.346 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-09-23T09:29:09.482 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2023-09-23T09:29:09.482 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-23T09:29:09.482 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-09-23T09:29:09.482 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.98:3300,172.21.15.123:3300" 2023-09-23T09:29:09.482 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:3300,172.21.15.98:3300,172.21.15.123:3300:/volumes/_nogroup/sv_1/03909e3e-9314-4e2a-91e9-07669c01496d 2023-09-23T09:29:09.482 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=654e5bfa-59f2-11ee-8db4-212e2dc638e7" will pass to kernel 2023-09-23T09:29:09.485 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-23T09:29:09.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:09.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /tmp/cephfs 2023-09-23T09:29:09.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:09.539 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-23T09:29:09.594 INFO:teuthology.orchestra.run.smithi045.stdout:client15243 2023-09-23T09:29:09.594 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:29:09.594 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T09:29:09.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:09.612 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-23T09:29:09.707 INFO:teuthology.orchestra.run.smithi045.stdout:client15243 2023-09-23T09:29:09.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:09.707 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-23T09:29:09.758 INFO:teuthology.orchestra.run.smithi045.stdout:client15243 2023-09-23T09:29:09.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:29:09.759 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7.client15243/status') 2023-09-23T09:29:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:09 smithi098 ceph-mon[102386]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:09.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:09 smithi045 ceph-mon[98627]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:09.816 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.15243 (3)192.168.0.1:0/690931035 2023-09-23T09:29:09.816 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2023-09-23T09:29:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2023-09-23T09:29:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2023-09-23T09:29:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:69 bytes copied, 2.3657e-05 s, 2.9 MB/s 2023-09-23T09:29:09.817 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-23T09:29:09.828 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-23T09:29:09.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:29:09.883 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:09 smithi123 ceph-mon[103577]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 -- 172.21.15.45:0/2776069068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 msgr2=0x7f68341081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 --2- 172.21.15.45:0/2776069068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f68341081b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f682c00b0a0 tx=0x7f682c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 -- 172.21.15.45:0/2776069068 shutdown_connections 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 --2- 172.21.15.45:0/2776069068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f68341081b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 --2- 172.21.15.45:0/2776069068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834101390 0x7f68341017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 --2- 172.21.15.45:0/2776069068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68340fff50 0x7f68341003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 -- 172.21.15.45:0/2776069068 >> 172.21.15.45:0/2776069068 conn(0x7f68340fa9e0 msgr2=0x7f68340fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:10.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 -- 172.21.15.45:0/2776069068 shutdown_connections 2023-09-23T09:29:10.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 -- 172.21.15.45:0/2776069068 wait complete. 2023-09-23T09:29:10.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 Processor -- start 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.008+0000 7f683a10d700 1 -- start start 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f683a10d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68340fff50 0x7f6834103d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f683a10d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f6834102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f683a10d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834101390 0x7f6834102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f683a10d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834103020 con 0x7f6834100910 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f683a10d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834103190 con 0x7f68340fff50 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f683a10d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68341a0c10 con 0x7f6834101390 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f6834102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f6834102340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60138/0 (socket says 172.21.15.45:60138) 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 -- 172.21.15.45:0/1343108502 learned_addr learned my addr 172.21.15.45:0/1343108502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f68337fe700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68340fff50 0x7f6834103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6833fff700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834101390 0x7f6834102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 -- 172.21.15.45:0/1343108502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834101390 msgr2=0x7f6834102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834101390 0x7f6834102880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 -- 172.21.15.45:0/1343108502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68340fff50 msgr2=0x7f6834103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68340fff50 0x7f6834103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 -- 172.21.15.45:0/1343108502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c009d00 con 0x7f6834100910 2023-09-23T09:29:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f68337fe700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68340fff50 0x7f6834103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:10.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6832ffd700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f6834102340 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f681c00d980 tx=0x7f681c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:10.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6830ff9700 1 -- 172.21.15.45:0/1343108502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00bb60 con 0x7f6834100910 2023-09-23T09:29:10.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f6830ff9700 1 -- 172.21.15.45:0/1343108502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f681c007640 con 0x7f6834100910 2023-09-23T09:29:10.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.009+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68341a0ef0 con 0x7f6834100910 2023-09-23T09:29:10.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.010+0000 7f6830ff9700 1 -- 172.21.15.45:0/1343108502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c004e90 con 0x7f6834100910 2023-09-23T09:29:10.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.010+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68341a1370 con 0x7f6834100910 2023-09-23T09:29:10.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.011+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683410b550 con 0x7f6834100910 2023-09-23T09:29:10.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.012+0000 7f6830ff9700 1 -- 172.21.15.45:0/1343108502 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f681c00bcc0 con 0x7f6834100910 2023-09-23T09:29:10.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.012+0000 7f6830ff9700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f68200a1670 0x7f68200a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.012+0000 7f68337fe700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f68200a1670 0x7f68200a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.012+0000 7f6830ff9700 1 -- 172.21.15.45:0/1343108502 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f681c0ccfc0 con 0x7f6834100910 2023-09-23T09:29:10.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.013+0000 7f68337fe700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f68200a1670 0x7f68200a3b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6824009c90 tx=0x7f6824009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:10.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.017+0000 7f6830ff9700 1 -- 172.21.15.45:0/1343108502 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f681c0ffec0 con 0x7f6834100910 2023-09-23T09:29:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.220+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68341a1590 con 0x7f6834100910 2023-09-23T09:29:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.221+0000 7f6830ff9700 1 -- 172.21.15.45:0/1343108502 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f681c016020 con 0x7f6834100910 2023-09-23T09:29:10.224 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:10.224 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:28:55.849323+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:29:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.223+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f68200a1670 msgr2=0x7f68200a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.223+0000 7f683a10d700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f68200a1670 0x7f68200a3b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6824009c90 tx=0x7f6824009260 comp rx=0 tx=0).stop 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.223+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 msgr2=0x7f6834102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.223+0000 7f683a10d700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f6834102340 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f681c00d980 tx=0x7f681c00dd40 comp rx=0 tx=0).stop 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 shutdown_connections 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f68200a1670 0x7f68200a3b30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834101390 0x7f6834102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68340fff50 0x7f6834103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 --2- 172.21.15.45:0/1343108502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6834100910 0x7f6834102340 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 >> 172.21.15.45:0/1343108502 conn(0x7f68340fa9e0 msgr2=0x7f68340fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 shutdown_connections 2023-09-23T09:29:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.224+0000 7f683a10d700 1 -- 172.21.15.45:0/1343108502 wait complete. 2023-09-23T09:29:10.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:29:10.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.369+0000 7f48ac997700 1 -- 172.21.15.45:0/2411222703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a81018a0 msgr2=0x7f48a8108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.369+0000 7f48ac997700 1 --2- 172.21.15.45:0/2411222703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a81018a0 0x7f48a8108150 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f48a000b0d0 tx=0x7f48a000b3e0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 -- 172.21.15.45:0/2411222703 shutdown_connections 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 --2- 172.21.15.45:0/2411222703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a81018a0 0x7f48a8108150 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 --2- 172.21.15.45:0/2411222703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48a80ffd10 0x7f48a8100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 --2- 172.21.15.45:0/2411222703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48a8100f00 0x7f48a8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 -- 172.21.15.45:0/2411222703 >> 172.21.15.45:0/2411222703 conn(0x7f48a80fb280 msgr2=0x7f48a80fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 -- 172.21.15.45:0/2411222703 shutdown_connections 2023-09-23T09:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 -- 172.21.15.45:0/2411222703 wait complete. 2023-09-23T09:29:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 Processor -- start 2023-09-23T09:29:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.370+0000 7f48ac997700 1 -- start start 2023-09-23T09:29:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48a80ffd10 0x7f48a8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a8100f00 0x7f48a8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48a81018a0 0x7f48a81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a81a09d0 con 0x7f48a8100f00 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a81a0b10 con 0x7f48a80ffd10 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a81a0ca0 con 0x7f48a81018a0 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a8100f00 0x7f48a8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a8100f00 0x7f48a8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60166/0 (socket says 172.21.15.45:60166) 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 -- 172.21.15.45:0/1237383596 learned_addr learned my addr 172.21.15.45:0/1237383596 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48a659c700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48a80ffd10 0x7f48a8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48a6d9d700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48a81018a0 0x7f48a81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 -- 172.21.15.45:0/1237383596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48a81018a0 msgr2=0x7f48a81a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48a81018a0 0x7f48a81a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 -- 172.21.15.45:0/1237383596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48a80ffd10 msgr2=0x7f48a8198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48a80ffd10 0x7f48a8198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 -- 172.21.15.45:0/1237383596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0009d00 con 0x7f48a8100f00 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489ffff700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a8100f00 0x7f48a8199390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f489000d950 tx=0x7f489000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489f7fe700 1 -- 172.21.15.45:0/1237383596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000f890 con 0x7f48a8100f00 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a81a0f80 con 0x7f48a8100f00 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489f7fe700 1 -- 172.21.15.45:0/1237383596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4890010460 con 0x7f48a8100f00 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f489f7fe700 1 -- 172.21.15.45:0/1237383596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000e610 con 0x7f48a8100f00 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.371+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a81a1400 con 0x7f48a8100f00 2023-09-23T09:29:10.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.372+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a810b580 con 0x7f48a8100f00 2023-09-23T09:29:10.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.374+0000 7f489f7fe700 1 -- 172.21.15.45:0/1237383596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f48900108d0 con 0x7f48a8100f00 2023-09-23T09:29:10.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.374+0000 7f489f7fe700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f48940a1630 0x7f48940a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.374+0000 7f489f7fe700 1 -- 172.21.15.45:0/1237383596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f48900cd1b0 con 0x7f48a8100f00 2023-09-23T09:29:10.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.374+0000 7f48a659c700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f48940a1630 0x7f48940a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.375+0000 7f48a659c700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f48940a1630 0x7f48940a3af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48a8100d60 tx=0x7f4898009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:10.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.378+0000 7f489f7fe700 1 -- 172.21.15.45:0/1237383596 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4890096640 con 0x7f48a8100f00 2023-09-23T09:29:10.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:10 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1343108502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.581+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48a81a15f0 con 0x7f48a8100f00 2023-09-23T09:29:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.581+0000 7f489f7fe700 1 -- 172.21.15.45:0/1237383596 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4890099c60 con 0x7f48a8100f00 2023-09-23T09:29:10.583 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:29:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f48940a1630 msgr2=0x7f48940a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f48940a1630 0x7f48940a3af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48a8100d60 tx=0x7f4898009260 comp rx=0 tx=0).stop 2023-09-23T09:29:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a8100f00 msgr2=0x7f48a8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a8100f00 0x7f48a8199390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f489000d950 tx=0x7f489000dc60 comp rx=0 tx=0).stop 2023-09-23T09:29:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 shutdown_connections 2023-09-23T09:29:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f48940a1630 0x7f48940a3af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48a81018a0 0x7f48a81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48a80ffd10 0x7f48a8198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 --2- 172.21.15.45:0/1237383596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48a8100f00 0x7f48a8199390 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 >> 172.21.15.45:0/1237383596 conn(0x7f48a80fb280 msgr2=0x7f48a8067fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 shutdown_connections 2023-09-23T09:29:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.583+0000 7f48ac997700 1 -- 172.21.15.45:0/1237383596 wait complete. 2023-09-23T09:29:10.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:29:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 -- 172.21.15.45:0/817978222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 msgr2=0x7ff064100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 --2- 172.21.15.45:0/817978222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064100130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff0580099c0 tx=0x7ff058009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 -- 172.21.15.45:0/817978222 shutdown_connections 2023-09-23T09:29:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 --2- 172.21.15.45:0/817978222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0641018a0 0x7ff064108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 --2- 172.21.15.45:0/817978222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064100130 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 --2- 172.21.15.45:0/817978222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064100f00 0x7ff064101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 -- 172.21.15.45:0/817978222 >> 172.21.15.45:0/817978222 conn(0x7ff0640fb280 msgr2=0x7ff0640fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 -- 172.21.15.45:0/817978222 shutdown_connections 2023-09-23T09:29:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 -- 172.21.15.45:0/817978222 wait complete. 2023-09-23T09:29:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.725+0000 7ff069832700 1 Processor -- start 2023-09-23T09:29:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff069832700 1 -- start start 2023-09-23T09:29:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff069832700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff069832700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064100f00 0x7ff064199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff069832700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0641018a0 0x7ff0641a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff069832700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641a0860 con 0x7ff0640ffd10 2023-09-23T09:29:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff069832700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641a09d0 con 0x7ff0641018a0 2023-09-23T09:29:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff069832700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641a0b90 con 0x7ff064100f00 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60178/0 (socket says 172.21.15.45:60178) 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 -- 172.21.15.45:0/3015831353 learned_addr learned my addr 172.21.15.45:0/3015831353 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff0627fc700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064100f00 0x7ff064199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff0637fe700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0641018a0 0x7ff0641a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 -- 172.21.15.45:0/3015831353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064100f00 msgr2=0x7ff064199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064100f00 0x7ff064199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 -- 172.21.15.45:0/3015831353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0641018a0 msgr2=0x7ff0641a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0641018a0 0x7ff0641a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff062ffd700 1 -- 172.21.15.45:0/3015831353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580095f0 con 0x7ff0640ffd10 2023-09-23T09:29:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.726+0000 7ff0637fe700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0641018a0 0x7ff0641a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.727+0000 7ff062ffd700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064198d20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff058009990 tx=0x7ff05800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.727+0000 7ff068830700 1 -- 172.21.15.45:0/3015831353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05801d070 con 0x7ff0640ffd10 2023-09-23T09:29:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.727+0000 7ff068830700 1 -- 172.21.15.45:0/3015831353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0580054f0 con 0x7ff0640ffd10 2023-09-23T09:29:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.727+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0641a0e10 con 0x7ff0640ffd10 2023-09-23T09:29:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.727+0000 7ff068830700 1 -- 172.21.15.45:0/3015831353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058004bd0 con 0x7ff0640ffd10 2023-09-23T09:29:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.727+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0641a1180 con 0x7ff0640ffd10 2023-09-23T09:29:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.728+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06410b570 con 0x7ff0640ffd10 2023-09-23T09:29:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.729+0000 7ff068830700 1 -- 172.21.15.45:0/3015831353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7ff05800fb80 con 0x7ff0640ffd10 2023-09-23T09:29:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.729+0000 7ff068830700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff04c0a1630 0x7ff04c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.730+0000 7ff0627fc700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff04c0a1630 0x7ff04c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.730+0000 7ff068830700 1 -- 172.21.15.45:0/3015831353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff0580cdf30 con 0x7ff0640ffd10 2023-09-23T09:29:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.730+0000 7ff0627fc700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff04c0a1630 0x7ff04c0a3af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff054009c60 tx=0x7ff054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.734+0000 7ff068830700 1 -- 172.21.15.45:0/3015831353 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff058097450 con 0x7ff0640ffd10 2023-09-23T09:29:10.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:10 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1343108502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:10.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:10 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1343108502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:10.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.935+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0641a1370 con 0x7ff0640ffd10 2023-09-23T09:29:10.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.936+0000 7ff068830700 1 -- 172.21.15.45:0/3015831353 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff05809aa70 con 0x7ff0640ffd10 2023-09-23T09:29:10.938 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff04c0a1630 msgr2=0x7ff04c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff04c0a1630 0x7ff04c0a3af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff054009c60 tx=0x7ff054009260 comp rx=0 tx=0).stop 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 msgr2=0x7ff064198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064198d20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff058009990 tx=0x7ff05800f690 comp rx=0 tx=0).stop 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 shutdown_connections 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0641018a0 0x7ff0641a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0640ffd10 0x7ff064198d20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff04c0a1630 0x7ff04c0a3af0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 --2- 172.21.15.45:0/3015831353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064100f00 0x7ff064199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 >> 172.21.15.45:0/3015831353 conn(0x7ff0640fb280 msgr2=0x7ff064067fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 shutdown_connections 2023-09-23T09:29:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:10.938+0000 7ff069832700 1 -- 172.21.15.45:0/3015831353 wait complete. 2023-09-23T09:29:10.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:29:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 -- 172.21.15.45:0/277258102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73480ffd10 msgr2=0x7f7348100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 --2- 172.21.15.45:0/277258102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73480ffd10 0x7f7348100130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f73380099c0 tx=0x7f7338009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 -- 172.21.15.45:0/277258102 shutdown_connections 2023-09-23T09:29:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 --2- 172.21.15.45:0/277258102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73481018a0 0x7f7348108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 --2- 172.21.15.45:0/277258102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73480ffd10 0x7f7348100130 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 --2- 172.21.15.45:0/277258102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7348100f00 0x7f7348101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 -- 172.21.15.45:0/277258102 >> 172.21.15.45:0/277258102 conn(0x7f73480fb280 msgr2=0x7f73480fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 -- 172.21.15.45:0/277258102 shutdown_connections 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.078+0000 7f73505d0700 1 -- 172.21.15.45:0/277258102 wait complete. 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 Processor -- start 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 -- start start 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73480ffd10 0x7f7348068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7348100f00 0x7f734819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481018a0 0x7f734819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734819e6e0 con 0x7f73481018a0 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734819e850 con 0x7f7348100f00 2023-09-23T09:29:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f73505d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734819ea10 con 0x7f73480ffd10 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481018a0 0x7f734819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481018a0 0x7f734819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60194/0 (socket says 172.21.15.45:60194) 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 -- 172.21.15.45:0/1583314066 learned_addr learned my addr 172.21.15.45:0/1583314066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734db6b700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7348100f00 0x7f734819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734e36c700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73480ffd10 0x7f7348068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 -- 172.21.15.45:0/1583314066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73480ffd10 msgr2=0x7f7348068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73480ffd10 0x7f7348068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 -- 172.21.15.45:0/1583314066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7348100f00 msgr2=0x7f734819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7348100f00 0x7f734819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 -- 172.21.15.45:0/1583314066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73380095f0 con 0x7f73481018a0 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734e36c700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73480ffd10 0x7f7348068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734db6b700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7348100f00 0x7f734819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.079+0000 7f734eb6d700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481018a0 0x7f734819df40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7340007ee0 tx=0x7f734000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.080+0000 7f733f7fe700 1 -- 172.21.15.45:0/1583314066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000ddd0 con 0x7f73481018a0 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.080+0000 7f733f7fe700 1 -- 172.21.15.45:0/1583314066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734000f040 con 0x7f73481018a0 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.080+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734819ecf0 con 0x7f73481018a0 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.080+0000 7f733f7fe700 1 -- 172.21.15.45:0/1583314066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340015400 con 0x7f73481018a0 2023-09-23T09:29:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.080+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734819f1b0 con 0x7f73481018a0 2023-09-23T09:29:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.081+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734810b570 con 0x7f73481018a0 2023-09-23T09:29:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.082+0000 7f733f7fe700 1 -- 172.21.15.45:0/1583314066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f7340004000 con 0x7f73481018a0 2023-09-23T09:29:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.082+0000 7f733f7fe700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f73340a1630 0x7f73340a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.083+0000 7f733f7fe700 1 -- 172.21.15.45:0/1583314066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f73400cd210 con 0x7f73481018a0 2023-09-23T09:29:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.083+0000 7f734e36c700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f73340a1630 0x7f73340a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.083+0000 7f734e36c700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f73340a1630 0x7f73340a3af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f73380098e0 tx=0x7f7338000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.086+0000 7f733f7fe700 1 -- 172.21.15.45:0/1583314066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f73400966a0 con 0x7f73481018a0 2023-09-23T09:29:11.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.288+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7348106c90 con 0x7f73481018a0 2023-09-23T09:29:11.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.288+0000 7f733f7fe700 1 -- 172.21.15.45:0/1583314066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7340023020 con 0x7f73481018a0 2023-09-23T09:29:11.290 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f73340a1630 msgr2=0x7f73340a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f73340a1630 0x7f73340a3af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f73380098e0 tx=0x7f7338000bc0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481018a0 msgr2=0x7f734819df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481018a0 0x7f734819df40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7340007ee0 tx=0x7f734000d3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 shutdown_connections 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481018a0 0x7f734819df40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f73340a1630 0x7f73340a3af0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73480ffd10 0x7f7348068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 --2- 172.21.15.45:0/1583314066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7348100f00 0x7f734819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 >> 172.21.15.45:0/1583314066 conn(0x7f73480fb280 msgr2=0x7f73480fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 shutdown_connections 2023-09-23T09:29:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.291+0000 7f73505d0700 1 -- 172.21.15.45:0/1583314066 wait complete. 2023-09-23T09:29:11.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:29:11.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 -- 172.21.15.45:0/3157097719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65481012a0 msgr2=0x7f65481016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 --2- 172.21.15.45:0/3157097719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65481012a0 0x7f65481016c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f653c009990 tx=0x7f653c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 -- 172.21.15.45:0/3157097719 shutdown_connections 2023-09-23T09:29:11.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 --2- 172.21.15.45:0/3157097719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65481008b0 0x7f6548108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 --2- 172.21.15.45:0/3157097719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65481012a0 0x7f65481016c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 --2- 172.21.15.45:0/3157097719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65480ffef0 0x7f6548100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 -- 172.21.15.45:0/3157097719 >> 172.21.15.45:0/3157097719 conn(0x7f65480fa960 msgr2=0x7f65480fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 -- 172.21.15.45:0/3157097719 shutdown_connections 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.430+0000 7f654f119700 1 -- 172.21.15.45:0/3157097719 wait complete. 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 Processor -- start 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 -- start start 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65480ffef0 0x7f6548196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65481008b0 0x7f6548197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65481012a0 0x7f654819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654819e660 con 0x7f65480ffef0 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654819e7a0 con 0x7f65481008b0 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654f119700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654819e930 con 0x7f65481012a0 2023-09-23T09:29:11.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65480ffef0 0x7f6548196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65480ffef0 0x7f6548196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60204/0 (socket says 172.21.15.45:60204) 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 -- 172.21.15.45:0/4078895027 learned_addr learned my addr 172.21.15.45:0/4078895027 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f6547fff700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65481008b0 0x7f6548197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 -- 172.21.15.45:0/4078895027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65481012a0 msgr2=0x7f654819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654d6b6700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65481012a0 0x7f654819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65481012a0 0x7f654819e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 -- 172.21.15.45:0/4078895027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65481008b0 msgr2=0x7f6548197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65481008b0 0x7f6548197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 -- 172.21.15.45:0/4078895027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c0095f0 con 0x7f65480ffef0 2023-09-23T09:29:11.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.431+0000 7f654ceb5700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65480ffef0 0x7f6548196ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f653c0098e0 tx=0x7f653c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:11.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.432+0000 7f6545ffb700 1 -- 172.21.15.45:0/4078895027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c01d070 con 0x7f65480ffef0 2023-09-23T09:29:11.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.432+0000 7f6545ffb700 1 -- 172.21.15.45:0/4078895027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f653c0054f0 con 0x7f65480ffef0 2023-09-23T09:29:11.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.432+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654819ebb0 con 0x7f65480ffef0 2023-09-23T09:29:11.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.432+0000 7f6545ffb700 1 -- 172.21.15.45:0/4078895027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c004ca0 con 0x7f65480ffef0 2023-09-23T09:29:11.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.432+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654819efd0 con 0x7f65480ffef0 2023-09-23T09:29:11.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.433+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654810b4e0 con 0x7f65480ffef0 2023-09-23T09:29:11.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.434+0000 7f6545ffb700 1 -- 172.21.15.45:0/4078895027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f653c00fc70 con 0x7f65480ffef0 2023-09-23T09:29:11.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.435+0000 7f6545ffb700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f65380a16f0 0x7f65380a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.435+0000 7f6545ffb700 1 -- 172.21.15.45:0/4078895027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f653c0ce830 con 0x7f65480ffef0 2023-09-23T09:29:11.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.435+0000 7f6547fff700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f65380a16f0 0x7f65380a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.435+0000 7f6547fff700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f65380a16f0 0x7f65380a3bb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6534005950 tx=0x7f653400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:11.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.438+0000 7f6545ffb700 1 -- 172.21.15.45:0/4078895027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f653c097d50 con 0x7f65480ffef0 2023-09-23T09:29:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:11 smithi045 ceph-mon[98627]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 10 B/s, 0 objects/s recovering 2023-09-23T09:29:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:11 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1237383596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:11 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3015831353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:11 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1583314066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.642+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f654819f220 con 0x7f65480ffef0 2023-09-23T09:29:11.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.643+0000 7f6545ffb700 1 -- 172.21.15.45:0/4078895027 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f653c09b370 con 0x7f65480ffef0 2023-09-23T09:29:11.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:11.645 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:28:55.849323+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f65380a16f0 msgr2=0x7f65380a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f65380a16f0 0x7f65380a3bb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6534005950 tx=0x7f653400b410 comp rx=0 tx=0).stop 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65480ffef0 msgr2=0x7f6548196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65480ffef0 0x7f6548196ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f653c0098e0 tx=0x7f653c004730 comp rx=0 tx=0).stop 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 shutdown_connections 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f65380a16f0 0x7f65380a3bb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65481012a0 0x7f654819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65480ffef0 0x7f6548196ae0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 --2- 172.21.15.45:0/4078895027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65481008b0 0x7f6548197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 >> 172.21.15.45:0/4078895027 conn(0x7f65480fa960 msgr2=0x7f65480fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 shutdown_connections 2023-09-23T09:29:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.646+0000 7f654f119700 1 -- 172.21.15.45:0/4078895027 wait complete. 2023-09-23T09:29:11.660 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:29:11.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:29:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 -- 172.21.15.45:0/783125422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 msgr2=0x7f69a8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 --2- 172.21.15.45:0/783125422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a8108f60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f69a000b0a0 tx=0x7f69a000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 -- 172.21.15.45:0/783125422 shutdown_connections 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 --2- 172.21.15.45:0/783125422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a8108f60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 --2- 172.21.15.45:0/783125422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a81018f0 0x7f69a8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 --2- 172.21.15.45:0/783125422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69a8104220 0x7f69a8106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 -- 172.21.15.45:0/783125422 >> 172.21.15.45:0/783125422 conn(0x7f69a80fb280 msgr2=0x7f69a80fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 -- 172.21.15.45:0/783125422 shutdown_connections 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.788+0000 7f69af3b8700 1 -- 172.21.15.45:0/783125422 wait complete. 2023-09-23T09:29:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 Processor -- start 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 -- start start 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a81018f0 0x7f69a8196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69a8104220 0x7f69a8197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a819e6d0 con 0x7f69a8106b50 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a819e810 con 0x7f69a8104220 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69af3b8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a819e9a0 con 0x7f69a81018f0 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a819e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60222/0 (socket says 172.21.15.45:60222) 2023-09-23T09:29:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 -- 172.21.15.45:0/19291852 learned_addr learned my addr 172.21.15.45:0/19291852 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad154700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a81018f0 0x7f69a8196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ac953700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69a8104220 0x7f69a8197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 -- 172.21.15.45:0/19291852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a81018f0 msgr2=0x7f69a8196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a81018f0 0x7f69a8196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 -- 172.21.15.45:0/19291852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69a8104220 msgr2=0x7f69a8197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69a8104220 0x7f69a8197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.789+0000 7f69ad955700 1 -- 172.21.15.45:0/19291852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a0009d00 con 0x7f69a8106b50 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.790+0000 7f69ad955700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a819e100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f69a000b780 tx=0x7f69a000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.790+0000 7f699e7fc700 1 -- 172.21.15.45:0/19291852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a000e040 con 0x7f69a8106b50 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.790+0000 7f699e7fc700 1 -- 172.21.15.45:0/19291852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69a00094a0 con 0x7f69a8106b50 2023-09-23T09:29:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.790+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a819ec20 con 0x7f69a8106b50 2023-09-23T09:29:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.790+0000 7f699e7fc700 1 -- 172.21.15.45:0/19291852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0005330 con 0x7f69a8106b50 2023-09-23T09:29:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.790+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a819f0a0 con 0x7f69a8106b50 2023-09-23T09:29:11.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:11 smithi098 ceph-mon[102386]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 10 B/s, 0 objects/s recovering 2023-09-23T09:29:11.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:11 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1237383596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:11 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3015831353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:11 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1583314066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.791+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a80fce00 con 0x7f69a8106b50 2023-09-23T09:29:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.792+0000 7f699e7fc700 1 -- 172.21.15.45:0/19291852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f69a001da30 con 0x7f69a8106b50 2023-09-23T09:29:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.793+0000 7f699e7fc700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69940a16b0 0x7f69940a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.793+0000 7f699e7fc700 1 -- 172.21.15.45:0/19291852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f69a00cebc0 con 0x7f69a8106b50 2023-09-23T09:29:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.793+0000 7f69ad154700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69940a16b0 0x7f69940a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.793+0000 7f69ad154700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69940a16b0 0x7f69940a3b70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6998009c60 tx=0x7f6998009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.796+0000 7f699e7fc700 1 -- 172.21.15.45:0/19291852 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f69a00980e0 con 0x7f69a8106b50 2023-09-23T09:29:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:11 smithi123 ceph-mon[103577]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 10 B/s, 0 objects/s recovering 2023-09-23T09:29:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:11 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1237383596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:11 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3015831353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:11 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1583314066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.999+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69a810c840 con 0x7f69a8106b50 2023-09-23T09:29:12.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:11.999+0000 7f699e7fc700 1 -- 172.21.15.45:0/19291852 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69a009b700 con 0x7f69a8106b50 2023-09-23T09:29:12.001 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:29:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69940a16b0 msgr2=0x7f69940a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69940a16b0 0x7f69940a3b70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6998009c60 tx=0x7f6998009260 comp rx=0 tx=0).stop 2023-09-23T09:29:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 msgr2=0x7f69a819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a819e100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f69a000b780 tx=0x7f69a000bd60 comp rx=0 tx=0).stop 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 shutdown_connections 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a8106b50 0x7f69a819e100 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f69940a16b0 0x7f69940a3b70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a81018f0 0x7f69a8196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 --2- 172.21.15.45:0/19291852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69a8104220 0x7f69a8197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 >> 172.21.15.45:0/19291852 conn(0x7f69a80fb280 msgr2=0x7f69a8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 shutdown_connections 2023-09-23T09:29:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.001+0000 7f69af3b8700 1 -- 172.21.15.45:0/19291852 wait complete. 2023-09-23T09:29:12.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 -- 172.21.15.45:0/3182504131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 msgr2=0x7f5f38108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 --2- 172.21.15.45:0/3182504131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f38108f60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b0a0 tx=0x7f5f3000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 -- 172.21.15.45:0/3182504131 shutdown_connections 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 --2- 172.21.15.45:0/3182504131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f38108f60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 --2- 172.21.15.45:0/3182504131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f381018f0 0x7f5f38103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 --2- 172.21.15.45:0/3182504131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f38104220 0x7f5f38106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 -- 172.21.15.45:0/3182504131 >> 172.21.15.45:0/3182504131 conn(0x7f5f380fb280 msgr2=0x7f5f380fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 -- 172.21.15.45:0/3182504131 shutdown_connections 2023-09-23T09:29:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.143+0000 7f5f40692700 1 -- 172.21.15.45:0/3182504131 wait complete. 2023-09-23T09:29:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 Processor -- start 2023-09-23T09:29:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 -- start start 2023-09-23T09:29:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f381018f0 0x7f5f38198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f38104220 0x7f5f38199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f381a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381a09a0 con 0x7f5f38106b50 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381a0ae0 con 0x7f5f381018f0 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f40692700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381a0c70 con 0x7f5f38104220 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f381a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f381a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60236/0 (socket says 172.21.15.45:60236) 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 -- 172.21.15.45:0/3774518838 learned_addr learned my addr 172.21.15.45:0/3774518838 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3e42e700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f381018f0 0x7f5f38198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3dc2d700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f38104220 0x7f5f38199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 -- 172.21.15.45:0/3774518838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f38104220 msgr2=0x7f5f38199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f38104220 0x7f5f38199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 -- 172.21.15.45:0/3774518838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f381018f0 msgr2=0x7f5f38198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f381018f0 0x7f5f38198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3ec2f700 1 -- 172.21.15.45:0/3774518838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30009d00 con 0x7f5f38106b50 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.144+0000 7f5f3e42e700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f381018f0 0x7f5f38198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.145+0000 7f5f3ec2f700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f381a03d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b780 tx=0x7f5f3000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.145+0000 7f5f2f7fe700 1 -- 172.21.15.45:0/3774518838 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000e040 con 0x7f5f38106b50 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.145+0000 7f5f2f7fe700 1 -- 172.21.15.45:0/3774518838 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f30004c50 con 0x7f5f38106b50 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.145+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f381a0ef0 con 0x7f5f38106b50 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.145+0000 7f5f2f7fe700 1 -- 172.21.15.45:0/3774518838 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300053b0 con 0x7f5f38106b50 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.145+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f381a1310 con 0x7f5f38106b50 2023-09-23T09:29:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.146+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f381086d0 con 0x7f5f38106b50 2023-09-23T09:29:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.147+0000 7f5f2f7fe700 1 -- 172.21.15.45:0/3774518838 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f5f3001d9c0 con 0x7f5f38106b50 2023-09-23T09:29:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.147+0000 7f5f2f7fe700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5f240a1630 0x7f5f240a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.148+0000 7f5f3e42e700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5f240a1630 0x7f5f240a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.148+0000 7f5f2f7fe700 1 -- 172.21.15.45:0/3774518838 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5f300ce9a0 con 0x7f5f38106b50 2023-09-23T09:29:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.148+0000 7f5f3e42e700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5f240a1630 0x7f5f240a3af0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5f34011490 tx=0x7f5f34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.151+0000 7f5f2f7fe700 1 -- 172.21.15.45:0/3774518838 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5f30097ec0 con 0x7f5f38106b50 2023-09-23T09:29:12.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.352+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f3810c840 con 0x7f5f38106b50 2023-09-23T09:29:12.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.352+0000 7f5f2f7fe700 1 -- 172.21.15.45:0/3774518838 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f3009b4e0 con 0x7f5f38106b50 2023-09-23T09:29:12.354 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:29:12.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.354+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5f240a1630 msgr2=0x7f5f240a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.354+0000 7f5f40692700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5f240a1630 0x7f5f240a3af0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5f34011490 tx=0x7f5f34009260 comp rx=0 tx=0).stop 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.354+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 msgr2=0x7f5f381a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.354+0000 7f5f40692700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f381a03d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b780 tx=0x7f5f3000bab0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 shutdown_connections 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f38106b50 0x7f5f381a03d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f381018f0 0x7f5f38198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f5f240a1630 0x7f5f240a3af0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 --2- 172.21.15.45:0/3774518838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f38104220 0x7f5f38199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 >> 172.21.15.45:0/3774518838 conn(0x7f5f380fb280 msgr2=0x7f5f380ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 shutdown_connections 2023-09-23T09:29:12.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.355+0000 7f5f40692700 1 -- 172.21.15.45:0/3774518838 wait complete. 2023-09-23T09:29:12.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:29:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.499+0000 7fb52860f700 1 -- 172.21.15.45:0/510263818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5200fff40 msgr2=0x7fb5201003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.499+0000 7fb52860f700 1 --2- 172.21.15.45:0/510263818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5200fff40 0x7fb5201003c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb510009990 tx=0x7fb510009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- 172.21.15.45:0/510263818 shutdown_connections 2023-09-23T09:29:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 --2- 172.21.15.45:0/510263818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb520100900 0x7fb5201081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 --2- 172.21.15.45:0/510263818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb520101380 0x7fb5201017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 --2- 172.21.15.45:0/510263818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5200fff40 0x7fb5201003c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- 172.21.15.45:0/510263818 >> 172.21.15.45:0/510263818 conn(0x7fb5200fa9f0 msgr2=0x7fb5200fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- 172.21.15.45:0/510263818 shutdown_connections 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- 172.21.15.45:0/510263818 wait complete. 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 Processor -- start 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- start start 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5200fff40 0x7fb520103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb520100900 0x7fb520102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb520101380 0x7fb520102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520103010 con 0x7fb520101380 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520103180 con 0x7fb520100900 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.500+0000 7fb52860f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201a0b70 con 0x7fb5200fff40 2023-09-23T09:29:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb520101380 0x7fb520102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb520101380 0x7fb520102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60250/0 (socket says 172.21.15.45:60250) 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 -- 172.21.15.45:0/151882646 learned_addr learned my addr 172.21.15.45:0/151882646 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb5263ab700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5200fff40 0x7fb520103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb525baa700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb520100900 0x7fb520102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 -- 172.21.15.45:0/151882646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5200fff40 msgr2=0x7fb520103cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5200fff40 0x7fb520103cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 -- 172.21.15.45:0/151882646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb520100900 msgr2=0x7fb520102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb520100900 0x7fb520102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 -- 172.21.15.45:0/151882646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100095f0 con 0x7fb520101380 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb526bac700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb520101380 0x7fb520102870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb518007d70 tx=0x7fb51800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb5177fe700 1 -- 172.21.15.45:0/151882646 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51800ad40 con 0x7fb520101380 2023-09-23T09:29:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb5177fe700 1 -- 172.21.15.45:0/151882646 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5180128e0 con 0x7fb520101380 2023-09-23T09:29:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5201a0e50 con 0x7fb520101380 2023-09-23T09:29:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb5177fe700 1 -- 172.21.15.45:0/151882646 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518011780 con 0x7fb520101380 2023-09-23T09:29:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.501+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5201a1280 con 0x7fb520101380 2023-09-23T09:29:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.502+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52010b510 con 0x7fb520101380 2023-09-23T09:29:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.503+0000 7fb5177fe700 1 -- 172.21.15.45:0/151882646 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fb518012a50 con 0x7fb520101380 2023-09-23T09:29:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.504+0000 7fb5177fe700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb50c0a1630 0x7fb50c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.504+0000 7fb5177fe700 1 -- 172.21.15.45:0/151882646 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb5180cd340 con 0x7fb520101380 2023-09-23T09:29:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.504+0000 7fb5263ab700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb50c0a1630 0x7fb50c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.504+0000 7fb5263ab700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb50c0a1630 0x7fb50c0a3af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00bbb0 tx=0x7fb51c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:12.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.508+0000 7fb5177fe700 1 -- 172.21.15.45:0/151882646 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb5180968a0 con 0x7fb520101380 2023-09-23T09:29:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.711+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5201a1540 con 0x7fb520101380 2023-09-23T09:29:12.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.711+0000 7fb5177fe700 1 -- 172.21.15.45:0/151882646 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb518096030 con 0x7fb520101380 2023-09-23T09:29:12.713 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:29:12.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb50c0a1630 msgr2=0x7fb50c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb50c0a1630 0x7fb50c0a3af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00bbb0 tx=0x7fb51c00b590 comp rx=0 tx=0).stop 2023-09-23T09:29:12.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb520101380 msgr2=0x7fb520102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb520101380 0x7fb520102870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb518007d70 tx=0x7fb51800a340 comp rx=0 tx=0).stop 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 shutdown_connections 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb520101380 0x7fb520102870 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fb50c0a1630 0x7fb50c0a3af0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5200fff40 0x7fb520103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 --2- 172.21.15.45:0/151882646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb520100900 0x7fb520102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 >> 172.21.15.45:0/151882646 conn(0x7fb5200fa9f0 msgr2=0x7fb5200fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 shutdown_connections 2023-09-23T09:29:12.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.713+0000 7fb52860f700 1 -- 172.21.15.45:0/151882646 wait complete. 2023-09-23T09:29:12.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/4078895027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:12.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/19291852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:12.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3774518838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:12.727 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-23T09:29:12.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:29:12.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/4078895027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:12.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/19291852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:12.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3774518838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:12.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/4078895027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:12.866 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/19291852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:12.866 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3774518838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.851+0000 7f41e1521700 1 -- 172.21.15.45:0/1438614413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102930 msgr2=0x7f41dc109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.851+0000 7f41e1521700 1 --2- 172.21.15.45:0/1438614413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102930 0x7f41dc109250 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f41d400b0a0 tx=0x7f41d400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.851+0000 7f41e1521700 1 -- 172.21.15.45:0/1438614413 shutdown_connections 2023-09-23T09:29:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.851+0000 7f41e1521700 1 --2- 172.21.15.45:0/1438614413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102930 0x7f41dc109250 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.851+0000 7f41e1521700 1 --2- 172.21.15.45:0/1438614413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc100e20 0x7f41dc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.851+0000 7f41e1521700 1 --2- 172.21.15.45:0/1438614413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41dc101f70 0x7f41dc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.851+0000 7f41e1521700 1 -- 172.21.15.45:0/1438614413 >> 172.21.15.45:0/1438614413 conn(0x7f41dc0fc370 msgr2=0x7f41dc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 -- 172.21.15.45:0/1438614413 shutdown_connections 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 -- 172.21.15.45:0/1438614413 wait complete. 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 Processor -- start 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 -- start start 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc100e20 0x7f41dc074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc101f70 0x7f41dc073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41dc102930 0x7f41dc073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc073ef0 con 0x7f41dc100e20 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc074060 con 0x7f41dc102930 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.852+0000 7f41e1521700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc1a0c30 con 0x7f41dc101f70 2023-09-23T09:29:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc100e20 0x7f41dc074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc100e20 0x7f41dc074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60260/0 (socket says 172.21.15.45:60260) 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 -- 172.21.15.45:0/3731521395 learned_addr learned my addr 172.21.15.45:0/3731521395 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41da7fc700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc101f70 0x7f41dc073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41db7fe700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41dc102930 0x7f41dc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 -- 172.21.15.45:0/3731521395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc101f70 msgr2=0x7f41dc073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc101f70 0x7f41dc073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 -- 172.21.15.45:0/3731521395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41dc102930 msgr2=0x7f41dc073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41dc102930 0x7f41dc073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 -- 172.21.15.45:0/3731521395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d4009d00 con 0x7f41dc100e20 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41daffd700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc100e20 0x7f41dc074bd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00d950 tx=0x7f41cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41c3fff700 1 -- 172.21.15.45:0/3731521395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc009940 con 0x7f41dc100e20 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc1a0f10 con 0x7f41dc100e20 2023-09-23T09:29:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41c3fff700 1 -- 172.21.15.45:0/3731521395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41cc010460 con 0x7f41dc100e20 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41c3fff700 1 -- 172.21.15.45:0/3731521395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00f620 con 0x7f41dc100e20 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.853+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc1a1330 con 0x7f41dc100e20 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.854+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc108350 con 0x7f41dc100e20 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.855+0000 7f41c3fff700 1 -- 172.21.15.45:0/3731521395 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f41cc0108d0 con 0x7f41dc100e20 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.856+0000 7f41c3fff700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f41c40a1670 0x7f41c40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.856+0000 7f41c3fff700 1 -- 172.21.15.45:0/3731521395 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f41cc0ccf70 con 0x7f41dc100e20 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.856+0000 7f41da7fc700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f41c40a1670 0x7f41c40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.856+0000 7f41da7fc700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f41c40a1670 0x7f41c40a3b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009c60 tx=0x7f41d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:12.859+0000 7f41c3fff700 1 -- 172.21.15.45:0/3731521395 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f41cc0963c0 con 0x7f41dc100e20 2023-09-23T09:29:13.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.102+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41dc02d270 con 0x7f41dc100e20 2023-09-23T09:29:13.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.103+0000 7f41c3fff700 1 -- 172.21.15.45:0/3731521395 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7f41cc020090 con 0x7f41dc100e20 2023-09-23T09:29:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:13.105 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:29:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f41c40a1670 msgr2=0x7f41c40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f41c40a1670 0x7f41c40a3b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009c60 tx=0x7f41d0009260 comp rx=0 tx=0).stop 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc100e20 msgr2=0x7f41dc074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc100e20 0x7f41dc074bd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00d950 tx=0x7f41cc00dc60 comp rx=0 tx=0).stop 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 shutdown_connections 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41dc102930 0x7f41dc073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc100e20 0x7f41dc074bd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f41c40a1670 0x7f41c40a3b30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 --2- 172.21.15.45:0/3731521395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc101f70 0x7f41dc073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 >> 172.21.15.45:0/3731521395 conn(0x7f41dc0fc370 msgr2=0x7f41dc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 shutdown_connections 2023-09-23T09:29:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.105+0000 7f41e1521700 1 -- 172.21.15.45:0/3731521395 wait complete. 2023-09-23T09:29:13.109 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:29:13.120 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-23T09:29:13.120 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-23T09:29:13.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T09:29:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.243+0000 7ff1271bb700 1 -- 172.21.15.45:0/1883884242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff120101880 msgr2=0x7ff120103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.243+0000 7ff1271bb700 1 --2- 172.21.15.45:0/1883884242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff120101880 0x7ff120103c70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff114009990 tx=0x7ff114009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- 172.21.15.45:0/1883884242 shutdown_connections 2023-09-23T09:29:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 --2- 172.21.15.45:0/1883884242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff120106ae0 0x7ff120108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 --2- 172.21.15.45:0/1883884242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff120101880 0x7ff120103c70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 --2- 172.21.15.45:0/1883884242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1201041b0 0x7ff1201065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- 172.21.15.45:0/1883884242 >> 172.21.15.45:0/1883884242 conn(0x7ff1200fb250 msgr2=0x7ff1200fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- 172.21.15.45:0/1883884242 shutdown_connections 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- 172.21.15.45:0/1883884242 wait complete. 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 Processor -- start 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- start start 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff120101880 0x7ff1201011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1201041b0 0x7ff1200ff830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff120106ae0 0x7ff1200ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120100510 con 0x7ff1201041b0 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120100680 con 0x7ff120106ae0 2023-09-23T09:29:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff1271bb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201a0b10 con 0x7ff120101880 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.244+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1201041b0 0x7ff1200ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1201041b0 0x7ff1200ff830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60276/0 (socket says 172.21.15.45:60276) 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 -- 172.21.15.45:0/2407165993 learned_addr learned my addr 172.21.15.45:0/2407165993 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff124f57700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff120101880 0x7ff1201011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff125758700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff120106ae0 0x7ff1200ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 -- 172.21.15.45:0/2407165993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff120101880 msgr2=0x7ff1201011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff120101880 0x7ff1201011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 -- 172.21.15.45:0/2407165993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff120106ae0 msgr2=0x7ff1200ffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff120106ae0 0x7ff1200ffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 -- 172.21.15.45:0/2407165993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1140095f0 con 0x7ff1201041b0 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11ffff700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1201041b0 0x7ff1200ff830 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00e9a0 tx=0x7ff10c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11dffb700 1 -- 172.21.15.45:0/2407165993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c009790 con 0x7ff1201041b0 2023-09-23T09:29:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11dffb700 1 -- 172.21.15.45:0/2407165993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff10c010430 con 0x7ff1201041b0 2023-09-23T09:29:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1201a0df0 con 0x7ff1201041b0 2023-09-23T09:29:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff11dffb700 1 -- 172.21.15.45:0/2407165993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c018680 con 0x7ff1201041b0 2023-09-23T09:29:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.245+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1201a1270 con 0x7ff1201041b0 2023-09-23T09:29:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.246+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12010c630 con 0x7ff1201041b0 2023-09-23T09:29:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.247+0000 7ff11dffb700 1 -- 172.21.15.45:0/2407165993 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7ff10c0105a0 con 0x7ff1201041b0 2023-09-23T09:29:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.248+0000 7ff11dffb700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff1100a1670 0x7ff1100a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.248+0000 7ff11dffb700 1 -- 172.21.15.45:0/2407165993 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff10c014070 con 0x7ff1201041b0 2023-09-23T09:29:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.248+0000 7ff124f57700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff1100a1670 0x7ff1100a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.249+0000 7ff124f57700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff1100a1670 0x7ff1100a3b30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b2e0 tx=0x7ff114005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:13.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.252+0000 7ff11dffb700 1 -- 172.21.15.45:0/2407165993 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff10c051ec0 con 0x7ff1201041b0 2023-09-23T09:29:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.452+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff120103460 con 0x7ff1100a1670 2023-09-23T09:29:13.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.465+0000 7ff11dffb700 1 -- 172.21.15.45:0/2407165993 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344355 (secure 0 0 0) 0x7ff120103460 con 0x7ff1100a1670 2023-09-23T09:29:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:13.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff1100a1670 msgr2=0x7ff1100a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff1100a1670 0x7ff1100a3b30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b2e0 tx=0x7ff114005fb0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1201041b0 msgr2=0x7ff1200ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1201041b0 0x7ff1200ff830 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00e9a0 tx=0x7ff10c00ecb0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 shutdown_connections 2023-09-23T09:29:13.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff120106ae0 0x7ff1200ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7ff1100a1670 0x7ff1100a3b30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff120101880 0x7ff1201011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 --2- 172.21.15.45:0/2407165993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1201041b0 0x7ff1200ff830 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 >> 172.21.15.45:0/2407165993 conn(0x7ff1200fb250 msgr2=0x7ff120102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:13.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 shutdown_connections 2023-09-23T09:29:13.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.468+0000 7ff1271bb700 1 -- 172.21.15.45:0/2407165993 wait complete. 2023-09-23T09:29:13.474 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-09-23T09:29:13.482 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-09-23T09:29:12.342399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651998,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":606,"num_read_kb":419,"num_write":334,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180859,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335512,"kb_used_data":11260,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606312,"statfs":{"total":1151940427776,"available":1151596863488,"internally_reserved":0,"allocated":11530240,"data_stored":7285610,"data_compressed":178287,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856847+0000","last_change":"2023-09-23T09:27:50.270599+0000","last_active":"2023-09-23T09:28:54.856847+0000","last_peered":"2023-09-23T09:28:54.856847+0000","last_clean":"2023-09-23T09:28:54.856847+0000","last_became_active":"2023-09-23T09:27:50.269926+0000","last_became_peered":"2023-09-23T09:27:50.269926+0000","last_unstale":"2023-09-23T09:28:54.856847+0000","last_undegraded":"2023-09-23T09:28:54.856847+0000","last_fullsized":"2023-09-23T09:28:54.856847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:12:08.908188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858982+0000","last_change":"2023-09-23T09:27:50.270451+0000","last_active":"2023-09-23T09:28:54.858982+0000","last_peered":"2023-09-23T09:28:54.858982+0000","last_clean":"2023-09-23T09:28:54.858982+0000","last_became_active":"2023-09-23T09:27:50.270055+0000","last_became_peered":"2023-09-23T09:27:50.270055+0000","last_unstale":"2023-09-23T09:28:54.858982+0000","last_undegraded":"2023-09-23T09:28:54.858982+0000","last_fullsized":"2023-09-23T09:28:54.858982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:56:43.388109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859067+0000","last_change":"2023-09-23T09:27:50.278101+0000","last_active":"2023-09-23T09:28:54.859067+0000","last_peered":"2023-09-23T09:28:54.859067+0000","last_clean":"2023-09-23T09:28:54.859067+0000","last_became_active":"2023-09-23T09:27:50.277739+0000","last_became_peered":"2023-09-23T09:27:50.277739+0000","last_unstale":"2023-09-23T09:28:54.859067+0000","last_undegraded":"2023-09-23T09:28:54.859067+0000","last_fullsized":"2023-09-23T09:28:54.859067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:52:31.582623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853757+0000","last_change":"2023-09-23T09:27:50.265962+0000","last_active":"2023-09-23T09:28:54.853757+0000","last_peered":"2023-09-23T09:28:54.853757+0000","last_clean":"2023-09-23T09:28:54.853757+0000","last_became_active":"2023-09-23T09:27:50.265681+0000","last_became_peered":"2023-09-23T09:27:50.265681+0000","last_unstale":"2023-09-23T09:28:54.853757+0000","last_undegraded":"2023-09-23T09:28:54.853757+0000","last_fullsized":"2023-09-23T09:28:54.853757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:12.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854809+0000","last_change":"2023-09-23T09:27:50.270356+0000","last_active":"2023-09-23T09:28:54.854809+0000","last_peered":"2023-09-23T09:28:54.854809+0000","last_clean":"2023-09-23T09:28:54.854809+0000","last_became_active":"2023-09-23T09:27:50.270124+0000","last_became_peered":"2023-09-23T09:27:50.270124+0000","last_unstale":"2023-09-23T09:28:54.854809+0000","last_undegraded":"2023-09-23T09:28:54.854809+0000","last_fullsized":"2023-09-23T09:28:54.854809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:14:00.508926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851625+0000","last_change":"2023-09-23T09:27:50.276122+0000","last_active":"2023-09-23T09:28:54.851625+0000","last_peered":"2023-09-23T09:28:54.851625+0000","last_clean":"2023-09-23T09:28:54.851625+0000","last_became_active":"2023-09-23T09:27:50.275847+0000","last_became_peered":"2023-09-23T09:27:50.275847+0000","last_unstale":"2023-09-23T09:28:54.851625+0000","last_undegraded":"2023-09-23T09:28:54.851625+0000","last_fullsized":"2023-09-23T09:28:54.851625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:04:03.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855358+0000","last_change":"2023-09-23T09:27:50.279420+0000","last_active":"2023-09-23T09:28:54.855358+0000","last_peered":"2023-09-23T09:28:54.855358+0000","last_clean":"2023-09-23T09:28:54.855358+0000","last_became_active":"2023-09-23T09:27:50.279120+0000","last_became_peered":"2023-09-23T09:27:50.279120+0000","last_unstale":"2023-09-23T09:28:54.855358+0000","last_undegraded":"2023-09-23T09:28:54.855358+0000","last_fullsized":"2023-09-23T09:28:54.855358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:48:03.461868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857315+0000","last_change":"2023-09-23T09:27:50.276364+0000","last_active":"2023-09-23T09:28:54.857315+0000","last_peered":"2023-09-23T09:28:54.857315+0000","last_clean":"2023-09-23T09:28:54.857315+0000","last_became_active":"2023-09-23T09:27:50.276060+0000","last_became_peered":"2023-09-23T09:27:50.276060+0000","last_unstale":"2023-09-23T09:28:54.857315+0000","last_undegraded":"2023-09-23T09:28:54.857315+0000","last_fullsized":"2023-09-23T09:28:54.857315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:26.998609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854943+0000","last_change":"2023-09-23T09:27:50.280302+0000","last_active":"2023-09-23T09:28:54.854943+0000","last_peered":"2023-09-23T09:28:54.854943+0000","last_clean":"2023-09-23T09:28:54.854943+0000","last_became_active":"2023-09-23T09:27:50.279295+0000","last_became_peered":"2023-09-23T09:27:50.279295+0000","last_unstale":"2023-09-23T09:28:54.854943+0000","last_undegraded":"2023-09-23T09:28:54.854943+0000","last_fullsized":"2023-09-23T09:28:54.854943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:15.329483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851997+0000","last_change":"2023-09-23T09:27:50.271876+0000","last_active":"2023-09-23T09:28:54.851997+0000","last_peered":"2023-09-23T09:28:54.851997+0000","last_clean":"2023-09-23T09:28:54.851997+0000","last_became_active":"2023-09-23T09:27:50.271576+0000","last_became_peered":"2023-09-23T09:27:50.271576+0000","last_unstale":"2023-09-23T09:28:54.851997+0000","last_undegraded":"2023-09-23T09:28:54.851997+0000","last_fullsized":"2023-09-23T09:28:54.851997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:17:22.656828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855756+0000","last_change":"2023-09-23T09:27:50.271405+0000","last_active":"2023-09-23T09:28:54.855756+0000","last_peered":"2023-09-23T09:28:54.855756+0000","last_clean":"2023-09-23T09:28:54.855756+0000","last_became_active":"2023-09-23T09:27:50.271087+0000","last_became_peered":"2023-09-23T09:27:50.271087+0000","last_unstale":"2023-09-23T09:28:54.855756+0000","last_undegraded":"2023-09-23T09:28:54.855756+0000","last_fullsized":"2023-09-23T09:28:54.855756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:09.363894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853720+0000","last_change":"2023-09-23T09:27:50.265855+0000","last_active":"2023-09-23T09:28:54.853720+0000","last_peered":"2023-09-23T09:28:54.853720+0000","last_clean":"2023-09-23T09:28:54.853720+0000","last_became_active":"2023-09-23T09:27:50.265101+0000","last_became_peered":"2023-09-23T09:27:50.265101+0000","last_unstale":"2023-09-23T09:28:54.853720+0000","last_undegraded":"2023-09-23T09:28:54.853720+0000","last_fullsized":"2023-09-23T09:28:54.853720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:52:40.456013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853875+0000","last_change":"2023-09-23T09:27:50.274405+0000","last_active":"2023-09-23T09:28:54.853875+0000","last_peered":"2023-09-23T09:28:54.853875+0000","last_clean":"2023-09-23T09:28:54.853875+0000","last_became_active":"2023-09-23T09:27:50.274036+0000","last_became_peered":"2023-09-23T09:27:50.274036+0000","last_unstale":"2023-09-23T09:28:54.853875+0000","last_undegraded":"2023-09-23T09:28:54.853875+0000","last_fullsized":"2023-09-23T09:28:54.853875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:38.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858871+0000","last_change":"2023-09-23T09:27:50.272237+0000","last_active":"2023-09-23T09:28:54.858871+0000","last_peered":"2023-09-23T09:28:54.858871+0000","last_clean":"2023-09-23T09:28:54.858871+0000","last_became_active":"2023-09-23T09:27:50.271897+0000","last_became_peered":"2023-09-23T09:27:50.271897+0000","last_unstale":"2023-09-23T09:28:54.858871+0000","last_undegraded":"2023-09-23T09:28:54.858871+0000","last_fullsized":"2023-09-23T09:28:54.858871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:31:31.115205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858427+0000","last_change":"2023-09-23T09:27:50.268902+0000","last_active":"2023-09-23T09:28:54.858427+0000","last_peered":"2023-09-23T09:28:54.858427+0000","last_clean":"2023-09-23T09:28:54.858427+0000","last_became_active":"2023-09-23T09:27:50.268590+0000","last_became_peered":"2023-09-23T09:27:50.268590+0000","last_unstale":"2023-09-23T09:28:54.858427+0000","last_undegraded":"2023-09-23T09:28:54.858427+0000","last_fullsized":"2023-09-23T09:28:54.858427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:54:39.384835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854373+0000","last_change":"2023-09-23T09:27:50.276934+0000","last_active":"2023-09-23T09:28:54.854373+0000","last_peered":"2023-09-23T09:28:54.854373+0000","last_clean":"2023-09-23T09:28:54.854373+0000","last_became_active":"2023-09-23T09:27:50.276645+0000","last_became_peered":"2023-09-23T09:27:50.276645+0000","last_unstale":"2023-09-23T09:28:54.854373+0000","last_undegraded":"2023-09-23T09:28:54.854373+0000","last_fullsized":"2023-09-23T09:28:54.854373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:07.100550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856855+0000","last_change":"2023-09-23T09:27:50.270781+0000","last_active":"2023-09-23T09:28:54.856855+0000","last_peered":"2023-09-23T09:28:54.856855+0000","last_clean":"2023-09-23T09:28:54.856855+0000","last_became_active":"2023-09-23T09:27:50.270319+0000","last_became_peered":"2023-09-23T09:27:50.270319+0000","last_unstale":"2023-09-23T09:28:54.856855+0000","last_undegraded":"2023-09-23T09:28:54.856855+0000","last_fullsized":"2023-09-23T09:28:54.856855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:42:44.882122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854722+0000","last_change":"2023-09-23T09:27:50.278775+0000","last_active":"2023-09-23T09:28:54.854722+0000","last_peered":"2023-09-23T09:28:54.854722+0000","last_clean":"2023-09-23T09:28:54.854722+0000","last_became_active":"2023-09-23T09:27:50.278375+0000","last_became_peered":"2023-09-23T09:27:50.278375+0000","last_unstale":"2023-09-23T09:28:54.854722+0000","last_undegraded":"2023-09-23T09:28:54.854722+0000","last_fullsized":"2023-09-23T09:28:54.854722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:14:00.076810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853885+0000","last_change":"2023-09-23T09:27:50.270141+0000","last_active":"2023-09-23T09:28:54.853885+0000","last_peered":"2023-09-23T09:28:54.853885+0000","last_clean":"2023-09-23T09:28:54.853885+0000","last_became_active":"2023-09-23T09:27:50.269849+0000","last_became_peered":"2023-09-23T09:27:50.269849+0000","last_unstale":"2023-09-23T09:28:54.853885+0000","last_undegraded":"2023-09-23T09:28:54.853885+0000","last_fullsized":"2023-09-23T09:28:54.853885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:49:41.837886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854235+0000","last_change":"2023-09-23T09:27:49.654458+0000","last_active":"2023-09-23T09:28:54.854235+0000","last_peered":"2023-09-23T09:28:54.854235+0000","last_clean":"2023-09-23T09:28:54.854235+0000","last_became_active":"2023-09-23T09:27:49.654084+0000","last_became_peered":"2023-09-23T09:27:49.654084+0000","last_unstale":"2023-09-23T09:28:54.854235+0000","last_undegraded":"2023-09-23T09:28:54.854235+0000","last_fullsized":"2023-09-23T09:28:54.854235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:47:32.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857109+0000","last_change":"2023-09-23T09:27:50.276558+0000","last_active":"2023-09-23T09:28:54.857109+0000","last_peered":"2023-09-23T09:28:54.857109+0000","last_clean":"2023-09-23T09:28:54.857109+0000","last_became_active":"2023-09-23T09:27:50.276255+0000","last_became_peered":"2023-09-23T09:27:50.276255+0000","last_unstale":"2023-09-23T09:28:54.857109+0000","last_undegraded":"2023-09-23T09:28:54.857109+0000","last_fullsized":"2023-09-23T09:28:54.857109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:13:09.870383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857231+0000","last_change":"2023-09-23T09:27:49.659256+0000","last_active":"2023-09-23T09:28:54.857231+0000","last_peered":"2023-09-23T09:28:54.857231+0000","last_clean":"2023-09-23T09:28:54.857231+0000","last_became_active":"2023-09-23T09:27:49.658793+0000","last_became_peered":"2023-09-23T09:27:49.658793+0000","last_unstale":"2023-09-23T09:28:54.857231+0000","last_undegraded":"2023-09-23T09:28:54.857231+0000","last_fullsized":"2023-09-23T09:28:54.857231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:03.416666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859099+0000","last_change":"2023-09-23T09:27:50.277338+0000","last_active":"2023-09-23T09:28:54.859099+0000","last_peered":"2023-09-23T09:28:54.859099+0000","last_clean":"2023-09-23T09:28:54.859099+0000","last_became_active":"2023-09-23T09:27:50.277011+0000","last_became_peered":"2023-09-23T09:27:50.277011+0000","last_unstale":"2023-09-23T09:28:54.859099+0000","last_undegraded":"2023-09-23T09:28:54.859099+0000","last_fullsized":"2023-09-23T09:28:54.859099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:52:23.028794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855646+0000","last_change":"2023-09-23T09:27:49.660679+0000","last_active":"2023-09-23T09:28:54.855646+0000","last_peered":"2023-09-23T09:28:54.855646+0000","last_clean":"2023-09-23T09:28:54.855646+0000","last_became_active":"2023-09-23T09:27:49.660201+0000","last_became_peered":"2023-09-23T09:27:49.660201+0000","last_unstale":"2023-09-23T09:28:54.855646+0000","last_undegraded":"2023-09-23T09:28:54.855646+0000","last_fullsized":"2023-09-23T09:28:54.855646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:33:56.205322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854501+0000","last_change":"2023-09-23T09:27:50.278373+0000","last_active":"2023-09-23T09:28:54.854501+0000","last_peered":"2023-09-23T09:28:54.854501+0000","last_clean":"2023-09-23T09:28:54.854501+0000","last_became_active":"2023-09-23T09:27:50.277837+0000","last_became_peered":"2023-09-23T09:27:50.277837+0000","last_unstale":"2023-09-23T09:28:54.854501+0000","last_undegraded":"2023-09-23T09:28:54.854501+0000","last_fullsized":"2023-09-23T09:28:54.854501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:24:26.914509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855285+0000","last_change":"2023-09-23T09:27:49.657578+0000","last_active":"2023-09-23T09:28:54.855285+0000","last_peered":"2023-09-23T09:28:54.855285+0000","last_clean":"2023-09-23T09:28:54.855285+0000","last_became_active":"2023-09-23T09:27:49.657278+0000","last_became_peered":"2023-09-23T09:27:49.657278+0000","last_unstale":"2023-09-23T09:28:54.855285+0000","last_undegraded":"2023-09-23T09:28:54.855285+0000","last_fullsized":"2023-09-23T09:28:54.855285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:53.440997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853150+0000","last_change":"2023-09-23T09:27:50.279784+0000","last_active":"2023-09-23T09:28:54.853150+0000","last_peered":"2023-09-23T09:28:54.853150+0000","last_clean":"2023-09-23T09:28:54.853150+0000","last_became_active":"2023-09-23T09:27:50.279473+0000","last_became_peered":"2023-09-23T09:27:50.279473+0000","last_unstale":"2023-09-23T09:28:54.853150+0000","last_undegraded":"2023-09-23T09:28:54.853150+0000","last_fullsized":"2023-09-23T09:28:54.853150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:30:13.010556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.374589+0000","last_change":"2023-09-23T09:27:49.655391+0000","last_active":"2023-09-23T09:29:07.374589+0000","last_peered":"2023-09-23T09:29:07.374589+0000","last_clean":"2023-09-23T09:29:07.374589+0000","last_became_active":"2023-09-23T09:27:49.655067+0000","last_became_peered":"2023-09-23T09:27:49.655067+0000","last_unstale":"2023-09-23T09:29:07.374589+0000","last_undegraded":"2023-09-23T09:29:07.374589+0000","last_fullsized":"2023-09-23T09:29:07.374589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.701471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.861185+0000","last_change":"2023-09-23T09:28:55.861185+0000","last_active":"2023-09-23T09:28:55.861185+0000","last_peered":"2023-09-23T09:28:55.861185+0000","last_clean":"2023-09-23T09:28:55.861185+0000","last_became_active":"2023-09-23T09:28:55.860687+0000","last_became_peered":"2023-09-23T09:28:55.860687+0000","last_unstale":"2023-09-23T09:28:55.861185+0000","last_undegraded":"2023-09-23T09:28:55.861185+0000","last_fullsized":"2023-09-23T09:28:55.861185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:58.691882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.862026+0000","last_change":"2023-09-23T09:28:55.862026+0000","last_active":"2023-09-23T09:28:55.862026+0000","last_peered":"2023-09-23T09:28:55.862026+0000","last_clean":"2023-09-23T09:28:55.862026+0000","last_became_active":"2023-09-23T09:28:55.858187+0000","last_became_peered":"2023-09-23T09:28:55.858187+0000","last_unstale":"2023-09-23T09:28:55.862026+0000","last_undegraded":"2023-09-23T09:28:55.862026+0000","last_fullsized":"2023-09-23T09:28:55.862026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:34:37.712098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852709+0000","last_change":"2023-09-23T09:27:50.273705+0000","last_active":"2023-09-23T09:28:54.852709+0000","last_peered":"2023-09-23T09:28:54.852709+0000","last_clean":"2023-09-23T09:28:54.852709+0000","last_became_active":"2023-09-23T09:27:50.272245+0000","last_became_peered":"2023-09-23T09:27:50.272245+0000","last_unstale":"2023-09-23T09:28:54.852709+0000","last_undegraded":"2023-09-23T09:28:54.852709+0000","last_fullsized":"2023-09-23T09:28:54.852709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:45:57.318326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851649+0000","last_change":"2023-09-23T09:27:49.651958+0000","last_active":"2023-09-23T09:28:54.851649+0000","last_peered":"2023-09-23T09:28:54.851649+0000","last_clean":"2023-09-23T09:28:54.851649+0000","last_became_active":"2023-09-23T09:27:49.651058+0000","last_became_peered":"2023-09-23T09:27:49.651058+0000","last_unstale":"2023-09-23T09:28:54.851649+0000","last_undegraded":"2023-09-23T09:28:54.851649+0000","last_fullsized":"2023-09-23T09:28:54.851649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:26:56.360566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857202+0000","last_change":"2023-09-23T09:27:50.270804+0000","last_active":"2023-09-23T09:28:54.857202+0000","last_peered":"2023-09-23T09:28:54.857202+0000","last_clean":"2023-09-23T09:28:54.857202+0000","last_became_active":"2023-09-23T09:27:50.270044+0000","last_became_peered":"2023-09-23T09:27:50.270044+0000","last_unstale":"2023-09-23T09:28:54.857202+0000","last_undegraded":"2023-09-23T09:28:54.857202+0000","last_fullsized":"2023-09-23T09:28:54.857202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:59:20.821949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852245+0000","last_change":"2023-09-23T09:27:49.657030+0000","last_active":"2023-09-23T09:28:54.852245+0000","last_peered":"2023-09-23T09:28:54.852245+0000","last_clean":"2023-09-23T09:28:54.852245+0000","last_became_active":"2023-09-23T09:27:49.656710+0000","last_became_peered":"2023-09-23T09:27:49.656710+0000","last_unstale":"2023-09-23T09:28:54.852245+0000","last_undegraded":"2023-09-23T09:28:54.852245+0000","last_fullsized":"2023-09-23T09:28:54.852245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:24:59.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853990+0000","last_change":"2023-09-23T09:27:50.271688+0000","last_active":"2023-09-23T09:28:54.853990+0000","last_peered":"2023-09-23T09:28:54.853990+0000","last_clean":"2023-09-23T09:28:54.853990+0000","last_became_active":"2023-09-23T09:27:50.271276+0000","last_became_peered":"2023-09-23T09:27:50.271276+0000","last_unstale":"2023-09-23T09:28:54.853990+0000","last_undegraded":"2023-09-23T09:28:54.853990+0000","last_fullsized":"2023-09-23T09:28:54.853990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:40:48.333279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.374069+0000","last_change":"2023-09-23T09:27:49.655183+0000","last_active":"2023-09-23T09:29:07.374069+0000","last_peered":"2023-09-23T09:29:07.374069+0000","last_clean":"2023-09-23T09:29:07.374069+0000","last_became_active":"2023-09-23T09:27:49.654875+0000","last_became_peered":"2023-09-23T09:27:49.654875+0000","last_unstale":"2023-09-23T09:29:07.374069+0000","last_undegraded":"2023-09-23T09:29:07.374069+0000","last_fullsized":"2023-09-23T09:29:07.374069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:12:49.944732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854944+0000","last_change":"2023-09-23T09:27:50.276347+0000","last_active":"2023-09-23T09:28:54.854944+0000","last_peered":"2023-09-23T09:28:54.854944+0000","last_clean":"2023-09-23T09:28:54.854944+0000","last_became_active":"2023-09-23T09:27:50.276095+0000","last_became_peered":"2023-09-23T09:27:50.276095+0000","last_unstale":"2023-09-23T09:28:54.854944+0000","last_undegraded":"2023-09-23T09:28:54.854944+0000","last_fullsized":"2023-09-23T09:28:54.854944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:42.952808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855651+0000","last_change":"2023-09-23T09:27:49.657115+0000","last_active":"2023-09-23T09:28:54.855651+0000","last_peered":"2023-09-23T09:28:54.855651+0000","last_clean":"2023-09-23T09:28:54.855651+0000","last_became_active":"2023-09-23T09:27:49.656816+0000","last_became_peered":"2023-09-23T09:27:49.656816+0000","last_unstale":"2023-09-23T09:28:54.855651+0000","last_undegraded":"2023-09-23T09:28:54.855651+0000","last_fullsized":"2023-09-23T09:28:54.855651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:37.205221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859056+0000","last_change":"2023-09-23T09:27:50.277477+0000","last_active":"2023-09-23T09:28:54.859056+0000","last_peered":"2023-09-23T09:28:54.859056+0000","last_clean":"2023-09-23T09:28:54.859056+0000","last_became_active":"2023-09-23T09:27:50.277200+0000","last_became_peered":"2023-09-23T09:27:50.277200+0000","last_unstale":"2023-09-23T09:28:54.859056+0000","last_undegraded":"2023-09-23T09:28:54.859056+0000","last_fullsized":"2023-09-23T09:28:54.859056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:45:03.167471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853242+0000","last_change":"2023-09-23T09:27:49.663678+0000","last_active":"2023-09-23T09:28:54.853242+0000","last_peered":"2023-09-23T09:28:54.853242+0000","last_clean":"2023-09-23T09:28:54.853242+0000","last_became_active":"2023-09-23T09:27:49.663142+0000","last_became_peered":"2023-09-23T09:27:49.663142+0000","last_unstale":"2023-09-23T09:28:54.853242+0000","last_undegraded":"2023-09-23T09:28:54.853242+0000","last_fullsized":"2023-09-23T09:28:54.853242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:14.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854754+0000","last_change":"2023-09-23T09:27:50.275044+0000","last_active":"2023-09-23T09:28:54.854754+0000","last_peered":"2023-09-23T09:28:54.854754+0000","last_clean":"2023-09-23T09:28:54.854754+0000","last_became_active":"2023-09-23T09:27:50.274752+0000","last_became_peered":"2023-09-23T09:27:50.274752+0000","last_unstale":"2023-09-23T09:28:54.854754+0000","last_undegraded":"2023-09-23T09:28:54.854754+0000","last_fullsized":"2023-09-23T09:28:54.854754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:19.914752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":196,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:08.375708+0000","last_change":"2023-09-23T09:27:49.653694+0000","last_active":"2023-09-23T09:29:08.375708+0000","last_peered":"2023-09-23T09:29:08.375708+0000","last_clean":"2023-09-23T09:29:08.375708+0000","last_became_active":"2023-09-23T09:27:49.653355+0000","last_became_peered":"2023-09-23T09:27:49.653355+0000","last_unstale":"2023-09-23T09:29:08.375708+0000","last_undegraded":"2023-09-23T09:29:08.375708+0000","last_fullsized":"2023-09-23T09:29:08.375708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:19:53.282723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56994,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":132,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855406+0000","last_change":"2023-09-23T09:27:50.272849+0000","last_active":"2023-09-23T09:28:54.855406+0000","last_peered":"2023-09-23T09:28:54.855406+0000","last_clean":"2023-09-23T09:28:54.855406+0000","last_became_active":"2023-09-23T09:27:50.272411+0000","last_became_peered":"2023-09-23T09:27:50.272411+0000","last_unstale":"2023-09-23T09:28:54.855406+0000","last_undegraded":"2023-09-23T09:28:54.855406+0000","last_fullsized":"2023-09-23T09:28:54.855406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:01:41.972357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853996+0000","last_change":"2023-09-23T09:27:49.652769+0000","last_active":"2023-09-23T09:28:54.853996+0000","last_peered":"2023-09-23T09:28:54.853996+0000","last_clean":"2023-09-23T09:28:54.853996+0000","last_became_active":"2023-09-23T09:27:49.652440+0000","last_became_peered":"2023-09-23T09:27:49.652440+0000","last_unstale":"2023-09-23T09:28:54.853996+0000","last_undegraded":"2023-09-23T09:28:54.853996+0000","last_fullsized":"2023-09-23T09:28:54.853996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:02.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851736+0000","last_change":"2023-09-23T09:27:50.272105+0000","last_active":"2023-09-23T09:28:54.851736+0000","last_peered":"2023-09-23T09:28:54.851736+0000","last_clean":"2023-09-23T09:28:54.851736+0000","last_became_active":"2023-09-23T09:27:50.271766+0000","last_became_peered":"2023-09-23T09:27:50.271766+0000","last_unstale":"2023-09-23T09:28:54.851736+0000","last_undegraded":"2023-09-23T09:28:54.851736+0000","last_fullsized":"2023-09-23T09:28:54.851736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:46:46.012381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852525+0000","last_change":"2023-09-23T09:27:49.663473+0000","last_active":"2023-09-23T09:28:54.852525+0000","last_peered":"2023-09-23T09:28:54.852525+0000","last_clean":"2023-09-23T09:28:54.852525+0000","last_became_active":"2023-09-23T09:27:49.663085+0000","last_became_peered":"2023-09-23T09:27:49.663085+0000","last_unstale":"2023-09-23T09:28:54.852525+0000","last_undegraded":"2023-09-23T09:28:54.852525+0000","last_fullsized":"2023-09-23T09:28:54.852525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:28:05.246720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.860449+0000","last_change":"2023-09-23T09:28:55.860449+0000","last_active":"2023-09-23T09:28:55.860449+0000","last_peered":"2023-09-23T09:28:55.860449+0000","last_clean":"2023-09-23T09:28:55.860449+0000","last_became_active":"2023-09-23T09:28:55.860050+0000","last_became_peered":"2023-09-23T09:28:55.860050+0000","last_unstale":"2023-09-23T09:28:55.860449+0000","last_undegraded":"2023-09-23T09:28:55.860449+0000","last_fullsized":"2023-09-23T09:28:55.860449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:43:35.092979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:06.372260+0000","last_change":"2023-09-23T09:27:49.654341+0000","last_active":"2023-09-23T09:29:06.372260+0000","last_peered":"2023-09-23T09:29:06.372260+0000","last_clean":"2023-09-23T09:29:06.372260+0000","last_became_active":"2023-09-23T09:27:49.653936+0000","last_became_peered":"2023-09-23T09:27:49.653936+0000","last_unstale":"2023-09-23T09:29:06.372260+0000","last_undegraded":"2023-09-23T09:29:06.372260+0000","last_fullsized":"2023-09-23T09:29:06.372260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:48.876752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855441+0000","last_change":"2023-09-23T09:27:50.278620+0000","last_active":"2023-09-23T09:28:54.855441+0000","last_peered":"2023-09-23T09:28:54.855441+0000","last_clean":"2023-09-23T09:28:54.855441+0000","last_became_active":"2023-09-23T09:27:50.278289+0000","last_became_peered":"2023-09-23T09:27:50.278289+0000","last_unstale":"2023-09-23T09:28:54.855441+0000","last_undegraded":"2023-09-23T09:28:54.855441+0000","last_fullsized":"2023-09-23T09:28:54.855441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:20:44.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854898+0000","last_change":"2023-09-23T09:27:49.662970+0000","last_active":"2023-09-23T09:28:54.854898+0000","last_peered":"2023-09-23T09:28:54.854898+0000","last_clean":"2023-09-23T09:28:54.854898+0000","last_became_active":"2023-09-23T09:27:49.662664+0000","last_became_peered":"2023-09-23T09:27:49.662664+0000","last_unstale":"2023-09-23T09:28:54.854898+0000","last_undegraded":"2023-09-23T09:28:54.854898+0000","last_fullsized":"2023-09-23T09:28:54.854898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:13.189686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852307+0000","last_change":"2023-09-23T09:27:50.278475+0000","last_active":"2023-09-23T09:28:54.852307+0000","last_peered":"2023-09-23T09:28:54.852307+0000","last_clean":"2023-09-23T09:28:54.852307+0000","last_became_active":"2023-09-23T09:27:50.277802+0000","last_became_peered":"2023-09-23T09:27:50.277802+0000","last_unstale":"2023-09-23T09:28:54.852307+0000","last_undegraded":"2023-09-23T09:28:54.852307+0000","last_fullsized":"2023-09-23T09:28:54.852307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:44.608172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855051+0000","last_change":"2023-09-23T09:27:49.657191+0000","last_active":"2023-09-23T09:28:54.855051+0000","last_peered":"2023-09-23T09:28:54.855051+0000","last_clean":"2023-09-23T09:28:54.855051+0000","last_became_active":"2023-09-23T09:27:49.656863+0000","last_became_peered":"2023-09-23T09:27:49.656863+0000","last_unstale":"2023-09-23T09:28:54.855051+0000","last_undegraded":"2023-09-23T09:28:54.855051+0000","last_fullsized":"2023-09-23T09:28:54.855051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:10.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856697+0000","last_change":"2023-09-23T09:27:50.274609+0000","last_active":"2023-09-23T09:28:54.856697+0000","last_peered":"2023-09-23T09:28:54.856697+0000","last_clean":"2023-09-23T09:28:54.856697+0000","last_became_active":"2023-09-23T09:27:50.274316+0000","last_became_peered":"2023-09-23T09:27:50.274316+0000","last_unstale":"2023-09-23T09:28:54.856697+0000","last_undegraded":"2023-09-23T09:28:54.856697+0000","last_fullsized":"2023-09-23T09:28:54.856697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:10:51.284306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:56.023463+0000","last_change":"2023-09-23T09:28:56.023463+0000","last_active":"2023-09-23T09:28:56.023463+0000","last_peered":"2023-09-23T09:28:56.023463+0000","last_clean":"2023-09-23T09:28:56.023463+0000","last_became_active":"2023-09-23T09:28:56.023001+0000","last_became_peered":"2023-09-23T09:28:56.023001+0000","last_unstale":"2023-09-23T09:28:56.023463+0000","last_undegraded":"2023-09-23T09:28:56.023463+0000","last_fullsized":"2023-09-23T09:28:56.023463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:48:56.424613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851865+0000","last_change":"2023-09-23T09:27:50.268568+0000","last_active":"2023-09-23T09:28:54.851865+0000","last_peered":"2023-09-23T09:28:54.851865+0000","last_clean":"2023-09-23T09:28:54.851865+0000","last_became_active":"2023-09-23T09:27:50.268268+0000","last_became_peered":"2023-09-23T09:27:50.268268+0000","last_unstale":"2023-09-23T09:28:54.851865+0000","last_undegraded":"2023-09-23T09:28:54.851865+0000","last_fullsized":"2023-09-23T09:28:54.851865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:59:22.516794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856046+0000","last_change":"2023-09-23T09:27:49.657160+0000","last_active":"2023-09-23T09:28:54.856046+0000","last_peered":"2023-09-23T09:28:54.856046+0000","last_clean":"2023-09-23T09:28:54.856046+0000","last_became_active":"2023-09-23T09:27:49.656853+0000","last_became_peered":"2023-09-23T09:27:49.656853+0000","last_unstale":"2023-09-23T09:28:54.856046+0000","last_undegraded":"2023-09-23T09:28:54.856046+0000","last_fullsized":"2023-09-23T09:28:54.856046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:17.489250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851769+0000","last_change":"2023-09-23T09:27:50.272495+0000","last_active":"2023-09-23T09:28:54.851769+0000","last_peered":"2023-09-23T09:28:54.851769+0000","last_clean":"2023-09-23T09:28:54.851769+0000","last_became_active":"2023-09-23T09:27:50.272207+0000","last_became_peered":"2023-09-23T09:27:50.272207+0000","last_unstale":"2023-09-23T09:28:54.851769+0000","last_undegraded":"2023-09-23T09:28:54.851769+0000","last_fullsized":"2023-09-23T09:28:54.851769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:24.952253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852430+0000","last_change":"2023-09-23T09:27:50.275551+0000","last_active":"2023-09-23T09:28:54.852430+0000","last_peered":"2023-09-23T09:28:54.852430+0000","last_clean":"2023-09-23T09:28:54.852430+0000","last_became_active":"2023-09-23T09:27:50.275235+0000","last_became_peered":"2023-09-23T09:27:50.275235+0000","last_unstale":"2023-09-23T09:28:54.852430+0000","last_undegraded":"2023-09-23T09:28:54.852430+0000","last_fullsized":"2023-09-23T09:28:54.852430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:41.009566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":93,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.373280+0000","last_change":"2023-09-23T09:27:49.652889+0000","last_active":"2023-09-23T09:29:07.373280+0000","last_peered":"2023-09-23T09:29:07.373280+0000","last_clean":"2023-09-23T09:29:07.373280+0000","last_became_active":"2023-09-23T09:27:49.652571+0000","last_became_peered":"2023-09-23T09:27:49.652571+0000","last_unstale":"2023-09-23T09:29:07.373280+0000","last_undegraded":"2023-09-23T09:29:07.373280+0000","last_fullsized":"2023-09-23T09:29:07.373280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:22.484052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856368+0000","last_change":"2023-09-23T09:27:50.278419+0000","last_active":"2023-09-23T09:28:54.856368+0000","last_peered":"2023-09-23T09:28:54.856368+0000","last_clean":"2023-09-23T09:28:54.856368+0000","last_became_active":"2023-09-23T09:27:50.278071+0000","last_became_peered":"2023-09-23T09:27:50.278071+0000","last_unstale":"2023-09-23T09:28:54.856368+0000","last_undegraded":"2023-09-23T09:28:54.856368+0000","last_fullsized":"2023-09-23T09:28:54.856368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:28:45.896479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854153+0000","last_change":"2023-09-23T09:27:49.651251+0000","last_active":"2023-09-23T09:28:54.854153+0000","last_peered":"2023-09-23T09:28:54.854153+0000","last_clean":"2023-09-23T09:28:54.854153+0000","last_became_active":"2023-09-23T09:27:49.650933+0000","last_became_peered":"2023-09-23T09:27:49.650933+0000","last_unstale":"2023-09-23T09:28:54.854153+0000","last_undegraded":"2023-09-23T09:28:54.854153+0000","last_fullsized":"2023-09-23T09:28:54.854153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:00:54.100400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":321,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855483+0000","last_change":"2023-09-23T09:26:51.296669+0000","last_active":"2023-09-23T09:28:54.855483+0000","last_peered":"2023-09-23T09:28:54.855483+0000","last_clean":"2023-09-23T09:28:54.855483+0000","last_became_active":"2023-09-23T09:26:51.296229+0000","last_became_peered":"2023-09-23T09:26:51.296229+0000","last_unstale":"2023-09-23T09:28:54.855483+0000","last_undegraded":"2023-09-23T09:28:54.855483+0000","last_fullsized":"2023-09-23T09:28:54.855483+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_clean_scrub_stamp":"2023-09-23T09:24:01.204422+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:31:24.888152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.860426+0000","last_change":"2023-09-23T09:28:55.860426+0000","last_active":"2023-09-23T09:28:55.860426+0000","last_peered":"2023-09-23T09:28:55.860426+0000","last_clean":"2023-09-23T09:28:55.860426+0000","last_became_active":"2023-09-23T09:28:55.860066+0000","last_became_peered":"2023-09-23T09:28:55.860066+0000","last_unstale":"2023-09-23T09:28:55.860426+0000","last_undegraded":"2023-09-23T09:28:55.860426+0000","last_fullsized":"2023-09-23T09:28:55.860426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:36:03.549733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855636+0000","last_change":"2023-09-23T09:27:49.659773+0000","last_active":"2023-09-23T09:28:54.855636+0000","last_peered":"2023-09-23T09:28:54.855636+0000","last_clean":"2023-09-23T09:28:54.855636+0000","last_became_active":"2023-09-23T09:27:49.659526+0000","last_became_peered":"2023-09-23T09:27:49.659526+0000","last_unstale":"2023-09-23T09:28:54.855636+0000","last_undegraded":"2023-09-23T09:28:54.855636+0000","last_fullsized":"2023-09-23T09:28:54.855636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:27.423680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855501+0000","last_change":"2023-09-23T09:27:49.661336+0000","last_active":"2023-09-23T09:28:54.855501+0000","last_peered":"2023-09-23T09:28:54.855501+0000","last_clean":"2023-09-23T09:28:54.855501+0000","last_became_active":"2023-09-23T09:27:49.661038+0000","last_became_peered":"2023-09-23T09:27:49.661038+0000","last_unstale":"2023-09-23T09:28:54.855501+0000","last_undegraded":"2023-09-23T09:28:54.855501+0000","last_fullsized":"2023-09-23T09:28:54.855501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:20.634410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852470+0000","last_change":"2023-09-23T09:27:50.272485+0000","last_active":"2023-09-23T09:28:54.852470+0000","last_peered":"2023-09-23T09:28:54.852470+0000","last_clean":"2023-09-23T09:28:54.852470+0000","last_became_active":"2023-09-23T09:27:50.272164+0000","last_became_peered":"2023-09-23T09:27:50.272164+0000","last_unstale":"2023-09-23T09:28:54.852470+0000","last_undegraded":"2023-09-23T09:28:54.852470+0000","last_fullsized":"2023-09-23T09:28:54.852470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:48:00.596034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:56.027111+0000","last_change":"2023-09-23T09:28:56.027111+0000","last_active":"2023-09-23T09:28:56.027111+0000","last_peered":"2023-09-23T09:28:56.027111+0000","last_clean":"2023-09-23T09:28:56.027111+0000","last_became_active":"2023-09-23T09:28:56.023192+0000","last_became_peered":"2023-09-23T09:28:56.023192+0000","last_unstale":"2023-09-23T09:28:56.027111+0000","last_undegraded":"2023-09-23T09:28:56.027111+0000","last_fullsized":"2023-09-23T09:28:56.027111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:53.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857537+0000","last_change":"2023-09-23T09:27:50.274959+0000","last_active":"2023-09-23T09:28:54.857537+0000","last_peered":"2023-09-23T09:28:54.857537+0000","last_clean":"2023-09-23T09:28:54.857537+0000","last_became_active":"2023-09-23T09:27:50.274614+0000","last_became_peered":"2023-09-23T09:27:50.274614+0000","last_unstale":"2023-09-23T09:28:54.857537+0000","last_undegraded":"2023-09-23T09:28:54.857537+0000","last_fullsized":"2023-09-23T09:28:54.857537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:00:09.831103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855933+0000","last_change":"2023-09-23T09:27:49.654673+0000","last_active":"2023-09-23T09:28:54.855933+0000","last_peered":"2023-09-23T09:28:54.855933+0000","last_clean":"2023-09-23T09:28:54.855933+0000","last_became_active":"2023-09-23T09:27:49.654356+0000","last_became_peered":"2023-09-23T09:27:49.654356+0000","last_unstale":"2023-09-23T09:28:54.855933+0000","last_undegraded":"2023-09-23T09:28:54.855933+0000","last_fullsized":"2023-09-23T09:28:54.855933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:39:50.628346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858948+0000","last_change":"2023-09-23T09:27:50.267368+0000","last_active":"2023-09-23T09:28:54.858948+0000","last_peered":"2023-09-23T09:28:54.858948+0000","last_clean":"2023-09-23T09:28:54.858948+0000","last_became_active":"2023-09-23T09:27:50.266962+0000","last_became_peered":"2023-09-23T09:27:50.266962+0000","last_unstale":"2023-09-23T09:28:54.858948+0000","last_undegraded":"2023-09-23T09:28:54.858948+0000","last_fullsized":"2023-09-23T09:28:54.858948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:24.660228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854472+0000","last_change":"2023-09-23T09:27:49.661200+0000","last_active":"2023-09-23T09:28:54.854472+0000","last_peered":"2023-09-23T09:28:54.854472+0000","last_clean":"2023-09-23T09:28:54.854472+0000","last_became_active":"2023-09-23T09:27:49.660902+0000","last_became_peered":"2023-09-23T09:27:49.660902+0000","last_unstale":"2023-09-23T09:28:54.854472+0000","last_undegraded":"2023-09-23T09:28:54.854472+0000","last_fullsized":"2023-09-23T09:28:54.854472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:41.036216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854827+0000","last_change":"2023-09-23T09:27:50.271834+0000","last_active":"2023-09-23T09:28:54.854827+0000","last_peered":"2023-09-23T09:28:54.854827+0000","last_clean":"2023-09-23T09:28:54.854827+0000","last_became_active":"2023-09-23T09:27:50.271519+0000","last_became_peered":"2023-09-23T09:27:50.271519+0000","last_unstale":"2023-09-23T09:28:54.854827+0000","last_undegraded":"2023-09-23T09:28:54.854827+0000","last_fullsized":"2023-09-23T09:28:54.854827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:08.036615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857519+0000","last_change":"2023-09-23T09:27:49.656794+0000","last_active":"2023-09-23T09:28:54.857519+0000","last_peered":"2023-09-23T09:28:54.857519+0000","last_clean":"2023-09-23T09:28:54.857519+0000","last_became_active":"2023-09-23T09:27:49.656480+0000","last_became_peered":"2023-09-23T09:27:49.656480+0000","last_unstale":"2023-09-23T09:28:54.857519+0000","last_undegraded":"2023-09-23T09:28:54.857519+0000","last_fullsized":"2023-09-23T09:28:54.857519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:35:46.948703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856490+0000","last_change":"2023-09-23T09:27:50.271805+0000","last_active":"2023-09-23T09:28:54.856490+0000","last_peered":"2023-09-23T09:28:54.856490+0000","last_clean":"2023-09-23T09:28:54.856490+0000","last_became_active":"2023-09-23T09:27:50.271463+0000","last_became_peered":"2023-09-23T09:27:50.271463+0000","last_unstale":"2023-09-23T09:28:54.856490+0000","last_undegraded":"2023-09-23T09:28:54.856490+0000","last_fullsized":"2023-09-23T09:28:54.856490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:00:03.608504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.854053+0000","last_change":"2023-09-23T09:27:49.656988+0000","last_active":"2023-09-23T09:28:55.854053+0000","last_peered":"2023-09-23T09:28:55.854053+0000","last_clean":"2023-09-23T09:28:55.854053+0000","last_became_active":"2023-09-23T09:27:49.656655+0000","last_became_peered":"2023-09-23T09:27:49.656655+0000","last_unstale":"2023-09-23T09:28:55.854053+0000","last_undegraded":"2023-09-23T09:28:55.854053+0000","last_fullsized":"2023-09-23T09:28:55.854053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:07:33.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856634+0000","last_change":"2023-09-23T09:27:50.275021+0000","last_active":"2023-09-23T09:28:54.856634+0000","last_peered":"2023-09-23T09:28:54.856634+0000","last_clean":"2023-09-23T09:28:54.856634+0000","last_became_active":"2023-09-23T09:27:50.274551+0000","last_became_peered":"2023-09-23T09:27:50.274551+0000","last_unstale":"2023-09-23T09:28:54.856634+0000","last_undegraded":"2023-09-23T09:28:54.856634+0000","last_fullsized":"2023-09-23T09:28:54.856634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:50:45.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854922+0000","last_change":"2023-09-23T09:27:49.659603+0000","last_active":"2023-09-23T09:28:54.854922+0000","last_peered":"2023-09-23T09:28:54.854922+0000","last_clean":"2023-09-23T09:28:54.854922+0000","last_became_active":"2023-09-23T09:27:49.659293+0000","last_became_peered":"2023-09-23T09:27:49.659293+0000","last_unstale":"2023-09-23T09:28:54.854922+0000","last_undegraded":"2023-09-23T09:28:54.854922+0000","last_fullsized":"2023-09-23T09:28:54.854922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:31:25.165888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856201+0000","last_change":"2023-09-23T09:27:50.275069+0000","last_active":"2023-09-23T09:28:54.856201+0000","last_peered":"2023-09-23T09:28:54.856201+0000","last_clean":"2023-09-23T09:28:54.856201+0000","last_became_active":"2023-09-23T09:27:50.274731+0000","last_became_peered":"2023-09-23T09:27:50.274731+0000","last_unstale":"2023-09-23T09:28:54.856201+0000","last_undegraded":"2023-09-23T09:28:54.856201+0000","last_fullsized":"2023-09-23T09:28:54.856201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:45:23.794774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854094+0000","last_change":"2023-09-23T09:27:49.659914+0000","last_active":"2023-09-23T09:28:54.854094+0000","last_peered":"2023-09-23T09:28:54.854094+0000","last_clean":"2023-09-23T09:28:54.854094+0000","last_became_active":"2023-09-23T09:27:49.659604+0000","last_became_peered":"2023-09-23T09:27:49.659604+0000","last_unstale":"2023-09-23T09:28:54.854094+0000","last_undegraded":"2023-09-23T09:28:54.854094+0000","last_fullsized":"2023-09-23T09:28:54.854094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:20.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855822+0000","last_change":"2023-09-23T09:27:50.271309+0000","last_active":"2023-09-23T09:28:54.855822+0000","last_peered":"2023-09-23T09:28:54.855822+0000","last_clean":"2023-09-23T09:28:54.855822+0000","last_became_active":"2023-09-23T09:27:50.270951+0000","last_became_peered":"2023-09-23T09:27:50.270951+0000","last_unstale":"2023-09-23T09:28:54.855822+0000","last_undegraded":"2023-09-23T09:28:54.855822+0000","last_fullsized":"2023-09-23T09:28:54.855822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:18.811428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853059+0000","last_change":"2023-09-23T09:27:49.663849+0000","last_active":"2023-09-23T09:28:54.853059+0000","last_peered":"2023-09-23T09:28:54.853059+0000","last_clean":"2023-09-23T09:28:54.853059+0000","last_became_active":"2023-09-23T09:27:49.663499+0000","last_became_peered":"2023-09-23T09:27:49.663499+0000","last_unstale":"2023-09-23T09:28:54.853059+0000","last_undegraded":"2023-09-23T09:28:54.853059+0000","last_fullsized":"2023-09-23T09:28:54.853059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:31:08.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.864630+0000","last_change":"2023-09-23T09:28:55.864630+0000","last_active":"2023-09-23T09:28:55.864630+0000","last_peered":"2023-09-23T09:28:55.864630+0000","last_clean":"2023-09-23T09:28:55.864630+0000","last_became_active":"2023-09-23T09:28:55.860402+0000","last_became_peered":"2023-09-23T09:28:55.860402+0000","last_unstale":"2023-09-23T09:28:55.864630+0000","last_undegraded":"2023-09-23T09:28:55.864630+0000","last_fullsized":"2023-09-23T09:28:55.864630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:58:52.288357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.910095+0000","last_change":"2023-09-23T09:28:55.910095+0000","last_active":"2023-09-23T09:28:55.910095+0000","last_peered":"2023-09-23T09:28:55.910095+0000","last_clean":"2023-09-23T09:28:55.910095+0000","last_became_active":"2023-09-23T09:28:55.907334+0000","last_became_peered":"2023-09-23T09:28:55.907334+0000","last_unstale":"2023-09-23T09:28:55.910095+0000","last_undegraded":"2023-09-23T09:28:55.910095+0000","last_fullsized":"2023-09-23T09:28:55.910095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:11:04.045821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855746+0000","last_change":"2023-09-23T09:27:50.271063+0000","last_active":"2023-09-23T09:28:54.855746+0000","last_peered":"2023-09-23T09:28:54.855746+0000","last_clean":"2023-09-23T09:28:54.855746+0000","last_became_active":"2023-09-23T09:27:50.270487+0000","last_became_peered":"2023-09-23T09:27:50.270487+0000","last_unstale":"2023-09-23T09:28:54.855746+0000","last_undegraded":"2023-09-23T09:28:54.855746+0000","last_fullsized":"2023-09-23T09:28:54.855746+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:11:05.621855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856550+0000","last_change":"2023-09-23T09:27:50.275900+0000","last_active":"2023-09-23T09:28:54.856550+0000","last_peered":"2023-09-23T09:28:54.856550+0000","last_clean":"2023-09-23T09:28:54.856550+0000","last_became_active":"2023-09-23T09:27:50.275565+0000","last_became_peered":"2023-09-23T09:27:50.275565+0000","last_unstale":"2023-09-23T09:28:54.856550+0000","last_undegraded":"2023-09-23T09:28:54.856550+0000","last_fullsized":"2023-09-23T09:28:54.856550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:57:37.004604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854247+0000","last_change":"2023-09-23T09:27:50.275911+0000","last_active":"2023-09-23T09:28:54.854247+0000","last_peered":"2023-09-23T09:28:54.854247+0000","last_clean":"2023-09-23T09:28:54.854247+0000","last_became_active":"2023-09-23T09:27:50.275626+0000","last_became_peered":"2023-09-23T09:27:50.275626+0000","last_unstale":"2023-09-23T09:28:54.854247+0000","last_undegraded":"2023-09-23T09:28:54.854247+0000","last_fullsized":"2023-09-23T09:28:54.854247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:46.790012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853758+0000","last_change":"2023-09-23T09:27:50.269483+0000","last_active":"2023-09-23T09:28:54.853758+0000","last_peered":"2023-09-23T09:28:54.853758+0000","last_clean":"2023-09-23T09:28:54.853758+0000","last_became_active":"2023-09-23T09:27:50.269199+0000","last_became_peered":"2023-09-23T09:27:50.269199+0000","last_unstale":"2023-09-23T09:28:54.853758+0000","last_undegraded":"2023-09-23T09:28:54.853758+0000","last_fullsized":"2023-09-23T09:28:54.853758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:58:34.304439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.859887+0000","last_change":"2023-09-23T09:28:55.859887+0000","last_active":"2023-09-23T09:28:55.859887+0000","last_peered":"2023-09-23T09:28:55.859887+0000","last_clean":"2023-09-23T09:28:55.859887+0000","last_became_active":"2023-09-23T09:28:55.859497+0000","last_became_peered":"2023-09-23T09:28:55.859497+0000","last_unstale":"2023-09-23T09:28:55.859887+0000","last_undegraded":"2023-09-23T09:28:55.859887+0000","last_fullsized":"2023-09-23T09:28:55.859887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:05:52.153365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852551+0000","last_change":"2023-09-23T09:27:50.269934+0000","last_active":"2023-09-23T09:28:54.852551+0000","last_peered":"2023-09-23T09:28:54.852551+0000","last_clean":"2023-09-23T09:28:54.852551+0000","last_became_active":"2023-09-23T09:27:50.269060+0000","last_became_peered":"2023-09-23T09:27:50.269060+0000","last_unstale":"2023-09-23T09:28:54.852551+0000","last_undegraded":"2023-09-23T09:28:54.852551+0000","last_fullsized":"2023-09-23T09:28:54.852551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:47:31.191479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856576+0000","last_change":"2023-09-23T09:27:50.277885+0000","last_active":"2023-09-23T09:28:54.856576+0000","last_peered":"2023-09-23T09:28:54.856576+0000","last_clean":"2023-09-23T09:28:54.856576+0000","last_became_active":"2023-09-23T09:27:50.277576+0000","last_became_peered":"2023-09-23T09:27:50.277576+0000","last_unstale":"2023-09-23T09:28:54.856576+0000","last_undegraded":"2023-09-23T09:28:54.856576+0000","last_fullsized":"2023-09-23T09:28:54.856576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:03:43.338185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854099+0000","last_change":"2023-09-23T09:27:50.278225+0000","last_active":"2023-09-23T09:28:54.854099+0000","last_peered":"2023-09-23T09:28:54.854099+0000","last_clean":"2023-09-23T09:28:54.854099+0000","last_became_active":"2023-09-23T09:27:50.277891+0000","last_became_peered":"2023-09-23T09:27:50.277891+0000","last_unstale":"2023-09-23T09:28:54.854099+0000","last_undegraded":"2023-09-23T09:28:54.854099+0000","last_fullsized":"2023-09-23T09:28:54.854099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:27:08.961439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857440+0000","last_change":"2023-09-23T09:27:50.273445+0000","last_active":"2023-09-23T09:28:54.857440+0000","last_peered":"2023-09-23T09:28:54.857440+0000","last_clean":"2023-09-23T09:28:54.857440+0000","last_became_active":"2023-09-23T09:27:50.273124+0000","last_became_peered":"2023-09-23T09:27:50.273124+0000","last_unstale":"2023-09-23T09:28:54.857440+0000","last_undegraded":"2023-09-23T09:28:54.857440+0000","last_fullsized":"2023-09-23T09:28:54.857440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:46.030212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852344+0000","last_change":"2023-09-23T09:27:50.276164+0000","last_active":"2023-09-23T09:28:54.852344+0000","last_peered":"2023-09-23T09:28:54.852344+0000","last_clean":"2023-09-23T09:28:54.852344+0000","last_became_active":"2023-09-23T09:27:50.275860+0000","last_became_peered":"2023-09-23T09:27:50.275860+0000","last_unstale":"2023-09-23T09:28:54.852344+0000","last_undegraded":"2023-09-23T09:28:54.852344+0000","last_fullsized":"2023-09-23T09:28:54.852344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:53:50.521872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857428+0000","last_change":"2023-09-23T09:27:50.275221+0000","last_active":"2023-09-23T09:28:54.857428+0000","last_peered":"2023-09-23T09:28:54.857428+0000","last_clean":"2023-09-23T09:28:54.857428+0000","last_became_active":"2023-09-23T09:27:50.274920+0000","last_became_peered":"2023-09-23T09:27:50.274920+0000","last_unstale":"2023-09-23T09:28:54.857428+0000","last_undegraded":"2023-09-23T09:28:54.857428+0000","last_fullsized":"2023-09-23T09:28:54.857428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:45:35.023433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854715+0000","last_change":"2023-09-23T09:27:50.277776+0000","last_active":"2023-09-23T09:28:54.854715+0000","last_peered":"2023-09-23T09:28:54.854715+0000","last_clean":"2023-09-23T09:28:54.854715+0000","last_became_active":"2023-09-23T09:27:50.277291+0000","last_became_peered":"2023-09-23T09:27:50.277291+0000","last_unstale":"2023-09-23T09:28:54.854715+0000","last_undegraded":"2023-09-23T09:28:54.854715+0000","last_fullsized":"2023-09-23T09:28:54.854715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:23:46.984450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852197+0000","last_change":"2023-09-23T09:27:50.271219+0000","last_active":"2023-09-23T09:28:54.852197+0000","last_peered":"2023-09-23T09:28:54.852197+0000","last_clean":"2023-09-23T09:28:54.852197+0000","last_became_active":"2023-09-23T09:27:50.270747+0000","last_became_peered":"2023-09-23T09:27:50.270747+0000","last_unstale":"2023-09-23T09:28:54.852197+0000","last_undegraded":"2023-09-23T09:28:54.852197+0000","last_fullsized":"2023-09-23T09:28:54.852197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:19.237260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856573+0000","last_change":"2023-09-23T09:27:50.263794+0000","last_active":"2023-09-23T09:28:54.856573+0000","last_peered":"2023-09-23T09:28:54.856573+0000","last_clean":"2023-09-23T09:28:54.856573+0000","last_became_active":"2023-09-23T09:27:50.263459+0000","last_became_peered":"2023-09-23T09:27:50.263459+0000","last_unstale":"2023-09-23T09:28:54.856573+0000","last_undegraded":"2023-09-23T09:28:54.856573+0000","last_fullsized":"2023-09-23T09:28:54.856573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:05:37.618959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250071,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61384,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":470,"num_read_kb":300,"num_write":151,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":185424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":444714,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79700000000000004}]},{"osd":1,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.748}]},{"osd":6,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71799999999999997}]},{"osd":8,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88500000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91600000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.81899999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1093585,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42199999999999999},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38100000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.74099999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.497},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60699999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.46200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72799999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.623},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84899999999999998,"5min":1.006,"15min":1.006},"last":0.65900000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81000000000000005,"5min":0.873,"15min":0.873},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.40100000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.39600000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.50800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.29,"15min":1.29},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.198,"15min":1.198},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91500000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91800000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57199999999999995}]},{"osd":9,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93200000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.63900000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93700000000000006,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61899999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59499999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":445073,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51400000000000001,"15min":0.51800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.67700000000000005,"15min":0.66100000000000003},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.139,"5min":0.79800000000000004,"15min":0.74099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.616,"5min":11.616,"15min":11.616},"last":0.443}]},{"osd":2,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":0.83999999999999997,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":11.647,"5min":11.647,"15min":11.647},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.9889999999999999,"5min":3.9889999999999999,"15min":3.9889999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.78800000000000003,"15min":0.751},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":11.702,"5min":11.702,"15min":11.702},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.6789999999999998,"5min":3.6789999999999998,"15min":3.6789999999999998},"last":0.376}]},{"osd":5,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":0.71299999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82099999999999995,"15min":0.83199999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88,"15min":0.88},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":0.999,"15min":0.96699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":11.557,"5min":11.557,"15min":11.557},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.004,"15min":0.96699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":11.675000000000001,"5min":11.675000000000001,"15min":11.675000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.89700000000000002,"15min":0.91600000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.65300000000000002}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.86699999999999999,"15min":0.86299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.67000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444870,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64800000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.55700000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.55900000000000005,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.45300000000000001},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63300000000000001,"5min":0.745,"15min":0.745},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.375,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60399999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80200000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.316,"15min":0.316},"max":{"1min":0.89100000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.76600000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.58099999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.999}]},{"osd":7,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.751,"15min":0.73099999999999998},"min":{"1min":0.60999999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89800000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.0800000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.749,"15min":0.73299999999999998},"min":{"1min":0.624,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":1.0169999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.042}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1036922,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.30299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.36799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67700000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.55200000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.83099999999999996}]},{"osd":3,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.497,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.879,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64400000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69499999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75800000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.53800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78800000000000003}]},{"osd":6,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.52800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83899999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67800000000000005,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.53700000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99299999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.89300000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.871,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.874},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94699999999999995,"5min":1.022,"15min":1.022},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.90800000000000003}]},{"osd":11,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93100000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":444345,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.48199999999999998,"15min":0.432},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.442,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.432}]},{"osd":1,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.65900000000000003,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.626}]},{"osd":2,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.48099999999999998},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.66700000000000004,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.312,"15min":0.312},"max":{"1min":11.151999999999999,"5min":11.151999999999999,"15min":11.151999999999999},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.749}]},{"osd":5,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.749,"15min":0.69799999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":6.952,"5min":6.952,"15min":6.952},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2230000000000001,"5min":1.294,"15min":1.294},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70799999999999996}]},{"osd":8,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":6.9850000000000003,"5min":6.9850000000000003,"15min":6.9850000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.72199999999999998}]},{"osd":9,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1,"5min":0.85599999999999998,"15min":0.83199999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":7.016,"5min":7.016,"15min":7.016},"last":0.78000000000000003}]},{"osd":10,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.878,"15min":0.85399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81399999999999995,"15min":0.82999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95899999999999996,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":501126,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.505,"15min":0.46500000000000002},"min":{"1min":0.34899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.753,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.58099999999999996,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.54200000000000004,"15min":0.497},"min":{"1min":0.34699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77000000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47199999999999998,"15min":0.44},"min":{"1min":0.33400000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.44900000000000001},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.85499999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.51600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.97699999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.64900000000000002,"15min":0.59699999999999998},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.7679999999999998,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":1.032},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":1.0109999999999999}]},{"osd":7,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55600000000000005,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.1220000000000001}]},{"osd":8,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.69499999999999995,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.73899999999999999,"15min":0.70599999999999996},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":1.107}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.94799999999999995}]},{"osd":10,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.75600000000000001,"15min":0.74099999999999999},"min":{"1min":0.64400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.78800000000000003,"15min":0.77300000000000002},"min":{"1min":0.69999999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.0600000000000001}]},{"osd":11,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.048},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.083}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":445610,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 09:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.435},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.32800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.7149999999999999,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58399999999999996,"15min":0.52800000000000002},"min":{"1min":0.47899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2230000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.60699999999999998,"15min":0.52900000000000003},"min":{"1min":0.42099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95499999999999996,"5min":2.653,"15min":2.653},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.38}]},{"osd":4,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78800000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.51300000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.97299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.504}]},{"osd":7,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.718,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.718},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":8,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0649999999999999,"5min":1.101,"15min":1.101},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.81599999999999995,"15min":0.83699999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60099999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.91400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93600000000000005,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.48999999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.877,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.58399999999999996}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.72699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503502,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.78200000000000003,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.68999999999999995,"5min":1.655,"15min":1.655},"last":0.39800000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.40100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85999999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.45100000000000001},"min":{"1min":0.35999999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.50700000000000001,"15min":0.47199999999999998},"min":{"1min":0.34499999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.503}]},{"osd":6,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85699999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.014,"15min":1.014},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95899999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.623}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93000000000000005,"5min":0.995,"15min":0.995},"last":0.755},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67900000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.61799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.61799999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73399999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70899999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444208,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.39100000000000001,"15min":0.35299999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.372},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.28199999999999997,"5min":0.191,"15min":0.191},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.307}]},{"osd":1,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55000000000000004,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96599999999999997,"5min":1.036,"15min":1.036},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79500000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64300000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83799999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86399999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.55300000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":1,"15min":1},"last":0.48499999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.53500000000000003},"min":{"1min":0.438,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4990000000000001,"5min":2.4990000000000001,"15min":2.4990000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.437}]},{"osd":7,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0620000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58899999999999997}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.46899999999999997}]},{"osd":10,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.232,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.999,"5min":1.115,"15min":1.115},"last":0.629}]},{"osd":11,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66700000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":445166,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.998,"15min":0.998},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.36099999999999999},"min":{"1min":0.28699999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.41199999999999998,"15min":0.38500000000000001},"min":{"1min":0.32600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.60699999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.56200000000000006},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60399999999999998,"15min":0.56599999999999995},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76600000000000001}]},{"osd":3,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.89700000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63,"15min":0.59099999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.753},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63500000000000001,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80500000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86699999999999999,"5min":1.071,"15min":1.071},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59099999999999997,"15min":0.56000000000000005},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92500000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69999999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.626}]},{"osd":9,"last update":"Sat Sep 23 09:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":1.089,"15min":1.089},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.86499999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96499999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70699999999999996}]},{"osd":11,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73999999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757301,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1036489,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.55300000000000005,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.52000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.51000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.56399999999999995,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.163,"5min":1.538,"15min":1.538},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.72199999999999998,"15min":0.67700000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.84799999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.004,"5min":1.238,"15min":1.238},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67300000000000004,"15min":0.65400000000000003},"min":{"1min":0.61399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":1.153,"15min":1.153},"last":0.70299999999999996}]},{"osd":8,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88100000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86199999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005}]},{"osd":9,"last update":"Sat Sep 23 09:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1020000000000003,"5min":4.1020000000000003,"15min":4.1020000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.83099999999999996,"15min":0.80700000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.65700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89600000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.72499999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.88800000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T09:29:13.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T09:29:13.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.646+0000 7f71bcb3c700 1 -- 172.21.15.45:0/1773630450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8101f70 msgr2=0x7f71b81023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.646+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/1773630450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8101f70 0x7f71b81023f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009990 tx=0x7f71a0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.646+0000 7f71bcb3c700 1 -- 172.21.15.45:0/1773630450 shutdown_connections 2023-09-23T09:29:13.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.646+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/1773630450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71b8102930 0x7f71b8109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.646+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/1773630450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b8100e20 0x7f71b8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.646+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/1773630450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8101f70 0x7f71b81023f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.646+0000 7f71bcb3c700 1 -- 172.21.15.45:0/1773630450 >> 172.21.15.45:0/1773630450 conn(0x7f71b80fc370 msgr2=0x7f71b80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.647+0000 7f71bcb3c700 1 -- 172.21.15.45:0/1773630450 shutdown_connections 2023-09-23T09:29:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.647+0000 7f71bcb3c700 1 -- 172.21.15.45:0/1773630450 wait complete. 2023-09-23T09:29:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.647+0000 7f71bcb3c700 1 Processor -- start 2023-09-23T09:29:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.647+0000 7f71bcb3c700 1 -- start start 2023-09-23T09:29:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.647+0000 7f71bcb3c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71b8100e20 0x7f71b8074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.647+0000 7f71bcb3c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b8101f70 0x7f71b8073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71bcb3c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8102930 0x7f71b8073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71bcb3c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8073ef0 con 0x7f71b8102930 2023-09-23T09:29:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71bcb3c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8074060 con 0x7f71b8101f70 2023-09-23T09:29:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71bcb3c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b81a0ab0 con 0x7f71b8100e20 2023-09-23T09:29:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8102930 0x7f71b8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8102930 0x7f71b8073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60292/0 (socket says 172.21.15.45:60292) 2023-09-23T09:29:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 -- 172.21.15.45:0/2921341853 learned_addr learned my addr 172.21.15.45:0/2921341853 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b659c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71b8100e20 0x7f71b8074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b5d9b700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b8101f70 0x7f71b8073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 -- 172.21.15.45:0/2921341853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71b8100e20 msgr2=0x7f71b8074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71b8100e20 0x7f71b8074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 -- 172.21.15.45:0/2921341853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b8101f70 msgr2=0x7f71b8073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b8101f70 0x7f71b8073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 -- 172.21.15.45:0/2921341853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00095f0 con 0x7f71b8102930 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b5d9b700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b8101f70 0x7f71b8073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71b6d9d700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8102930 0x7f71b8073750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f71b0007ee0 tx=0x7f71b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71af7fe700 1 -- 172.21.15.45:0/2921341853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0017070 con 0x7f71b8102930 2023-09-23T09:29:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71af7fe700 1 -- 172.21.15.45:0/2921341853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71b000f040 con 0x7f71b8102930 2023-09-23T09:29:13.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b81a0d90 con 0x7f71b8102930 2023-09-23T09:29:13.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71af7fe700 1 -- 172.21.15.45:0/2921341853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0005740 con 0x7f71b8102930 2023-09-23T09:29:13.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.648+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b81a11f0 con 0x7f71b8102930 2023-09-23T09:29:13.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.649+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b810c620 con 0x7f71b8102930 2023-09-23T09:29:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.651+0000 7f71af7fe700 1 -- 172.21.15.45:0/2921341853 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f71b000dc60 con 0x7f71b8102930 2023-09-23T09:29:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.651+0000 7f71af7fe700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f71a40a1670 0x7f71a40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.652+0000 7f71b659c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f71a40a1670 0x7f71a40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.652+0000 7f71af7fe700 1 -- 172.21.15.45:0/2921341853 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71b00cce40 con 0x7f71b8102930 2023-09-23T09:29:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.652+0000 7f71b659c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f71a40a1670 0x7f71a40a3b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71a8007c60 tx=0x7f71a80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.655+0000 7f71af7fe700 1 -- 172.21.15.45:0/2921341853 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f71b00962d0 con 0x7f71b8102930 2023-09-23T09:29:13.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:13 smithi098 ceph-mon[102386]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-23T09:29:13.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:13 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/151882646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:13.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:13 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3731521395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:13.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:13 smithi045 ceph-mon[98627]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-23T09:29:13.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:13 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/151882646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:13.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:13 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3731521395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:13 smithi123 ceph-mon[103577]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-23T09:29:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:13 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/151882646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:13 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3731521395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:13.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.855+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71b8061260 con 0x7f71a40a1670 2023-09-23T09:29:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.867+0000 7f71af7fe700 1 -- 172.21.15.45:0/2921341853 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344355 (secure 0 0 0) 0x7f71b8061260 con 0x7f71a40a1670 2023-09-23T09:29:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:13.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f71a40a1670 msgr2=0x7f71a40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f71a40a1670 0x7f71a40a3b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71a8007c60 tx=0x7f71a80073d0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8102930 msgr2=0x7f71b8073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:13.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8102930 0x7f71b8073750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f71b0007ee0 tx=0x7f71b000d3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 shutdown_connections 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71b8102930 0x7f71b8073750 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f71a40a1670 0x7f71a40a3b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71b8100e20 0x7f71b8074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 --2- 172.21.15.45:0/2921341853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b8101f70 0x7f71b8073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 >> 172.21.15.45:0/2921341853 conn(0x7f71b80fc370 msgr2=0x7f71b80fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 shutdown_connections 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:13.871+0000 7f71bcb3c700 1 -- 172.21.15.45:0/2921341853 wait complete. 2023-09-23T09:29:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-09-23T09:29:13.885 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-09-23T09:29:12.342399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651998,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":606,"num_read_kb":419,"num_write":334,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180859,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335512,"kb_used_data":11260,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606312,"statfs":{"total":1151940427776,"available":1151596863488,"internally_reserved":0,"allocated":11530240,"data_stored":7285610,"data_compressed":178287,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856847+0000","last_change":"2023-09-23T09:27:50.270599+0000","last_active":"2023-09-23T09:28:54.856847+0000","last_peered":"2023-09-23T09:28:54.856847+0000","last_clean":"2023-09-23T09:28:54.856847+0000","last_became_active":"2023-09-23T09:27:50.269926+0000","last_became_peered":"2023-09-23T09:27:50.269926+0000","last_unstale":"2023-09-23T09:28:54.856847+0000","last_undegraded":"2023-09-23T09:28:54.856847+0000","last_fullsized":"2023-09-23T09:28:54.856847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:12:08.908188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858982+0000","last_change":"2023-09-23T09:27:50.270451+0000","last_active":"2023-09-23T09:28:54.858982+0000","last_peered":"2023-09-23T09:28:54.858982+0000","last_clean":"2023-09-23T09:28:54.858982+0000","last_became_active":"2023-09-23T09:27:50.270055+0000","last_became_peered":"2023-09-23T09:27:50.270055+0000","last_unstale":"2023-09-23T09:28:54.858982+0000","last_undegraded":"2023-09-23T09:28:54.858982+0000","last_fullsized":"2023-09-23T09:28:54.858982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:56:43.388109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859067+0000","last_change":"2023-09-23T09:27:50.278101+0000","last_active":"2023-09-23T09:28:54.859067+0000","last_peered":"2023-09-23T09:28:54.859067+0000","last_clean":"2023-09-23T09:28:54.859067+0000","last_became_active":"2023-09-23T09:27:50.277739+0000","last_became_peered":"2023-09-23T09:27:50.277739+0000","last_unstale":"2023-09-23T09:28:54.859067+0000","last_undegraded":"2023-09-23T09:28:54.859067+0000","last_fullsized":"2023-09-23T09:28:54.859067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:52:31.582623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853757+0000","last_change":"2023-09-23T09:27:50.265962+0000","last_active":"2023-09-23T09:28:54.853757+0000","last_peered":"2023-09-23T09:28:54.853757+0000","last_clean":"2023-09-23T09:28:54.853757+0000","last_became_active":"2023-09-23T09:27:50.265681+0000","last_became_peered":"2023-09-23T09:27:50.265681+0000","last_unstale":"2023-09-23T09:28:54.853757+0000","last_undegraded":"2023-09-23T09:28:54.853757+0000","last_fullsized":"2023-09-23T09:28:54.853757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:12.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854809+0000","last_change":"2023-09-23T09:27:50.270356+0000","last_active":"2023-09-23T09:28:54.854809+0000","last_peered":"2023-09-23T09:28:54.854809+0000","last_clean":"2023-09-23T09:28:54.854809+0000","last_became_active":"2023-09-23T09:27:50.270124+0000","last_became_peered":"2023-09-23T09:27:50.270124+0000","last_unstale":"2023-09-23T09:28:54.854809+0000","last_undegraded":"2023-09-23T09:28:54.854809+0000","last_fullsized":"2023-09-23T09:28:54.854809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:14:00.508926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851625+0000","last_change":"2023-09-23T09:27:50.276122+0000","last_active":"2023-09-23T09:28:54.851625+0000","last_peered":"2023-09-23T09:28:54.851625+0000","last_clean":"2023-09-23T09:28:54.851625+0000","last_became_active":"2023-09-23T09:27:50.275847+0000","last_became_peered":"2023-09-23T09:27:50.275847+0000","last_unstale":"2023-09-23T09:28:54.851625+0000","last_undegraded":"2023-09-23T09:28:54.851625+0000","last_fullsized":"2023-09-23T09:28:54.851625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:04:03.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855358+0000","last_change":"2023-09-23T09:27:50.279420+0000","last_active":"2023-09-23T09:28:54.855358+0000","last_peered":"2023-09-23T09:28:54.855358+0000","last_clean":"2023-09-23T09:28:54.855358+0000","last_became_active":"2023-09-23T09:27:50.279120+0000","last_became_peered":"2023-09-23T09:27:50.279120+0000","last_unstale":"2023-09-23T09:28:54.855358+0000","last_undegraded":"2023-09-23T09:28:54.855358+0000","last_fullsized":"2023-09-23T09:28:54.855358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:48:03.461868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857315+0000","last_change":"2023-09-23T09:27:50.276364+0000","last_active":"2023-09-23T09:28:54.857315+0000","last_peered":"2023-09-23T09:28:54.857315+0000","last_clean":"2023-09-23T09:28:54.857315+0000","last_became_active":"2023-09-23T09:27:50.276060+0000","last_became_peered":"2023-09-23T09:27:50.276060+0000","last_unstale":"2023-09-23T09:28:54.857315+0000","last_undegraded":"2023-09-23T09:28:54.857315+0000","last_fullsized":"2023-09-23T09:28:54.857315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:26.998609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854943+0000","last_change":"2023-09-23T09:27:50.280302+0000","last_active":"2023-09-23T09:28:54.854943+0000","last_peered":"2023-09-23T09:28:54.854943+0000","last_clean":"2023-09-23T09:28:54.854943+0000","last_became_active":"2023-09-23T09:27:50.279295+0000","last_became_peered":"2023-09-23T09:27:50.279295+0000","last_unstale":"2023-09-23T09:28:54.854943+0000","last_undegraded":"2023-09-23T09:28:54.854943+0000","last_fullsized":"2023-09-23T09:28:54.854943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:15.329483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851997+0000","last_change":"2023-09-23T09:27:50.271876+0000","last_active":"2023-09-23T09:28:54.851997+0000","last_peered":"2023-09-23T09:28:54.851997+0000","last_clean":"2023-09-23T09:28:54.851997+0000","last_became_active":"2023-09-23T09:27:50.271576+0000","last_became_peered":"2023-09-23T09:27:50.271576+0000","last_unstale":"2023-09-23T09:28:54.851997+0000","last_undegraded":"2023-09-23T09:28:54.851997+0000","last_fullsized":"2023-09-23T09:28:54.851997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:17:22.656828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855756+0000","last_change":"2023-09-23T09:27:50.271405+0000","last_active":"2023-09-23T09:28:54.855756+0000","last_peered":"2023-09-23T09:28:54.855756+0000","last_clean":"2023-09-23T09:28:54.855756+0000","last_became_active":"2023-09-23T09:27:50.271087+0000","last_became_peered":"2023-09-23T09:27:50.271087+0000","last_unstale":"2023-09-23T09:28:54.855756+0000","last_undegraded":"2023-09-23T09:28:54.855756+0000","last_fullsized":"2023-09-23T09:28:54.855756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:09.363894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853720+0000","last_change":"2023-09-23T09:27:50.265855+0000","last_active":"2023-09-23T09:28:54.853720+0000","last_peered":"2023-09-23T09:28:54.853720+0000","last_clean":"2023-09-23T09:28:54.853720+0000","last_became_active":"2023-09-23T09:27:50.265101+0000","last_became_peered":"2023-09-23T09:27:50.265101+0000","last_unstale":"2023-09-23T09:28:54.853720+0000","last_undegraded":"2023-09-23T09:28:54.853720+0000","last_fullsized":"2023-09-23T09:28:54.853720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:52:40.456013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853875+0000","last_change":"2023-09-23T09:27:50.274405+0000","last_active":"2023-09-23T09:28:54.853875+0000","last_peered":"2023-09-23T09:28:54.853875+0000","last_clean":"2023-09-23T09:28:54.853875+0000","last_became_active":"2023-09-23T09:27:50.274036+0000","last_became_peered":"2023-09-23T09:27:50.274036+0000","last_unstale":"2023-09-23T09:28:54.853875+0000","last_undegraded":"2023-09-23T09:28:54.853875+0000","last_fullsized":"2023-09-23T09:28:54.853875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:38.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858871+0000","last_change":"2023-09-23T09:27:50.272237+0000","last_active":"2023-09-23T09:28:54.858871+0000","last_peered":"2023-09-23T09:28:54.858871+0000","last_clean":"2023-09-23T09:28:54.858871+0000","last_became_active":"2023-09-23T09:27:50.271897+0000","last_became_peered":"2023-09-23T09:27:50.271897+0000","last_unstale":"2023-09-23T09:28:54.858871+0000","last_undegraded":"2023-09-23T09:28:54.858871+0000","last_fullsized":"2023-09-23T09:28:54.858871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:31:31.115205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858427+0000","last_change":"2023-09-23T09:27:50.268902+0000","last_active":"2023-09-23T09:28:54.858427+0000","last_peered":"2023-09-23T09:28:54.858427+0000","last_clean":"2023-09-23T09:28:54.858427+0000","last_became_active":"2023-09-23T09:27:50.268590+0000","last_became_peered":"2023-09-23T09:27:50.268590+0000","last_unstale":"2023-09-23T09:28:54.858427+0000","last_undegraded":"2023-09-23T09:28:54.858427+0000","last_fullsized":"2023-09-23T09:28:54.858427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:54:39.384835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854373+0000","last_change":"2023-09-23T09:27:50.276934+0000","last_active":"2023-09-23T09:28:54.854373+0000","last_peered":"2023-09-23T09:28:54.854373+0000","last_clean":"2023-09-23T09:28:54.854373+0000","last_became_active":"2023-09-23T09:27:50.276645+0000","last_became_peered":"2023-09-23T09:27:50.276645+0000","last_unstale":"2023-09-23T09:28:54.854373+0000","last_undegraded":"2023-09-23T09:28:54.854373+0000","last_fullsized":"2023-09-23T09:28:54.854373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:07.100550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856855+0000","last_change":"2023-09-23T09:27:50.270781+0000","last_active":"2023-09-23T09:28:54.856855+0000","last_peered":"2023-09-23T09:28:54.856855+0000","last_clean":"2023-09-23T09:28:54.856855+0000","last_became_active":"2023-09-23T09:27:50.270319+0000","last_became_peered":"2023-09-23T09:27:50.270319+0000","last_unstale":"2023-09-23T09:28:54.856855+0000","last_undegraded":"2023-09-23T09:28:54.856855+0000","last_fullsized":"2023-09-23T09:28:54.856855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:42:44.882122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854722+0000","last_change":"2023-09-23T09:27:50.278775+0000","last_active":"2023-09-23T09:28:54.854722+0000","last_peered":"2023-09-23T09:28:54.854722+0000","last_clean":"2023-09-23T09:28:54.854722+0000","last_became_active":"2023-09-23T09:27:50.278375+0000","last_became_peered":"2023-09-23T09:27:50.278375+0000","last_unstale":"2023-09-23T09:28:54.854722+0000","last_undegraded":"2023-09-23T09:28:54.854722+0000","last_fullsized":"2023-09-23T09:28:54.854722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:14:00.076810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853885+0000","last_change":"2023-09-23T09:27:50.270141+0000","last_active":"2023-09-23T09:28:54.853885+0000","last_peered":"2023-09-23T09:28:54.853885+0000","last_clean":"2023-09-23T09:28:54.853885+0000","last_became_active":"2023-09-23T09:27:50.269849+0000","last_became_peered":"2023-09-23T09:27:50.269849+0000","last_unstale":"2023-09-23T09:28:54.853885+0000","last_undegraded":"2023-09-23T09:28:54.853885+0000","last_fullsized":"2023-09-23T09:28:54.853885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:49:41.837886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854235+0000","last_change":"2023-09-23T09:27:49.654458+0000","last_active":"2023-09-23T09:28:54.854235+0000","last_peered":"2023-09-23T09:28:54.854235+0000","last_clean":"2023-09-23T09:28:54.854235+0000","last_became_active":"2023-09-23T09:27:49.654084+0000","last_became_peered":"2023-09-23T09:27:49.654084+0000","last_unstale":"2023-09-23T09:28:54.854235+0000","last_undegraded":"2023-09-23T09:28:54.854235+0000","last_fullsized":"2023-09-23T09:28:54.854235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:47:32.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857109+0000","last_change":"2023-09-23T09:27:50.276558+0000","last_active":"2023-09-23T09:28:54.857109+0000","last_peered":"2023-09-23T09:28:54.857109+0000","last_clean":"2023-09-23T09:28:54.857109+0000","last_became_active":"2023-09-23T09:27:50.276255+0000","last_became_peered":"2023-09-23T09:27:50.276255+0000","last_unstale":"2023-09-23T09:28:54.857109+0000","last_undegraded":"2023-09-23T09:28:54.857109+0000","last_fullsized":"2023-09-23T09:28:54.857109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:13:09.870383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857231+0000","last_change":"2023-09-23T09:27:49.659256+0000","last_active":"2023-09-23T09:28:54.857231+0000","last_peered":"2023-09-23T09:28:54.857231+0000","last_clean":"2023-09-23T09:28:54.857231+0000","last_became_active":"2023-09-23T09:27:49.658793+0000","last_became_peered":"2023-09-23T09:27:49.658793+0000","last_unstale":"2023-09-23T09:28:54.857231+0000","last_undegraded":"2023-09-23T09:28:54.857231+0000","last_fullsized":"2023-09-23T09:28:54.857231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:03.416666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859099+0000","last_change":"2023-09-23T09:27:50.277338+0000","last_active":"2023-09-23T09:28:54.859099+0000","last_peered":"2023-09-23T09:28:54.859099+0000","last_clean":"2023-09-23T09:28:54.859099+0000","last_became_active":"2023-09-23T09:27:50.277011+0000","last_became_peered":"2023-09-23T09:27:50.277011+0000","last_unstale":"2023-09-23T09:28:54.859099+0000","last_undegraded":"2023-09-23T09:28:54.859099+0000","last_fullsized":"2023-09-23T09:28:54.859099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:52:23.028794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855646+0000","last_change":"2023-09-23T09:27:49.660679+0000","last_active":"2023-09-23T09:28:54.855646+0000","last_peered":"2023-09-23T09:28:54.855646+0000","last_clean":"2023-09-23T09:28:54.855646+0000","last_became_active":"2023-09-23T09:27:49.660201+0000","last_became_peered":"2023-09-23T09:27:49.660201+0000","last_unstale":"2023-09-23T09:28:54.855646+0000","last_undegraded":"2023-09-23T09:28:54.855646+0000","last_fullsized":"2023-09-23T09:28:54.855646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:33:56.205322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854501+0000","last_change":"2023-09-23T09:27:50.278373+0000","last_active":"2023-09-23T09:28:54.854501+0000","last_peered":"2023-09-23T09:28:54.854501+0000","last_clean":"2023-09-23T09:28:54.854501+0000","last_became_active":"2023-09-23T09:27:50.277837+0000","last_became_peered":"2023-09-23T09:27:50.277837+0000","last_unstale":"2023-09-23T09:28:54.854501+0000","last_undegraded":"2023-09-23T09:28:54.854501+0000","last_fullsized":"2023-09-23T09:28:54.854501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:24:26.914509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855285+0000","last_change":"2023-09-23T09:27:49.657578+0000","last_active":"2023-09-23T09:28:54.855285+0000","last_peered":"2023-09-23T09:28:54.855285+0000","last_clean":"2023-09-23T09:28:54.855285+0000","last_became_active":"2023-09-23T09:27:49.657278+0000","last_became_peered":"2023-09-23T09:27:49.657278+0000","last_unstale":"2023-09-23T09:28:54.855285+0000","last_undegraded":"2023-09-23T09:28:54.855285+0000","last_fullsized":"2023-09-23T09:28:54.855285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:53.440997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853150+0000","last_change":"2023-09-23T09:27:50.279784+0000","last_active":"2023-09-23T09:28:54.853150+0000","last_peered":"2023-09-23T09:28:54.853150+0000","last_clean":"2023-09-23T09:28:54.853150+0000","last_became_active":"2023-09-23T09:27:50.279473+0000","last_became_peered":"2023-09-23T09:27:50.279473+0000","last_unstale":"2023-09-23T09:28:54.853150+0000","last_undegraded":"2023-09-23T09:28:54.853150+0000","last_fullsized":"2023-09-23T09:28:54.853150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:30:13.010556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.374589+0000","last_change":"2023-09-23T09:27:49.655391+0000","last_active":"2023-09-23T09:29:07.374589+0000","last_peered":"2023-09-23T09:29:07.374589+0000","last_clean":"2023-09-23T09:29:07.374589+0000","last_became_active":"2023-09-23T09:27:49.655067+0000","last_became_peered":"2023-09-23T09:27:49.655067+0000","last_unstale":"2023-09-23T09:29:07.374589+0000","last_undegraded":"2023-09-23T09:29:07.374589+0000","last_fullsized":"2023-09-23T09:29:07.374589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.701471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.861185+0000","last_change":"2023-09-23T09:28:55.861185+0000","last_active":"2023-09-23T09:28:55.861185+0000","last_peered":"2023-09-23T09:28:55.861185+0000","last_clean":"2023-09-23T09:28:55.861185+0000","last_became_active":"2023-09-23T09:28:55.860687+0000","last_became_peered":"2023-09-23T09:28:55.860687+0000","last_unstale":"2023-09-23T09:28:55.861185+0000","last_undegraded":"2023-09-23T09:28:55.861185+0000","last_fullsized":"2023-09-23T09:28:55.861185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:58.691882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.862026+0000","last_change":"2023-09-23T09:28:55.862026+0000","last_active":"2023-09-23T09:28:55.862026+0000","last_peered":"2023-09-23T09:28:55.862026+0000","last_clean":"2023-09-23T09:28:55.862026+0000","last_became_active":"2023-09-23T09:28:55.858187+0000","last_became_peered":"2023-09-23T09:28:55.858187+0000","last_unstale":"2023-09-23T09:28:55.862026+0000","last_undegraded":"2023-09-23T09:28:55.862026+0000","last_fullsized":"2023-09-23T09:28:55.862026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:34:37.712098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852709+0000","last_change":"2023-09-23T09:27:50.273705+0000","last_active":"2023-09-23T09:28:54.852709+0000","last_peered":"2023-09-23T09:28:54.852709+0000","last_clean":"2023-09-23T09:28:54.852709+0000","last_became_active":"2023-09-23T09:27:50.272245+0000","last_became_peered":"2023-09-23T09:27:50.272245+0000","last_unstale":"2023-09-23T09:28:54.852709+0000","last_undegraded":"2023-09-23T09:28:54.852709+0000","last_fullsized":"2023-09-23T09:28:54.852709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:45:57.318326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851649+0000","last_change":"2023-09-23T09:27:49.651958+0000","last_active":"2023-09-23T09:28:54.851649+0000","last_peered":"2023-09-23T09:28:54.851649+0000","last_clean":"2023-09-23T09:28:54.851649+0000","last_became_active":"2023-09-23T09:27:49.651058+0000","last_became_peered":"2023-09-23T09:27:49.651058+0000","last_unstale":"2023-09-23T09:28:54.851649+0000","last_undegraded":"2023-09-23T09:28:54.851649+0000","last_fullsized":"2023-09-23T09:28:54.851649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:26:56.360566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857202+0000","last_change":"2023-09-23T09:27:50.270804+0000","last_active":"2023-09-23T09:28:54.857202+0000","last_peered":"2023-09-23T09:28:54.857202+0000","last_clean":"2023-09-23T09:28:54.857202+0000","last_became_active":"2023-09-23T09:27:50.270044+0000","last_became_peered":"2023-09-23T09:27:50.270044+0000","last_unstale":"2023-09-23T09:28:54.857202+0000","last_undegraded":"2023-09-23T09:28:54.857202+0000","last_fullsized":"2023-09-23T09:28:54.857202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:59:20.821949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852245+0000","last_change":"2023-09-23T09:27:49.657030+0000","last_active":"2023-09-23T09:28:54.852245+0000","last_peered":"2023-09-23T09:28:54.852245+0000","last_clean":"2023-09-23T09:28:54.852245+0000","last_became_active":"2023-09-23T09:27:49.656710+0000","last_became_peered":"2023-09-23T09:27:49.656710+0000","last_unstale":"2023-09-23T09:28:54.852245+0000","last_undegraded":"2023-09-23T09:28:54.852245+0000","last_fullsized":"2023-09-23T09:28:54.852245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:24:59.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853990+0000","last_change":"2023-09-23T09:27:50.271688+0000","last_active":"2023-09-23T09:28:54.853990+0000","last_peered":"2023-09-23T09:28:54.853990+0000","last_clean":"2023-09-23T09:28:54.853990+0000","last_became_active":"2023-09-23T09:27:50.271276+0000","last_became_peered":"2023-09-23T09:27:50.271276+0000","last_unstale":"2023-09-23T09:28:54.853990+0000","last_undegraded":"2023-09-23T09:28:54.853990+0000","last_fullsized":"2023-09-23T09:28:54.853990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:40:48.333279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.374069+0000","last_change":"2023-09-23T09:27:49.655183+0000","last_active":"2023-09-23T09:29:07.374069+0000","last_peered":"2023-09-23T09:29:07.374069+0000","last_clean":"2023-09-23T09:29:07.374069+0000","last_became_active":"2023-09-23T09:27:49.654875+0000","last_became_peered":"2023-09-23T09:27:49.654875+0000","last_unstale":"2023-09-23T09:29:07.374069+0000","last_undegraded":"2023-09-23T09:29:07.374069+0000","last_fullsized":"2023-09-23T09:29:07.374069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:12:49.944732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854944+0000","last_change":"2023-09-23T09:27:50.276347+0000","last_active":"2023-09-23T09:28:54.854944+0000","last_peered":"2023-09-23T09:28:54.854944+0000","last_clean":"2023-09-23T09:28:54.854944+0000","last_became_active":"2023-09-23T09:27:50.276095+0000","last_became_peered":"2023-09-23T09:27:50.276095+0000","last_unstale":"2023-09-23T09:28:54.854944+0000","last_undegraded":"2023-09-23T09:28:54.854944+0000","last_fullsized":"2023-09-23T09:28:54.854944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:42.952808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855651+0000","last_change":"2023-09-23T09:27:49.657115+0000","last_active":"2023-09-23T09:28:54.855651+0000","last_peered":"2023-09-23T09:28:54.855651+0000","last_clean":"2023-09-23T09:28:54.855651+0000","last_became_active":"2023-09-23T09:27:49.656816+0000","last_became_peered":"2023-09-23T09:27:49.656816+0000","last_unstale":"2023-09-23T09:28:54.855651+0000","last_undegraded":"2023-09-23T09:28:54.855651+0000","last_fullsized":"2023-09-23T09:28:54.855651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:37.205221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859056+0000","last_change":"2023-09-23T09:27:50.277477+0000","last_active":"2023-09-23T09:28:54.859056+0000","last_peered":"2023-09-23T09:28:54.859056+0000","last_clean":"2023-09-23T09:28:54.859056+0000","last_became_active":"2023-09-23T09:27:50.277200+0000","last_became_peered":"2023-09-23T09:27:50.277200+0000","last_unstale":"2023-09-23T09:28:54.859056+0000","last_undegraded":"2023-09-23T09:28:54.859056+0000","last_fullsized":"2023-09-23T09:28:54.859056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:45:03.167471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853242+0000","last_change":"2023-09-23T09:27:49.663678+0000","last_active":"2023-09-23T09:28:54.853242+0000","last_peered":"2023-09-23T09:28:54.853242+0000","last_clean":"2023-09-23T09:28:54.853242+0000","last_became_active":"2023-09-23T09:27:49.663142+0000","last_became_peered":"2023-09-23T09:27:49.663142+0000","last_unstale":"2023-09-23T09:28:54.853242+0000","last_undegraded":"2023-09-23T09:28:54.853242+0000","last_fullsized":"2023-09-23T09:28:54.853242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:14.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854754+0000","last_change":"2023-09-23T09:27:50.275044+0000","last_active":"2023-09-23T09:28:54.854754+0000","last_peered":"2023-09-23T09:28:54.854754+0000","last_clean":"2023-09-23T09:28:54.854754+0000","last_became_active":"2023-09-23T09:27:50.274752+0000","last_became_peered":"2023-09-23T09:27:50.274752+0000","last_unstale":"2023-09-23T09:28:54.854754+0000","last_undegraded":"2023-09-23T09:28:54.854754+0000","last_fullsized":"2023-09-23T09:28:54.854754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:19.914752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":196,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:08.375708+0000","last_change":"2023-09-23T09:27:49.653694+0000","last_active":"2023-09-23T09:29:08.375708+0000","last_peered":"2023-09-23T09:29:08.375708+0000","last_clean":"2023-09-23T09:29:08.375708+0000","last_became_active":"2023-09-23T09:27:49.653355+0000","last_became_peered":"2023-09-23T09:27:49.653355+0000","last_unstale":"2023-09-23T09:29:08.375708+0000","last_undegraded":"2023-09-23T09:29:08.375708+0000","last_fullsized":"2023-09-23T09:29:08.375708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:19:53.282723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56994,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":132,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855406+0000","last_change":"2023-09-23T09:27:50.272849+0000","last_active":"2023-09-23T09:28:54.855406+0000","last_peered":"2023-09-23T09:28:54.855406+0000","last_clean":"2023-09-23T09:28:54.855406+0000","last_became_active":"2023-09-23T09:27:50.272411+0000","last_became_peered":"2023-09-23T09:27:50.272411+0000","last_unstale":"2023-09-23T09:28:54.855406+0000","last_undegraded":"2023-09-23T09:28:54.855406+0000","last_fullsized":"2023-09-23T09:28:54.855406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:01:41.972357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853996+0000","last_change":"2023-09-23T09:27:49.652769+0000","last_active":"2023-09-23T09:28:54.853996+0000","last_peered":"2023-09-23T09:28:54.853996+0000","last_clean":"2023-09-23T09:28:54.853996+0000","last_became_active":"2023-09-23T09:27:49.652440+0000","last_became_peered":"2023-09-23T09:27:49.652440+0000","last_unstale":"2023-09-23T09:28:54.853996+0000","last_undegraded":"2023-09-23T09:28:54.853996+0000","last_fullsized":"2023-09-23T09:28:54.853996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:02.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851736+0000","last_change":"2023-09-23T09:27:50.272105+0000","last_active":"2023-09-23T09:28:54.851736+0000","last_peered":"2023-09-23T09:28:54.851736+0000","last_clean":"2023-09-23T09:28:54.851736+0000","last_became_active":"2023-09-23T09:27:50.271766+0000","last_became_peered":"2023-09-23T09:27:50.271766+0000","last_unstale":"2023-09-23T09:28:54.851736+0000","last_undegraded":"2023-09-23T09:28:54.851736+0000","last_fullsized":"2023-09-23T09:28:54.851736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:46:46.012381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852525+0000","last_change":"2023-09-23T09:27:49.663473+0000","last_active":"2023-09-23T09:28:54.852525+0000","last_peered":"2023-09-23T09:28:54.852525+0000","last_clean":"2023-09-23T09:28:54.852525+0000","last_became_active":"2023-09-23T09:27:49.663085+0000","last_became_peered":"2023-09-23T09:27:49.663085+0000","last_unstale":"2023-09-23T09:28:54.852525+0000","last_undegraded":"2023-09-23T09:28:54.852525+0000","last_fullsized":"2023-09-23T09:28:54.852525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:28:05.246720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.860449+0000","last_change":"2023-09-23T09:28:55.860449+0000","last_active":"2023-09-23T09:28:55.860449+0000","last_peered":"2023-09-23T09:28:55.860449+0000","last_clean":"2023-09-23T09:28:55.860449+0000","last_became_active":"2023-09-23T09:28:55.860050+0000","last_became_peered":"2023-09-23T09:28:55.860050+0000","last_unstale":"2023-09-23T09:28:55.860449+0000","last_undegraded":"2023-09-23T09:28:55.860449+0000","last_fullsized":"2023-09-23T09:28:55.860449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:43:35.092979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:06.372260+0000","last_change":"2023-09-23T09:27:49.654341+0000","last_active":"2023-09-23T09:29:06.372260+0000","last_peered":"2023-09-23T09:29:06.372260+0000","last_clean":"2023-09-23T09:29:06.372260+0000","last_became_active":"2023-09-23T09:27:49.653936+0000","last_became_peered":"2023-09-23T09:27:49.653936+0000","last_unstale":"2023-09-23T09:29:06.372260+0000","last_undegraded":"2023-09-23T09:29:06.372260+0000","last_fullsized":"2023-09-23T09:29:06.372260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:48.876752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855441+0000","last_change":"2023-09-23T09:27:50.278620+0000","last_active":"2023-09-23T09:28:54.855441+0000","last_peered":"2023-09-23T09:28:54.855441+0000","last_clean":"2023-09-23T09:28:54.855441+0000","last_became_active":"2023-09-23T09:27:50.278289+0000","last_became_peered":"2023-09-23T09:27:50.278289+0000","last_unstale":"2023-09-23T09:28:54.855441+0000","last_undegraded":"2023-09-23T09:28:54.855441+0000","last_fullsized":"2023-09-23T09:28:54.855441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:20:44.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854898+0000","last_change":"2023-09-23T09:27:49.662970+0000","last_active":"2023-09-23T09:28:54.854898+0000","last_peered":"2023-09-23T09:28:54.854898+0000","last_clean":"2023-09-23T09:28:54.854898+0000","last_became_active":"2023-09-23T09:27:49.662664+0000","last_became_peered":"2023-09-23T09:27:49.662664+0000","last_unstale":"2023-09-23T09:28:54.854898+0000","last_undegraded":"2023-09-23T09:28:54.854898+0000","last_fullsized":"2023-09-23T09:28:54.854898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:13.189686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852307+0000","last_change":"2023-09-23T09:27:50.278475+0000","last_active":"2023-09-23T09:28:54.852307+0000","last_peered":"2023-09-23T09:28:54.852307+0000","last_clean":"2023-09-23T09:28:54.852307+0000","last_became_active":"2023-09-23T09:27:50.277802+0000","last_became_peered":"2023-09-23T09:27:50.277802+0000","last_unstale":"2023-09-23T09:28:54.852307+0000","last_undegraded":"2023-09-23T09:28:54.852307+0000","last_fullsized":"2023-09-23T09:28:54.852307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:44.608172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855051+0000","last_change":"2023-09-23T09:27:49.657191+0000","last_active":"2023-09-23T09:28:54.855051+0000","last_peered":"2023-09-23T09:28:54.855051+0000","last_clean":"2023-09-23T09:28:54.855051+0000","last_became_active":"2023-09-23T09:27:49.656863+0000","last_became_peered":"2023-09-23T09:27:49.656863+0000","last_unstale":"2023-09-23T09:28:54.855051+0000","last_undegraded":"2023-09-23T09:28:54.855051+0000","last_fullsized":"2023-09-23T09:28:54.855051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:10.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856697+0000","last_change":"2023-09-23T09:27:50.274609+0000","last_active":"2023-09-23T09:28:54.856697+0000","last_peered":"2023-09-23T09:28:54.856697+0000","last_clean":"2023-09-23T09:28:54.856697+0000","last_became_active":"2023-09-23T09:27:50.274316+0000","last_became_peered":"2023-09-23T09:27:50.274316+0000","last_unstale":"2023-09-23T09:28:54.856697+0000","last_undegraded":"2023-09-23T09:28:54.856697+0000","last_fullsized":"2023-09-23T09:28:54.856697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:10:51.284306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:56.023463+0000","last_change":"2023-09-23T09:28:56.023463+0000","last_active":"2023-09-23T09:28:56.023463+0000","last_peered":"2023-09-23T09:28:56.023463+0000","last_clean":"2023-09-23T09:28:56.023463+0000","last_became_active":"2023-09-23T09:28:56.023001+0000","last_became_peered":"2023-09-23T09:28:56.023001+0000","last_unstale":"2023-09-23T09:28:56.023463+0000","last_undegraded":"2023-09-23T09:28:56.023463+0000","last_fullsized":"2023-09-23T09:28:56.023463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:48:56.424613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851865+0000","last_change":"2023-09-23T09:27:50.268568+0000","last_active":"2023-09-23T09:28:54.851865+0000","last_peered":"2023-09-23T09:28:54.851865+0000","last_clean":"2023-09-23T09:28:54.851865+0000","last_became_active":"2023-09-23T09:27:50.268268+0000","last_became_peered":"2023-09-23T09:27:50.268268+0000","last_unstale":"2023-09-23T09:28:54.851865+0000","last_undegraded":"2023-09-23T09:28:54.851865+0000","last_fullsized":"2023-09-23T09:28:54.851865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:59:22.516794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856046+0000","last_change":"2023-09-23T09:27:49.657160+0000","last_active":"2023-09-23T09:28:54.856046+0000","last_peered":"2023-09-23T09:28:54.856046+0000","last_clean":"2023-09-23T09:28:54.856046+0000","last_became_active":"2023-09-23T09:27:49.656853+0000","last_became_peered":"2023-09-23T09:27:49.656853+0000","last_unstale":"2023-09-23T09:28:54.856046+0000","last_undegraded":"2023-09-23T09:28:54.856046+0000","last_fullsized":"2023-09-23T09:28:54.856046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:17.489250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851769+0000","last_change":"2023-09-23T09:27:50.272495+0000","last_active":"2023-09-23T09:28:54.851769+0000","last_peered":"2023-09-23T09:28:54.851769+0000","last_clean":"2023-09-23T09:28:54.851769+0000","last_became_active":"2023-09-23T09:27:50.272207+0000","last_became_peered":"2023-09-23T09:27:50.272207+0000","last_unstale":"2023-09-23T09:28:54.851769+0000","last_undegraded":"2023-09-23T09:28:54.851769+0000","last_fullsized":"2023-09-23T09:28:54.851769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:24.952253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852430+0000","last_change":"2023-09-23T09:27:50.275551+0000","last_active":"2023-09-23T09:28:54.852430+0000","last_peered":"2023-09-23T09:28:54.852430+0000","last_clean":"2023-09-23T09:28:54.852430+0000","last_became_active":"2023-09-23T09:27:50.275235+0000","last_became_peered":"2023-09-23T09:27:50.275235+0000","last_unstale":"2023-09-23T09:28:54.852430+0000","last_undegraded":"2023-09-23T09:28:54.852430+0000","last_fullsized":"2023-09-23T09:28:54.852430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:41.009566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":93,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.373280+0000","last_change":"2023-09-23T09:27:49.652889+0000","last_active":"2023-09-23T09:29:07.373280+0000","last_peered":"2023-09-23T09:29:07.373280+0000","last_clean":"2023-09-23T09:29:07.373280+0000","last_became_active":"2023-09-23T09:27:49.652571+0000","last_became_peered":"2023-09-23T09:27:49.652571+0000","last_unstale":"2023-09-23T09:29:07.373280+0000","last_undegraded":"2023-09-23T09:29:07.373280+0000","last_fullsized":"2023-09-23T09:29:07.373280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:22.484052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856368+0000","last_change":"2023-09-23T09:27:50.278419+0000","last_active":"2023-09-23T09:28:54.856368+0000","last_peered":"2023-09-23T09:28:54.856368+0000","last_clean":"2023-09-23T09:28:54.856368+0000","last_became_active":"2023-09-23T09:27:50.278071+0000","last_became_peered":"2023-09-23T09:27:50.278071+0000","last_unstale":"2023-09-23T09:28:54.856368+0000","last_undegraded":"2023-09-23T09:28:54.856368+0000","last_fullsized":"2023-09-23T09:28:54.856368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:28:45.896479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854153+0000","last_change":"2023-09-23T09:27:49.651251+0000","last_active":"2023-09-23T09:28:54.854153+0000","last_peered":"2023-09-23T09:28:54.854153+0000","last_clean":"2023-09-23T09:28:54.854153+0000","last_became_active":"2023-09-23T09:27:49.650933+0000","last_became_peered":"2023-09-23T09:27:49.650933+0000","last_unstale":"2023-09-23T09:28:54.854153+0000","last_undegraded":"2023-09-23T09:28:54.854153+0000","last_fullsized":"2023-09-23T09:28:54.854153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:00:54.100400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":321,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855483+0000","last_change":"2023-09-23T09:26:51.296669+0000","last_active":"2023-09-23T09:28:54.855483+0000","last_peered":"2023-09-23T09:28:54.855483+0000","last_clean":"2023-09-23T09:28:54.855483+0000","last_became_active":"2023-09-23T09:26:51.296229+0000","last_became_peered":"2023-09-23T09:26:51.296229+0000","last_unstale":"2023-09-23T09:28:54.855483+0000","last_undegraded":"2023-09-23T09:28:54.855483+0000","last_fullsized":"2023-09-23T09:28:54.855483+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_clean_scrub_stamp":"2023-09-23T09:24:01.204422+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:31:24.888152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.860426+0000","last_change":"2023-09-23T09:28:55.860426+0000","last_active":"2023-09-23T09:28:55.860426+0000","last_peered":"2023-09-23T09:28:55.860426+0000","last_clean":"2023-09-23T09:28:55.860426+0000","last_became_active":"2023-09-23T09:28:55.860066+0000","last_became_peered":"2023-09-23T09:28:55.860066+0000","last_unstale":"2023-09-23T09:28:55.860426+0000","last_undegraded":"2023-09-23T09:28:55.860426+0000","last_fullsized":"2023-09-23T09:28:55.860426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:36:03.549733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855636+0000","last_change":"2023-09-23T09:27:49.659773+0000","last_active":"2023-09-23T09:28:54.855636+0000","last_peered":"2023-09-23T09:28:54.855636+0000","last_clean":"2023-09-23T09:28:54.855636+0000","last_became_active":"2023-09-23T09:27:49.659526+0000","last_became_peered":"2023-09-23T09:27:49.659526+0000","last_unstale":"2023-09-23T09:28:54.855636+0000","last_undegraded":"2023-09-23T09:28:54.855636+0000","last_fullsized":"2023-09-23T09:28:54.855636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:27.423680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855501+0000","last_change":"2023-09-23T09:27:49.661336+0000","last_active":"2023-09-23T09:28:54.855501+0000","last_peered":"2023-09-23T09:28:54.855501+0000","last_clean":"2023-09-23T09:28:54.855501+0000","last_became_active":"2023-09-23T09:27:49.661038+0000","last_became_peered":"2023-09-23T09:27:49.661038+0000","last_unstale":"2023-09-23T09:28:54.855501+0000","last_undegraded":"2023-09-23T09:28:54.855501+0000","last_fullsized":"2023-09-23T09:28:54.855501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:20.634410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852470+0000","last_change":"2023-09-23T09:27:50.272485+0000","last_active":"2023-09-23T09:28:54.852470+0000","last_peered":"2023-09-23T09:28:54.852470+0000","last_clean":"2023-09-23T09:28:54.852470+0000","last_became_active":"2023-09-23T09:27:50.272164+0000","last_became_peered":"2023-09-23T09:27:50.272164+0000","last_unstale":"2023-09-23T09:28:54.852470+0000","last_undegraded":"2023-09-23T09:28:54.852470+0000","last_fullsized":"2023-09-23T09:28:54.852470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:48:00.596034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:56.027111+0000","last_change":"2023-09-23T09:28:56.027111+0000","last_active":"2023-09-23T09:28:56.027111+0000","last_peered":"2023-09-23T09:28:56.027111+0000","last_clean":"2023-09-23T09:28:56.027111+0000","last_became_active":"2023-09-23T09:28:56.023192+0000","last_became_peered":"2023-09-23T09:28:56.023192+0000","last_unstale":"2023-09-23T09:28:56.027111+0000","last_undegraded":"2023-09-23T09:28:56.027111+0000","last_fullsized":"2023-09-23T09:28:56.027111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:53.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857537+0000","last_change":"2023-09-23T09:27:50.274959+0000","last_active":"2023-09-23T09:28:54.857537+0000","last_peered":"2023-09-23T09:28:54.857537+0000","last_clean":"2023-09-23T09:28:54.857537+0000","last_became_active":"2023-09-23T09:27:50.274614+0000","last_became_peered":"2023-09-23T09:27:50.274614+0000","last_unstale":"2023-09-23T09:28:54.857537+0000","last_undegraded":"2023-09-23T09:28:54.857537+0000","last_fullsized":"2023-09-23T09:28:54.857537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:00:09.831103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855933+0000","last_change":"2023-09-23T09:27:49.654673+0000","last_active":"2023-09-23T09:28:54.855933+0000","last_peered":"2023-09-23T09:28:54.855933+0000","last_clean":"2023-09-23T09:28:54.855933+0000","last_became_active":"2023-09-23T09:27:49.654356+0000","last_became_peered":"2023-09-23T09:27:49.654356+0000","last_unstale":"2023-09-23T09:28:54.855933+0000","last_undegraded":"2023-09-23T09:28:54.855933+0000","last_fullsized":"2023-09-23T09:28:54.855933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:39:50.628346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858948+0000","last_change":"2023-09-23T09:27:50.267368+0000","last_active":"2023-09-23T09:28:54.858948+0000","last_peered":"2023-09-23T09:28:54.858948+0000","last_clean":"2023-09-23T09:28:54.858948+0000","last_became_active":"2023-09-23T09:27:50.266962+0000","last_became_peered":"2023-09-23T09:27:50.266962+0000","last_unstale":"2023-09-23T09:28:54.858948+0000","last_undegraded":"2023-09-23T09:28:54.858948+0000","last_fullsized":"2023-09-23T09:28:54.858948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:24.660228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854472+0000","last_change":"2023-09-23T09:27:49.661200+0000","last_active":"2023-09-23T09:28:54.854472+0000","last_peered":"2023-09-23T09:28:54.854472+0000","last_clean":"2023-09-23T09:28:54.854472+0000","last_became_active":"2023-09-23T09:27:49.660902+0000","last_became_peered":"2023-09-23T09:27:49.660902+0000","last_unstale":"2023-09-23T09:28:54.854472+0000","last_undegraded":"2023-09-23T09:28:54.854472+0000","last_fullsized":"2023-09-23T09:28:54.854472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:41.036216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854827+0000","last_change":"2023-09-23T09:27:50.271834+0000","last_active":"2023-09-23T09:28:54.854827+0000","last_peered":"2023-09-23T09:28:54.854827+0000","last_clean":"2023-09-23T09:28:54.854827+0000","last_became_active":"2023-09-23T09:27:50.271519+0000","last_became_peered":"2023-09-23T09:27:50.271519+0000","last_unstale":"2023-09-23T09:28:54.854827+0000","last_undegraded":"2023-09-23T09:28:54.854827+0000","last_fullsized":"2023-09-23T09:28:54.854827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:08.036615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857519+0000","last_change":"2023-09-23T09:27:49.656794+0000","last_active":"2023-09-23T09:28:54.857519+0000","last_peered":"2023-09-23T09:28:54.857519+0000","last_clean":"2023-09-23T09:28:54.857519+0000","last_became_active":"2023-09-23T09:27:49.656480+0000","last_became_peered":"2023-09-23T09:27:49.656480+0000","last_unstale":"2023-09-23T09:28:54.857519+0000","last_undegraded":"2023-09-23T09:28:54.857519+0000","last_fullsized":"2023-09-23T09:28:54.857519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:35:46.948703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856490+0000","last_change":"2023-09-23T09:27:50.271805+0000","last_active":"2023-09-23T09:28:54.856490+0000","last_peered":"2023-09-23T09:28:54.856490+0000","last_clean":"2023-09-23T09:28:54.856490+0000","last_became_active":"2023-09-23T09:27:50.271463+0000","last_became_peered":"2023-09-23T09:27:50.271463+0000","last_unstale":"2023-09-23T09:28:54.856490+0000","last_undegraded":"2023-09-23T09:28:54.856490+0000","last_fullsized":"2023-09-23T09:28:54.856490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:00:03.608504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.854053+0000","last_change":"2023-09-23T09:27:49.656988+0000","last_active":"2023-09-23T09:28:55.854053+0000","last_peered":"2023-09-23T09:28:55.854053+0000","last_clean":"2023-09-23T09:28:55.854053+0000","last_became_active":"2023-09-23T09:27:49.656655+0000","last_became_peered":"2023-09-23T09:27:49.656655+0000","last_unstale":"2023-09-23T09:28:55.854053+0000","last_undegraded":"2023-09-23T09:28:55.854053+0000","last_fullsized":"2023-09-23T09:28:55.854053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:07:33.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856634+0000","last_change":"2023-09-23T09:27:50.275021+0000","last_active":"2023-09-23T09:28:54.856634+0000","last_peered":"2023-09-23T09:28:54.856634+0000","last_clean":"2023-09-23T09:28:54.856634+0000","last_became_active":"2023-09-23T09:27:50.274551+0000","last_became_peered":"2023-09-23T09:27:50.274551+0000","last_unstale":"2023-09-23T09:28:54.856634+0000","last_undegraded":"2023-09-23T09:28:54.856634+0000","last_fullsized":"2023-09-23T09:28:54.856634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:50:45.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854922+0000","last_change":"2023-09-23T09:27:49.659603+0000","last_active":"2023-09-23T09:28:54.854922+0000","last_peered":"2023-09-23T09:28:54.854922+0000","last_clean":"2023-09-23T09:28:54.854922+0000","last_became_active":"2023-09-23T09:27:49.659293+0000","last_became_peered":"2023-09-23T09:27:49.659293+0000","last_unstale":"2023-09-23T09:28:54.854922+0000","last_undegraded":"2023-09-23T09:28:54.854922+0000","last_fullsized":"2023-09-23T09:28:54.854922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:31:25.165888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856201+0000","last_change":"2023-09-23T09:27:50.275069+0000","last_active":"2023-09-23T09:28:54.856201+0000","last_peered":"2023-09-23T09:28:54.856201+0000","last_clean":"2023-09-23T09:28:54.856201+0000","last_became_active":"2023-09-23T09:27:50.274731+0000","last_became_peered":"2023-09-23T09:27:50.274731+0000","last_unstale":"2023-09-23T09:28:54.856201+0000","last_undegraded":"2023-09-23T09:28:54.856201+0000","last_fullsized":"2023-09-23T09:28:54.856201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:45:23.794774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854094+0000","last_change":"2023-09-23T09:27:49.659914+0000","last_active":"2023-09-23T09:28:54.854094+0000","last_peered":"2023-09-23T09:28:54.854094+0000","last_clean":"2023-09-23T09:28:54.854094+0000","last_became_active":"2023-09-23T09:27:49.659604+0000","last_became_peered":"2023-09-23T09:27:49.659604+0000","last_unstale":"2023-09-23T09:28:54.854094+0000","last_undegraded":"2023-09-23T09:28:54.854094+0000","last_fullsized":"2023-09-23T09:28:54.854094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:20.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855822+0000","last_change":"2023-09-23T09:27:50.271309+0000","last_active":"2023-09-23T09:28:54.855822+0000","last_peered":"2023-09-23T09:28:54.855822+0000","last_clean":"2023-09-23T09:28:54.855822+0000","last_became_active":"2023-09-23T09:27:50.270951+0000","last_became_peered":"2023-09-23T09:27:50.270951+0000","last_unstale":"2023-09-23T09:28:54.855822+0000","last_undegraded":"2023-09-23T09:28:54.855822+0000","last_fullsized":"2023-09-23T09:28:54.855822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:18.811428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853059+0000","last_change":"2023-09-23T09:27:49.663849+0000","last_active":"2023-09-23T09:28:54.853059+0000","last_peered":"2023-09-23T09:28:54.853059+0000","last_clean":"2023-09-23T09:28:54.853059+0000","last_became_active":"2023-09-23T09:27:49.663499+0000","last_became_peered":"2023-09-23T09:27:49.663499+0000","last_unstale":"2023-09-23T09:28:54.853059+0000","last_undegraded":"2023-09-23T09:28:54.853059+0000","last_fullsized":"2023-09-23T09:28:54.853059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:31:08.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.864630+0000","last_change":"2023-09-23T09:28:55.864630+0000","last_active":"2023-09-23T09:28:55.864630+0000","last_peered":"2023-09-23T09:28:55.864630+0000","last_clean":"2023-09-23T09:28:55.864630+0000","last_became_active":"2023-09-23T09:28:55.860402+0000","last_became_peered":"2023-09-23T09:28:55.860402+0000","last_unstale":"2023-09-23T09:28:55.864630+0000","last_undegraded":"2023-09-23T09:28:55.864630+0000","last_fullsized":"2023-09-23T09:28:55.864630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:58:52.288357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.910095+0000","last_change":"2023-09-23T09:28:55.910095+0000","last_active":"2023-09-23T09:28:55.910095+0000","last_peered":"2023-09-23T09:28:55.910095+0000","last_clean":"2023-09-23T09:28:55.910095+0000","last_became_active":"2023-09-23T09:28:55.907334+0000","last_became_peered":"2023-09-23T09:28:55.907334+0000","last_unstale":"2023-09-23T09:28:55.910095+0000","last_undegraded":"2023-09-23T09:28:55.910095+0000","last_fullsized":"2023-09-23T09:28:55.910095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:11:04.045821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855746+0000","last_change":"2023-09-23T09:27:50.271063+0000","last_active":"2023-09-23T09:28:54.855746+0000","last_peered":"2023-09-23T09:28:54.855746+0000","last_clean":"2023-09-23T09:28:54.855746+0000","last_became_active":"2023-09-23T09:27:50.270487+0000","last_became_peered":"2023-09-23T09:27:50.270487+0000","last_unstale":"2023-09-23T09:28:54.855746+0000","last_undegraded":"2023-09-23T09:28:54.855746+0000","last_fullsized":"2023-09-23T09:28:54.855746+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:11:05.621855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856550+0000","last_change":"2023-09-23T09:27:50.275900+0000","last_active":"2023-09-23T09:28:54.856550+0000","last_peered":"2023-09-23T09:28:54.856550+0000","last_clean":"2023-09-23T09:28:54.856550+0000","last_became_active":"2023-09-23T09:27:50.275565+0000","last_became_peered":"2023-09-23T09:27:50.275565+0000","last_unstale":"2023-09-23T09:28:54.856550+0000","last_undegraded":"2023-09-23T09:28:54.856550+0000","last_fullsized":"2023-09-23T09:28:54.856550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:57:37.004604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854247+0000","last_change":"2023-09-23T09:27:50.275911+0000","last_active":"2023-09-23T09:28:54.854247+0000","last_peered":"2023-09-23T09:28:54.854247+0000","last_clean":"2023-09-23T09:28:54.854247+0000","last_became_active":"2023-09-23T09:27:50.275626+0000","last_became_peered":"2023-09-23T09:27:50.275626+0000","last_unstale":"2023-09-23T09:28:54.854247+0000","last_undegraded":"2023-09-23T09:28:54.854247+0000","last_fullsized":"2023-09-23T09:28:54.854247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:46.790012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853758+0000","last_change":"2023-09-23T09:27:50.269483+0000","last_active":"2023-09-23T09:28:54.853758+0000","last_peered":"2023-09-23T09:28:54.853758+0000","last_clean":"2023-09-23T09:28:54.853758+0000","last_became_active":"2023-09-23T09:27:50.269199+0000","last_became_peered":"2023-09-23T09:27:50.269199+0000","last_unstale":"2023-09-23T09:28:54.853758+0000","last_undegraded":"2023-09-23T09:28:54.853758+0000","last_fullsized":"2023-09-23T09:28:54.853758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:58:34.304439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.859887+0000","last_change":"2023-09-23T09:28:55.859887+0000","last_active":"2023-09-23T09:28:55.859887+0000","last_peered":"2023-09-23T09:28:55.859887+0000","last_clean":"2023-09-23T09:28:55.859887+0000","last_became_active":"2023-09-23T09:28:55.859497+0000","last_became_peered":"2023-09-23T09:28:55.859497+0000","last_unstale":"2023-09-23T09:28:55.859887+0000","last_undegraded":"2023-09-23T09:28:55.859887+0000","last_fullsized":"2023-09-23T09:28:55.859887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:05:52.153365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852551+0000","last_change":"2023-09-23T09:27:50.269934+0000","last_active":"2023-09-23T09:28:54.852551+0000","last_peered":"2023-09-23T09:28:54.852551+0000","last_clean":"2023-09-23T09:28:54.852551+0000","last_became_active":"2023-09-23T09:27:50.269060+0000","last_became_peered":"2023-09-23T09:27:50.269060+0000","last_unstale":"2023-09-23T09:28:54.852551+0000","last_undegraded":"2023-09-23T09:28:54.852551+0000","last_fullsized":"2023-09-23T09:28:54.852551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:47:31.191479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856576+0000","last_change":"2023-09-23T09:27:50.277885+0000","last_active":"2023-09-23T09:28:54.856576+0000","last_peered":"2023-09-23T09:28:54.856576+0000","last_clean":"2023-09-23T09:28:54.856576+0000","last_became_active":"2023-09-23T09:27:50.277576+0000","last_became_peered":"2023-09-23T09:27:50.277576+0000","last_unstale":"2023-09-23T09:28:54.856576+0000","last_undegraded":"2023-09-23T09:28:54.856576+0000","last_fullsized":"2023-09-23T09:28:54.856576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:03:43.338185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854099+0000","last_change":"2023-09-23T09:27:50.278225+0000","last_active":"2023-09-23T09:28:54.854099+0000","last_peered":"2023-09-23T09:28:54.854099+0000","last_clean":"2023-09-23T09:28:54.854099+0000","last_became_active":"2023-09-23T09:27:50.277891+0000","last_became_peered":"2023-09-23T09:27:50.277891+0000","last_unstale":"2023-09-23T09:28:54.854099+0000","last_undegraded":"2023-09-23T09:28:54.854099+0000","last_fullsized":"2023-09-23T09:28:54.854099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:27:08.961439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857440+0000","last_change":"2023-09-23T09:27:50.273445+0000","last_active":"2023-09-23T09:28:54.857440+0000","last_peered":"2023-09-23T09:28:54.857440+0000","last_clean":"2023-09-23T09:28:54.857440+0000","last_became_active":"2023-09-23T09:27:50.273124+0000","last_became_peered":"2023-09-23T09:27:50.273124+0000","last_unstale":"2023-09-23T09:28:54.857440+0000","last_undegraded":"2023-09-23T09:28:54.857440+0000","last_fullsized":"2023-09-23T09:28:54.857440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:46.030212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852344+0000","last_change":"2023-09-23T09:27:50.276164+0000","last_active":"2023-09-23T09:28:54.852344+0000","last_peered":"2023-09-23T09:28:54.852344+0000","last_clean":"2023-09-23T09:28:54.852344+0000","last_became_active":"2023-09-23T09:27:50.275860+0000","last_became_peered":"2023-09-23T09:27:50.275860+0000","last_unstale":"2023-09-23T09:28:54.852344+0000","last_undegraded":"2023-09-23T09:28:54.852344+0000","last_fullsized":"2023-09-23T09:28:54.852344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:53:50.521872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857428+0000","last_change":"2023-09-23T09:27:50.275221+0000","last_active":"2023-09-23T09:28:54.857428+0000","last_peered":"2023-09-23T09:28:54.857428+0000","last_clean":"2023-09-23T09:28:54.857428+0000","last_became_active":"2023-09-23T09:27:50.274920+0000","last_became_peered":"2023-09-23T09:27:50.274920+0000","last_unstale":"2023-09-23T09:28:54.857428+0000","last_undegraded":"2023-09-23T09:28:54.857428+0000","last_fullsized":"2023-09-23T09:28:54.857428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:45:35.023433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854715+0000","last_change":"2023-09-23T09:27:50.277776+0000","last_active":"2023-09-23T09:28:54.854715+0000","last_peered":"2023-09-23T09:28:54.854715+0000","last_clean":"2023-09-23T09:28:54.854715+0000","last_became_active":"2023-09-23T09:27:50.277291+0000","last_became_peered":"2023-09-23T09:27:50.277291+0000","last_unstale":"2023-09-23T09:28:54.854715+0000","last_undegraded":"2023-09-23T09:28:54.854715+0000","last_fullsized":"2023-09-23T09:28:54.854715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:23:46.984450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852197+0000","last_change":"2023-09-23T09:27:50.271219+0000","last_active":"2023-09-23T09:28:54.852197+0000","last_peered":"2023-09-23T09:28:54.852197+0000","last_clean":"2023-09-23T09:28:54.852197+0000","last_became_active":"2023-09-23T09:27:50.270747+0000","last_became_peered":"2023-09-23T09:27:50.270747+0000","last_unstale":"2023-09-23T09:28:54.852197+0000","last_undegraded":"2023-09-23T09:28:54.852197+0000","last_fullsized":"2023-09-23T09:28:54.852197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:19.237260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856573+0000","last_change":"2023-09-23T09:27:50.263794+0000","last_active":"2023-09-23T09:28:54.856573+0000","last_peered":"2023-09-23T09:28:54.856573+0000","last_clean":"2023-09-23T09:28:54.856573+0000","last_became_active":"2023-09-23T09:27:50.263459+0000","last_became_peered":"2023-09-23T09:27:50.263459+0000","last_unstale":"2023-09-23T09:28:54.856573+0000","last_undegraded":"2023-09-23T09:28:54.856573+0000","last_fullsized":"2023-09-23T09:28:54.856573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:05:37.618959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250071,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61384,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":470,"num_read_kb":300,"num_write":151,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":185424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":444714,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79700000000000004}]},{"osd":1,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.748}]},{"osd":6,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71799999999999997}]},{"osd":8,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88500000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91600000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.81899999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1093585,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42199999999999999},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38100000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.74099999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.497},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60699999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.46200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72799999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.623},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84899999999999998,"5min":1.006,"15min":1.006},"last":0.65900000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81000000000000005,"5min":0.873,"15min":0.873},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.40100000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.39600000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.50800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.29,"15min":1.29},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.198,"15min":1.198},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91500000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91800000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57199999999999995}]},{"osd":9,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93200000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.63900000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93700000000000006,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61899999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59499999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":445073,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51400000000000001,"15min":0.51800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.67700000000000005,"15min":0.66100000000000003},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.139,"5min":0.79800000000000004,"15min":0.74099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.616,"5min":11.616,"15min":11.616},"last":0.443}]},{"osd":2,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":0.83999999999999997,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":11.647,"5min":11.647,"15min":11.647},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.9889999999999999,"5min":3.9889999999999999,"15min":3.9889999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.78800000000000003,"15min":0.751},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":11.702,"5min":11.702,"15min":11.702},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.6789999999999998,"5min":3.6789999999999998,"15min":3.6789999999999998},"last":0.376}]},{"osd":5,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":0.71299999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82099999999999995,"15min":0.83199999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88,"15min":0.88},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":0.999,"15min":0.96699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":11.557,"5min":11.557,"15min":11.557},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.004,"15min":0.96699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":11.675000000000001,"5min":11.675000000000001,"15min":11.675000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.89700000000000002,"15min":0.91600000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.65300000000000002}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.86699999999999999,"15min":0.86299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.67000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444870,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64800000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.55700000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.55900000000000005,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.45300000000000001},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63300000000000001,"5min":0.745,"15min":0.745},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.375,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60399999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80200000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.316,"15min":0.316},"max":{"1min":0.89100000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.76600000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.58099999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.999}]},{"osd":7,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.751,"15min":0.73099999999999998},"min":{"1min":0.60999999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89800000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.0800000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.749,"15min":0.73299999999999998},"min":{"1min":0.624,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":1.0169999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.042}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1036922,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.30299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.36799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67700000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.55200000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.83099999999999996}]},{"osd":3,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.497,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.879,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64400000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69499999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75800000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.53800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78800000000000003}]},{"osd":6,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.52800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83899999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67800000000000005,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.53700000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99299999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.89300000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.871,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.874},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94699999999999995,"5min":1.022,"15min":1.022},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.90800000000000003}]},{"osd":11,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93100000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":444345,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.48199999999999998,"15min":0.432},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.442,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.432}]},{"osd":1,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.65900000000000003,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.626}]},{"osd":2,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.48099999999999998},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.66700000000000004,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.312,"15min":0.312},"max":{"1min":11.151999999999999,"5min":11.151999999999999,"15min":11.151999999999999},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.749}]},{"osd":5,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.749,"15min":0.69799999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":6.952,"5min":6.952,"15min":6.952},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2230000000000001,"5min":1.294,"15min":1.294},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70799999999999996}]},{"osd":8,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":6.9850000000000003,"5min":6.9850000000000003,"15min":6.9850000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.72199999999999998}]},{"osd":9,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1,"5min":0.85599999999999998,"15min":0.83199999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":7.016,"5min":7.016,"15min":7.016},"last":0.78000000000000003}]},{"osd":10,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.878,"15min":0.85399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81399999999999995,"15min":0.82999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95899999999999996,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":501126,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.505,"15min":0.46500000000000002},"min":{"1min":0.34899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.753,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.58099999999999996,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.54200000000000004,"15min":0.497},"min":{"1min":0.34699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77000000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47199999999999998,"15min":0.44},"min":{"1min":0.33400000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.44900000000000001},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.85499999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.51600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.97699999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.64900000000000002,"15min":0.59699999999999998},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.7679999999999998,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":1.032},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":1.0109999999999999}]},{"osd":7,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55600000000000005,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.1220000000000001}]},{"osd":8,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.69499999999999995,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.73899999999999999,"15min":0.70599999999999996},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":1.107}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.94799999999999995}]},{"osd":10,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.75600000000000001,"15min":0.74099999999999999},"min":{"1min":0.64400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.78800000000000003,"15min":0.77300000000000002},"min":{"1min":0.69999999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.0600000000000001}]},{"osd":11,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.048},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.083}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":445610,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 09:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.435},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.32800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.7149999999999999,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58399999999999996,"15min":0.52800000000000002},"min":{"1min":0.47899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2230000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.60699999999999998,"15min":0.52900000000000003},"min":{"1min":0.42099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95499999999999996,"5min":2.653,"15min":2.653},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.38}]},{"osd":4,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78800000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.51300000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.97299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.504}]},{"osd":7,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.718,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.718},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":8,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0649999999999999,"5min":1.101,"15min":1.101},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.81599999999999995,"15min":0.83699999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60099999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.91400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93600000000000005,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.48999999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.877,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.58399999999999996}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.72699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503502,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.78200000000000003,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.68999999999999995,"5min":1.655,"15min":1.655},"last":0.39800000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.40100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85999999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.45100000000000001},"min":{"1min":0.35999999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.50700000000000001,"15min":0.47199999999999998},"min":{"1min":0.34499999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.503}]},{"osd":6,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85699999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.014,"15min":1.014},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95899999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.623}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93000000000000005,"5min":0.995,"15min":0.995},"last":0.755},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67900000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.61799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.61799999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73399999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70899999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444208,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.39100000000000001,"15min":0.35299999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.372},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.28199999999999997,"5min":0.191,"15min":0.191},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.307}]},{"osd":1,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55000000000000004,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96599999999999997,"5min":1.036,"15min":1.036},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79500000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64300000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83799999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86399999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.55300000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":1,"15min":1},"last":0.48499999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.53500000000000003},"min":{"1min":0.438,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4990000000000001,"5min":2.4990000000000001,"15min":2.4990000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.437}]},{"osd":7,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0620000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58899999999999997}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.46899999999999997}]},{"osd":10,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.232,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.999,"5min":1.115,"15min":1.115},"last":0.629}]},{"osd":11,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66700000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":445166,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.998,"15min":0.998},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.36099999999999999},"min":{"1min":0.28699999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.41199999999999998,"15min":0.38500000000000001},"min":{"1min":0.32600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.60699999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.56200000000000006},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60399999999999998,"15min":0.56599999999999995},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76600000000000001}]},{"osd":3,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.89700000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63,"15min":0.59099999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.753},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63500000000000001,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80500000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86699999999999999,"5min":1.071,"15min":1.071},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59099999999999997,"15min":0.56000000000000005},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92500000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69999999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.626}]},{"osd":9,"last update":"Sat Sep 23 09:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":1.089,"15min":1.089},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.86499999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96499999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70699999999999996}]},{"osd":11,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73999999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757301,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1036489,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.55300000000000005,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.52000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.51000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.56399999999999995,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.163,"5min":1.538,"15min":1.538},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.72199999999999998,"15min":0.67700000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.84799999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.004,"5min":1.238,"15min":1.238},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67300000000000004,"15min":0.65400000000000003},"min":{"1min":0.61399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":1.153,"15min":1.153},"last":0.70299999999999996}]},{"osd":8,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88100000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86199999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005}]},{"osd":9,"last update":"Sat Sep 23 09:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1020000000000003,"5min":4.1020000000000003,"15min":4.1020000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.83099999999999996,"15min":0.80700000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.65700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89600000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.72499999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.88800000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T09:29:13.893 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-23T09:29:13.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 -- 172.21.15.45:0/196436650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e00ffd00 msgr2=0x7f42e0100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 --2- 172.21.15.45:0/196436650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e00ffd00 0x7f42e0100120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42d00099c0 tx=0x7f42d0009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 -- 172.21.15.45:0/196436650 shutdown_connections 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 --2- 172.21.15.45:0/196436650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e0101890 0x7f42e0108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 --2- 172.21.15.45:0/196436650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e00ffd00 0x7f42e0100120 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 --2- 172.21.15.45:0/196436650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e0100ef0 0x7f42e0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 -- 172.21.15.45:0/196436650 >> 172.21.15.45:0/196436650 conn(0x7f42e00fb250 msgr2=0x7f42e00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 -- 172.21.15.45:0/196436650 shutdown_connections 2023-09-23T09:29:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.050+0000 7f42e8209700 1 -- 172.21.15.45:0/196436650 wait complete. 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 Processor -- start 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 -- start start 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e00ffd00 0x7f42e0196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e0100ef0 0x7f42e0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e0101890 0x7f42e019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e019e690 con 0x7f42e0100ef0 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e019e7d0 con 0x7f42e00ffd00 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e8209700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e019e960 con 0x7f42e0101890 2023-09-23T09:29:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e0100ef0 0x7f42e0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e0100ef0 0x7f42e0197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60304/0 (socket says 172.21.15.45:60304) 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 -- 172.21.15.45:0/2987780654 learned_addr learned my addr 172.21.15.45:0/2987780654 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e5fa5700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e00ffd00 0x7f42e0196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e67a6700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e0101890 0x7f42e019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 -- 172.21.15.45:0/2987780654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e0101890 msgr2=0x7f42e019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e0101890 0x7f42e019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 -- 172.21.15.45:0/2987780654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e00ffd00 msgr2=0x7f42e0196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e00ffd00 0x7f42e0196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 -- 172.21.15.45:0/2987780654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00095f0 con 0x7f42e0100ef0 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e67a6700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e0101890 0x7f42e019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.051+0000 7f42e57a4700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e0100ef0 0x7f42e0197050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00d950 tx=0x7f42dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.052+0000 7f42d6ffd700 1 -- 172.21.15.45:0/2987780654 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc009940 con 0x7f42e0100ef0 2023-09-23T09:29:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.052+0000 7f42d6ffd700 1 -- 172.21.15.45:0/2987780654 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42dc010460 con 0x7f42e0100ef0 2023-09-23T09:29:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.052+0000 7f42d6ffd700 1 -- 172.21.15.45:0/2987780654 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00f620 con 0x7f42e0100ef0 2023-09-23T09:29:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.052+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e019ec40 con 0x7f42e0100ef0 2023-09-23T09:29:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.052+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e019f100 con 0x7f42e0100ef0 2023-09-23T09:29:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.053+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e0107e80 con 0x7f42e0100ef0 2023-09-23T09:29:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.054+0000 7f42d6ffd700 1 -- 172.21.15.45:0/2987780654 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f42dc0108d0 con 0x7f42e0100ef0 2023-09-23T09:29:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.054+0000 7f42d6ffd700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f42cc0a1670 0x7f42cc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.054+0000 7f42d6ffd700 1 -- 172.21.15.45:0/2987780654 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42dc0cced0 con 0x7f42e0100ef0 2023-09-23T09:29:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.054+0000 7f42e5fa5700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f42cc0a1670 0x7f42cc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.055+0000 7f42e5fa5700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f42cc0a1670 0x7f42cc0a3b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42d000b580 tx=0x7f42d0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.058+0000 7f42d6ffd700 1 -- 172.21.15.45:0/2987780654 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f42dc096360 con 0x7f42e0100ef0 2023-09-23T09:29:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.256+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42e002d270 con 0x7f42cc0a1670 2023-09-23T09:29:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.268+0000 7f42d6ffd700 1 -- 172.21.15.45:0/2987780654 <== mgr.14152 v2:172.21.15.45:6800/944196506 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344355 (secure 0 0 0) 0x7f42e002d270 con 0x7f42cc0a1670 2023-09-23T09:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f42cc0a1670 msgr2=0x7f42cc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f42cc0a1670 0x7f42cc0a3b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42d000b580 tx=0x7f42d0009f10 comp rx=0 tx=0).stop 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e0100ef0 msgr2=0x7f42e0197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e0100ef0 0x7f42e0197050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00d950 tx=0x7f42dc00dc60 comp rx=0 tx=0).stop 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 shutdown_connections 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f42cc0a1670 0x7f42cc0a3b30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e0101890 0x7f42e019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e00ffd00 0x7f42e0196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 --2- 172.21.15.45:0/2987780654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e0100ef0 0x7f42e0197050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 >> 172.21.15.45:0/2987780654 conn(0x7f42e00fb250 msgr2=0x7f42e00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 shutdown_connections 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.272+0000 7f42e8209700 1 -- 172.21.15.45:0/2987780654 wait complete. 2023-09-23T09:29:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-09-23T09:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-09-23T09:29:12.342399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651998,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":606,"num_read_kb":419,"num_write":334,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180859,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335512,"kb_used_data":11260,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606312,"statfs":{"total":1151940427776,"available":1151596863488,"internally_reserved":0,"allocated":11530240,"data_stored":7285610,"data_compressed":178287,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856847+0000","last_change":"2023-09-23T09:27:50.270599+0000","last_active":"2023-09-23T09:28:54.856847+0000","last_peered":"2023-09-23T09:28:54.856847+0000","last_clean":"2023-09-23T09:28:54.856847+0000","last_became_active":"2023-09-23T09:27:50.269926+0000","last_became_peered":"2023-09-23T09:27:50.269926+0000","last_unstale":"2023-09-23T09:28:54.856847+0000","last_undegraded":"2023-09-23T09:28:54.856847+0000","last_fullsized":"2023-09-23T09:28:54.856847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:12:08.908188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858982+0000","last_change":"2023-09-23T09:27:50.270451+0000","last_active":"2023-09-23T09:28:54.858982+0000","last_peered":"2023-09-23T09:28:54.858982+0000","last_clean":"2023-09-23T09:28:54.858982+0000","last_became_active":"2023-09-23T09:27:50.270055+0000","last_became_peered":"2023-09-23T09:27:50.270055+0000","last_unstale":"2023-09-23T09:28:54.858982+0000","last_undegraded":"2023-09-23T09:28:54.858982+0000","last_fullsized":"2023-09-23T09:28:54.858982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:56:43.388109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859067+0000","last_change":"2023-09-23T09:27:50.278101+0000","last_active":"2023-09-23T09:28:54.859067+0000","last_peered":"2023-09-23T09:28:54.859067+0000","last_clean":"2023-09-23T09:28:54.859067+0000","last_became_active":"2023-09-23T09:27:50.277739+0000","last_became_peered":"2023-09-23T09:27:50.277739+0000","last_unstale":"2023-09-23T09:28:54.859067+0000","last_undegraded":"2023-09-23T09:28:54.859067+0000","last_fullsized":"2023-09-23T09:28:54.859067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:52:31.582623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853757+0000","last_change":"2023-09-23T09:27:50.265962+0000","last_active":"2023-09-23T09:28:54.853757+0000","last_peered":"2023-09-23T09:28:54.853757+0000","last_clean":"2023-09-23T09:28:54.853757+0000","last_became_active":"2023-09-23T09:27:50.265681+0000","last_became_peered":"2023-09-23T09:27:50.265681+0000","last_unstale":"2023-09-23T09:28:54.853757+0000","last_undegraded":"2023-09-23T09:28:54.853757+0000","last_fullsized":"2023-09-23T09:28:54.853757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:12.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854809+0000","last_change":"2023-09-23T09:27:50.270356+0000","last_active":"2023-09-23T09:28:54.854809+0000","last_peered":"2023-09-23T09:28:54.854809+0000","last_clean":"2023-09-23T09:28:54.854809+0000","last_became_active":"2023-09-23T09:27:50.270124+0000","last_became_peered":"2023-09-23T09:27:50.270124+0000","last_unstale":"2023-09-23T09:28:54.854809+0000","last_undegraded":"2023-09-23T09:28:54.854809+0000","last_fullsized":"2023-09-23T09:28:54.854809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:14:00.508926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851625+0000","last_change":"2023-09-23T09:27:50.276122+0000","last_active":"2023-09-23T09:28:54.851625+0000","last_peered":"2023-09-23T09:28:54.851625+0000","last_clean":"2023-09-23T09:28:54.851625+0000","last_became_active":"2023-09-23T09:27:50.275847+0000","last_became_peered":"2023-09-23T09:27:50.275847+0000","last_unstale":"2023-09-23T09:28:54.851625+0000","last_undegraded":"2023-09-23T09:28:54.851625+0000","last_fullsized":"2023-09-23T09:28:54.851625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:04:03.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855358+0000","last_change":"2023-09-23T09:27:50.279420+0000","last_active":"2023-09-23T09:28:54.855358+0000","last_peered":"2023-09-23T09:28:54.855358+0000","last_clean":"2023-09-23T09:28:54.855358+0000","last_became_active":"2023-09-23T09:27:50.279120+0000","last_became_peered":"2023-09-23T09:27:50.279120+0000","last_unstale":"2023-09-23T09:28:54.855358+0000","last_undegraded":"2023-09-23T09:28:54.855358+0000","last_fullsized":"2023-09-23T09:28:54.855358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:48:03.461868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857315+0000","last_change":"2023-09-23T09:27:50.276364+0000","last_active":"2023-09-23T09:28:54.857315+0000","last_peered":"2023-09-23T09:28:54.857315+0000","last_clean":"2023-09-23T09:28:54.857315+0000","last_became_active":"2023-09-23T09:27:50.276060+0000","last_became_peered":"2023-09-23T09:27:50.276060+0000","last_unstale":"2023-09-23T09:28:54.857315+0000","last_undegraded":"2023-09-23T09:28:54.857315+0000","last_fullsized":"2023-09-23T09:28:54.857315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:26.998609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854943+0000","last_change":"2023-09-23T09:27:50.280302+0000","last_active":"2023-09-23T09:28:54.854943+0000","last_peered":"2023-09-23T09:28:54.854943+0000","last_clean":"2023-09-23T09:28:54.854943+0000","last_became_active":"2023-09-23T09:27:50.279295+0000","last_became_peered":"2023-09-23T09:27:50.279295+0000","last_unstale":"2023-09-23T09:28:54.854943+0000","last_undegraded":"2023-09-23T09:28:54.854943+0000","last_fullsized":"2023-09-23T09:28:54.854943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:15.329483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851997+0000","last_change":"2023-09-23T09:27:50.271876+0000","last_active":"2023-09-23T09:28:54.851997+0000","last_peered":"2023-09-23T09:28:54.851997+0000","last_clean":"2023-09-23T09:28:54.851997+0000","last_became_active":"2023-09-23T09:27:50.271576+0000","last_became_peered":"2023-09-23T09:27:50.271576+0000","last_unstale":"2023-09-23T09:28:54.851997+0000","last_undegraded":"2023-09-23T09:28:54.851997+0000","last_fullsized":"2023-09-23T09:28:54.851997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:17:22.656828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855756+0000","last_change":"2023-09-23T09:27:50.271405+0000","last_active":"2023-09-23T09:28:54.855756+0000","last_peered":"2023-09-23T09:28:54.855756+0000","last_clean":"2023-09-23T09:28:54.855756+0000","last_became_active":"2023-09-23T09:27:50.271087+0000","last_became_peered":"2023-09-23T09:27:50.271087+0000","last_unstale":"2023-09-23T09:28:54.855756+0000","last_undegraded":"2023-09-23T09:28:54.855756+0000","last_fullsized":"2023-09-23T09:28:54.855756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:17:09.363894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853720+0000","last_change":"2023-09-23T09:27:50.265855+0000","last_active":"2023-09-23T09:28:54.853720+0000","last_peered":"2023-09-23T09:28:54.853720+0000","last_clean":"2023-09-23T09:28:54.853720+0000","last_became_active":"2023-09-23T09:27:50.265101+0000","last_became_peered":"2023-09-23T09:27:50.265101+0000","last_unstale":"2023-09-23T09:28:54.853720+0000","last_undegraded":"2023-09-23T09:28:54.853720+0000","last_fullsized":"2023-09-23T09:28:54.853720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:52:40.456013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853875+0000","last_change":"2023-09-23T09:27:50.274405+0000","last_active":"2023-09-23T09:28:54.853875+0000","last_peered":"2023-09-23T09:28:54.853875+0000","last_clean":"2023-09-23T09:28:54.853875+0000","last_became_active":"2023-09-23T09:27:50.274036+0000","last_became_peered":"2023-09-23T09:27:50.274036+0000","last_unstale":"2023-09-23T09:28:54.853875+0000","last_undegraded":"2023-09-23T09:28:54.853875+0000","last_fullsized":"2023-09-23T09:28:54.853875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:38.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858871+0000","last_change":"2023-09-23T09:27:50.272237+0000","last_active":"2023-09-23T09:28:54.858871+0000","last_peered":"2023-09-23T09:28:54.858871+0000","last_clean":"2023-09-23T09:28:54.858871+0000","last_became_active":"2023-09-23T09:27:50.271897+0000","last_became_peered":"2023-09-23T09:27:50.271897+0000","last_unstale":"2023-09-23T09:28:54.858871+0000","last_undegraded":"2023-09-23T09:28:54.858871+0000","last_fullsized":"2023-09-23T09:28:54.858871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:31:31.115205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858427+0000","last_change":"2023-09-23T09:27:50.268902+0000","last_active":"2023-09-23T09:28:54.858427+0000","last_peered":"2023-09-23T09:28:54.858427+0000","last_clean":"2023-09-23T09:28:54.858427+0000","last_became_active":"2023-09-23T09:27:50.268590+0000","last_became_peered":"2023-09-23T09:27:50.268590+0000","last_unstale":"2023-09-23T09:28:54.858427+0000","last_undegraded":"2023-09-23T09:28:54.858427+0000","last_fullsized":"2023-09-23T09:28:54.858427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:54:39.384835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854373+0000","last_change":"2023-09-23T09:27:50.276934+0000","last_active":"2023-09-23T09:28:54.854373+0000","last_peered":"2023-09-23T09:28:54.854373+0000","last_clean":"2023-09-23T09:28:54.854373+0000","last_became_active":"2023-09-23T09:27:50.276645+0000","last_became_peered":"2023-09-23T09:27:50.276645+0000","last_unstale":"2023-09-23T09:28:54.854373+0000","last_undegraded":"2023-09-23T09:28:54.854373+0000","last_fullsized":"2023-09-23T09:28:54.854373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:07.100550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856855+0000","last_change":"2023-09-23T09:27:50.270781+0000","last_active":"2023-09-23T09:28:54.856855+0000","last_peered":"2023-09-23T09:28:54.856855+0000","last_clean":"2023-09-23T09:28:54.856855+0000","last_became_active":"2023-09-23T09:27:50.270319+0000","last_became_peered":"2023-09-23T09:27:50.270319+0000","last_unstale":"2023-09-23T09:28:54.856855+0000","last_undegraded":"2023-09-23T09:28:54.856855+0000","last_fullsized":"2023-09-23T09:28:54.856855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:42:44.882122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854722+0000","last_change":"2023-09-23T09:27:50.278775+0000","last_active":"2023-09-23T09:28:54.854722+0000","last_peered":"2023-09-23T09:28:54.854722+0000","last_clean":"2023-09-23T09:28:54.854722+0000","last_became_active":"2023-09-23T09:27:50.278375+0000","last_became_peered":"2023-09-23T09:27:50.278375+0000","last_unstale":"2023-09-23T09:28:54.854722+0000","last_undegraded":"2023-09-23T09:28:54.854722+0000","last_fullsized":"2023-09-23T09:28:54.854722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:14:00.076810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853885+0000","last_change":"2023-09-23T09:27:50.270141+0000","last_active":"2023-09-23T09:28:54.853885+0000","last_peered":"2023-09-23T09:28:54.853885+0000","last_clean":"2023-09-23T09:28:54.853885+0000","last_became_active":"2023-09-23T09:27:50.269849+0000","last_became_peered":"2023-09-23T09:27:50.269849+0000","last_unstale":"2023-09-23T09:28:54.853885+0000","last_undegraded":"2023-09-23T09:28:54.853885+0000","last_fullsized":"2023-09-23T09:28:54.853885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:49:41.837886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854235+0000","last_change":"2023-09-23T09:27:49.654458+0000","last_active":"2023-09-23T09:28:54.854235+0000","last_peered":"2023-09-23T09:28:54.854235+0000","last_clean":"2023-09-23T09:28:54.854235+0000","last_became_active":"2023-09-23T09:27:49.654084+0000","last_became_peered":"2023-09-23T09:27:49.654084+0000","last_unstale":"2023-09-23T09:28:54.854235+0000","last_undegraded":"2023-09-23T09:28:54.854235+0000","last_fullsized":"2023-09-23T09:28:54.854235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:47:32.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857109+0000","last_change":"2023-09-23T09:27:50.276558+0000","last_active":"2023-09-23T09:28:54.857109+0000","last_peered":"2023-09-23T09:28:54.857109+0000","last_clean":"2023-09-23T09:28:54.857109+0000","last_became_active":"2023-09-23T09:27:50.276255+0000","last_became_peered":"2023-09-23T09:27:50.276255+0000","last_unstale":"2023-09-23T09:28:54.857109+0000","last_undegraded":"2023-09-23T09:28:54.857109+0000","last_fullsized":"2023-09-23T09:28:54.857109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:13:09.870383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857231+0000","last_change":"2023-09-23T09:27:49.659256+0000","last_active":"2023-09-23T09:28:54.857231+0000","last_peered":"2023-09-23T09:28:54.857231+0000","last_clean":"2023-09-23T09:28:54.857231+0000","last_became_active":"2023-09-23T09:27:49.658793+0000","last_became_peered":"2023-09-23T09:27:49.658793+0000","last_unstale":"2023-09-23T09:28:54.857231+0000","last_undegraded":"2023-09-23T09:28:54.857231+0000","last_fullsized":"2023-09-23T09:28:54.857231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:03.416666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859099+0000","last_change":"2023-09-23T09:27:50.277338+0000","last_active":"2023-09-23T09:28:54.859099+0000","last_peered":"2023-09-23T09:28:54.859099+0000","last_clean":"2023-09-23T09:28:54.859099+0000","last_became_active":"2023-09-23T09:27:50.277011+0000","last_became_peered":"2023-09-23T09:27:50.277011+0000","last_unstale":"2023-09-23T09:28:54.859099+0000","last_undegraded":"2023-09-23T09:28:54.859099+0000","last_fullsized":"2023-09-23T09:28:54.859099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:52:23.028794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855646+0000","last_change":"2023-09-23T09:27:49.660679+0000","last_active":"2023-09-23T09:28:54.855646+0000","last_peered":"2023-09-23T09:28:54.855646+0000","last_clean":"2023-09-23T09:28:54.855646+0000","last_became_active":"2023-09-23T09:27:49.660201+0000","last_became_peered":"2023-09-23T09:27:49.660201+0000","last_unstale":"2023-09-23T09:28:54.855646+0000","last_undegraded":"2023-09-23T09:28:54.855646+0000","last_fullsized":"2023-09-23T09:28:54.855646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:33:56.205322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854501+0000","last_change":"2023-09-23T09:27:50.278373+0000","last_active":"2023-09-23T09:28:54.854501+0000","last_peered":"2023-09-23T09:28:54.854501+0000","last_clean":"2023-09-23T09:28:54.854501+0000","last_became_active":"2023-09-23T09:27:50.277837+0000","last_became_peered":"2023-09-23T09:27:50.277837+0000","last_unstale":"2023-09-23T09:28:54.854501+0000","last_undegraded":"2023-09-23T09:28:54.854501+0000","last_fullsized":"2023-09-23T09:28:54.854501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:24:26.914509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855285+0000","last_change":"2023-09-23T09:27:49.657578+0000","last_active":"2023-09-23T09:28:54.855285+0000","last_peered":"2023-09-23T09:28:54.855285+0000","last_clean":"2023-09-23T09:28:54.855285+0000","last_became_active":"2023-09-23T09:27:49.657278+0000","last_became_peered":"2023-09-23T09:27:49.657278+0000","last_unstale":"2023-09-23T09:28:54.855285+0000","last_undegraded":"2023-09-23T09:28:54.855285+0000","last_fullsized":"2023-09-23T09:28:54.855285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:53.440997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853150+0000","last_change":"2023-09-23T09:27:50.279784+0000","last_active":"2023-09-23T09:28:54.853150+0000","last_peered":"2023-09-23T09:28:54.853150+0000","last_clean":"2023-09-23T09:28:54.853150+0000","last_became_active":"2023-09-23T09:27:50.279473+0000","last_became_peered":"2023-09-23T09:27:50.279473+0000","last_unstale":"2023-09-23T09:28:54.853150+0000","last_undegraded":"2023-09-23T09:28:54.853150+0000","last_fullsized":"2023-09-23T09:28:54.853150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:30:13.010556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.374589+0000","last_change":"2023-09-23T09:27:49.655391+0000","last_active":"2023-09-23T09:29:07.374589+0000","last_peered":"2023-09-23T09:29:07.374589+0000","last_clean":"2023-09-23T09:29:07.374589+0000","last_became_active":"2023-09-23T09:27:49.655067+0000","last_became_peered":"2023-09-23T09:27:49.655067+0000","last_unstale":"2023-09-23T09:29:07.374589+0000","last_undegraded":"2023-09-23T09:29:07.374589+0000","last_fullsized":"2023-09-23T09:29:07.374589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.701471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.861185+0000","last_change":"2023-09-23T09:28:55.861185+0000","last_active":"2023-09-23T09:28:55.861185+0000","last_peered":"2023-09-23T09:28:55.861185+0000","last_clean":"2023-09-23T09:28:55.861185+0000","last_became_active":"2023-09-23T09:28:55.860687+0000","last_became_peered":"2023-09-23T09:28:55.860687+0000","last_unstale":"2023-09-23T09:28:55.861185+0000","last_undegraded":"2023-09-23T09:28:55.861185+0000","last_fullsized":"2023-09-23T09:28:55.861185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:58.691882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.862026+0000","last_change":"2023-09-23T09:28:55.862026+0000","last_active":"2023-09-23T09:28:55.862026+0000","last_peered":"2023-09-23T09:28:55.862026+0000","last_clean":"2023-09-23T09:28:55.862026+0000","last_became_active":"2023-09-23T09:28:55.858187+0000","last_became_peered":"2023-09-23T09:28:55.858187+0000","last_unstale":"2023-09-23T09:28:55.862026+0000","last_undegraded":"2023-09-23T09:28:55.862026+0000","last_fullsized":"2023-09-23T09:28:55.862026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:34:37.712098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852709+0000","last_change":"2023-09-23T09:27:50.273705+0000","last_active":"2023-09-23T09:28:54.852709+0000","last_peered":"2023-09-23T09:28:54.852709+0000","last_clean":"2023-09-23T09:28:54.852709+0000","last_became_active":"2023-09-23T09:27:50.272245+0000","last_became_peered":"2023-09-23T09:27:50.272245+0000","last_unstale":"2023-09-23T09:28:54.852709+0000","last_undegraded":"2023-09-23T09:28:54.852709+0000","last_fullsized":"2023-09-23T09:28:54.852709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:45:57.318326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851649+0000","last_change":"2023-09-23T09:27:49.651958+0000","last_active":"2023-09-23T09:28:54.851649+0000","last_peered":"2023-09-23T09:28:54.851649+0000","last_clean":"2023-09-23T09:28:54.851649+0000","last_became_active":"2023-09-23T09:27:49.651058+0000","last_became_peered":"2023-09-23T09:27:49.651058+0000","last_unstale":"2023-09-23T09:28:54.851649+0000","last_undegraded":"2023-09-23T09:28:54.851649+0000","last_fullsized":"2023-09-23T09:28:54.851649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:26:56.360566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857202+0000","last_change":"2023-09-23T09:27:50.270804+0000","last_active":"2023-09-23T09:28:54.857202+0000","last_peered":"2023-09-23T09:28:54.857202+0000","last_clean":"2023-09-23T09:28:54.857202+0000","last_became_active":"2023-09-23T09:27:50.270044+0000","last_became_peered":"2023-09-23T09:27:50.270044+0000","last_unstale":"2023-09-23T09:28:54.857202+0000","last_undegraded":"2023-09-23T09:28:54.857202+0000","last_fullsized":"2023-09-23T09:28:54.857202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:59:20.821949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852245+0000","last_change":"2023-09-23T09:27:49.657030+0000","last_active":"2023-09-23T09:28:54.852245+0000","last_peered":"2023-09-23T09:28:54.852245+0000","last_clean":"2023-09-23T09:28:54.852245+0000","last_became_active":"2023-09-23T09:27:49.656710+0000","last_became_peered":"2023-09-23T09:27:49.656710+0000","last_unstale":"2023-09-23T09:28:54.852245+0000","last_undegraded":"2023-09-23T09:28:54.852245+0000","last_fullsized":"2023-09-23T09:28:54.852245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:24:59.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853990+0000","last_change":"2023-09-23T09:27:50.271688+0000","last_active":"2023-09-23T09:28:54.853990+0000","last_peered":"2023-09-23T09:28:54.853990+0000","last_clean":"2023-09-23T09:28:54.853990+0000","last_became_active":"2023-09-23T09:27:50.271276+0000","last_became_peered":"2023-09-23T09:27:50.271276+0000","last_unstale":"2023-09-23T09:28:54.853990+0000","last_undegraded":"2023-09-23T09:28:54.853990+0000","last_fullsized":"2023-09-23T09:28:54.853990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:40:48.333279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.374069+0000","last_change":"2023-09-23T09:27:49.655183+0000","last_active":"2023-09-23T09:29:07.374069+0000","last_peered":"2023-09-23T09:29:07.374069+0000","last_clean":"2023-09-23T09:29:07.374069+0000","last_became_active":"2023-09-23T09:27:49.654875+0000","last_became_peered":"2023-09-23T09:27:49.654875+0000","last_unstale":"2023-09-23T09:29:07.374069+0000","last_undegraded":"2023-09-23T09:29:07.374069+0000","last_fullsized":"2023-09-23T09:29:07.374069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:12:49.944732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854944+0000","last_change":"2023-09-23T09:27:50.276347+0000","last_active":"2023-09-23T09:28:54.854944+0000","last_peered":"2023-09-23T09:28:54.854944+0000","last_clean":"2023-09-23T09:28:54.854944+0000","last_became_active":"2023-09-23T09:27:50.276095+0000","last_became_peered":"2023-09-23T09:27:50.276095+0000","last_unstale":"2023-09-23T09:28:54.854944+0000","last_undegraded":"2023-09-23T09:28:54.854944+0000","last_fullsized":"2023-09-23T09:28:54.854944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:42.952808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855651+0000","last_change":"2023-09-23T09:27:49.657115+0000","last_active":"2023-09-23T09:28:54.855651+0000","last_peered":"2023-09-23T09:28:54.855651+0000","last_clean":"2023-09-23T09:28:54.855651+0000","last_became_active":"2023-09-23T09:27:49.656816+0000","last_became_peered":"2023-09-23T09:27:49.656816+0000","last_unstale":"2023-09-23T09:28:54.855651+0000","last_undegraded":"2023-09-23T09:28:54.855651+0000","last_fullsized":"2023-09-23T09:28:54.855651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:37.205221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.859056+0000","last_change":"2023-09-23T09:27:50.277477+0000","last_active":"2023-09-23T09:28:54.859056+0000","last_peered":"2023-09-23T09:28:54.859056+0000","last_clean":"2023-09-23T09:28:54.859056+0000","last_became_active":"2023-09-23T09:27:50.277200+0000","last_became_peered":"2023-09-23T09:27:50.277200+0000","last_unstale":"2023-09-23T09:28:54.859056+0000","last_undegraded":"2023-09-23T09:28:54.859056+0000","last_fullsized":"2023-09-23T09:28:54.859056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:45:03.167471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853242+0000","last_change":"2023-09-23T09:27:49.663678+0000","last_active":"2023-09-23T09:28:54.853242+0000","last_peered":"2023-09-23T09:28:54.853242+0000","last_clean":"2023-09-23T09:28:54.853242+0000","last_became_active":"2023-09-23T09:27:49.663142+0000","last_became_peered":"2023-09-23T09:27:49.663142+0000","last_unstale":"2023-09-23T09:28:54.853242+0000","last_undegraded":"2023-09-23T09:28:54.853242+0000","last_fullsized":"2023-09-23T09:28:54.853242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:14.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854754+0000","last_change":"2023-09-23T09:27:50.275044+0000","last_active":"2023-09-23T09:28:54.854754+0000","last_peered":"2023-09-23T09:28:54.854754+0000","last_clean":"2023-09-23T09:28:54.854754+0000","last_became_active":"2023-09-23T09:27:50.274752+0000","last_became_peered":"2023-09-23T09:27:50.274752+0000","last_unstale":"2023-09-23T09:28:54.854754+0000","last_undegraded":"2023-09-23T09:28:54.854754+0000","last_fullsized":"2023-09-23T09:28:54.854754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:19.914752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":196,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:08.375708+0000","last_change":"2023-09-23T09:27:49.653694+0000","last_active":"2023-09-23T09:29:08.375708+0000","last_peered":"2023-09-23T09:29:08.375708+0000","last_clean":"2023-09-23T09:29:08.375708+0000","last_became_active":"2023-09-23T09:27:49.653355+0000","last_became_peered":"2023-09-23T09:27:49.653355+0000","last_unstale":"2023-09-23T09:29:08.375708+0000","last_undegraded":"2023-09-23T09:29:08.375708+0000","last_fullsized":"2023-09-23T09:29:08.375708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:19:53.282723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56994,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":132,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855406+0000","last_change":"2023-09-23T09:27:50.272849+0000","last_active":"2023-09-23T09:28:54.855406+0000","last_peered":"2023-09-23T09:28:54.855406+0000","last_clean":"2023-09-23T09:28:54.855406+0000","last_became_active":"2023-09-23T09:27:50.272411+0000","last_became_peered":"2023-09-23T09:27:50.272411+0000","last_unstale":"2023-09-23T09:28:54.855406+0000","last_undegraded":"2023-09-23T09:28:54.855406+0000","last_fullsized":"2023-09-23T09:28:54.855406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:01:41.972357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853996+0000","last_change":"2023-09-23T09:27:49.652769+0000","last_active":"2023-09-23T09:28:54.853996+0000","last_peered":"2023-09-23T09:28:54.853996+0000","last_clean":"2023-09-23T09:28:54.853996+0000","last_became_active":"2023-09-23T09:27:49.652440+0000","last_became_peered":"2023-09-23T09:27:49.652440+0000","last_unstale":"2023-09-23T09:28:54.853996+0000","last_undegraded":"2023-09-23T09:28:54.853996+0000","last_fullsized":"2023-09-23T09:28:54.853996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:02.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851736+0000","last_change":"2023-09-23T09:27:50.272105+0000","last_active":"2023-09-23T09:28:54.851736+0000","last_peered":"2023-09-23T09:28:54.851736+0000","last_clean":"2023-09-23T09:28:54.851736+0000","last_became_active":"2023-09-23T09:27:50.271766+0000","last_became_peered":"2023-09-23T09:27:50.271766+0000","last_unstale":"2023-09-23T09:28:54.851736+0000","last_undegraded":"2023-09-23T09:28:54.851736+0000","last_fullsized":"2023-09-23T09:28:54.851736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:46:46.012381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852525+0000","last_change":"2023-09-23T09:27:49.663473+0000","last_active":"2023-09-23T09:28:54.852525+0000","last_peered":"2023-09-23T09:28:54.852525+0000","last_clean":"2023-09-23T09:28:54.852525+0000","last_became_active":"2023-09-23T09:27:49.663085+0000","last_became_peered":"2023-09-23T09:27:49.663085+0000","last_unstale":"2023-09-23T09:28:54.852525+0000","last_undegraded":"2023-09-23T09:28:54.852525+0000","last_fullsized":"2023-09-23T09:28:54.852525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:28:05.246720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.860449+0000","last_change":"2023-09-23T09:28:55.860449+0000","last_active":"2023-09-23T09:28:55.860449+0000","last_peered":"2023-09-23T09:28:55.860449+0000","last_clean":"2023-09-23T09:28:55.860449+0000","last_became_active":"2023-09-23T09:28:55.860050+0000","last_became_peered":"2023-09-23T09:28:55.860050+0000","last_unstale":"2023-09-23T09:28:55.860449+0000","last_undegraded":"2023-09-23T09:28:55.860449+0000","last_fullsized":"2023-09-23T09:28:55.860449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:43:35.092979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:06.372260+0000","last_change":"2023-09-23T09:27:49.654341+0000","last_active":"2023-09-23T09:29:06.372260+0000","last_peered":"2023-09-23T09:29:06.372260+0000","last_clean":"2023-09-23T09:29:06.372260+0000","last_became_active":"2023-09-23T09:27:49.653936+0000","last_became_peered":"2023-09-23T09:27:49.653936+0000","last_unstale":"2023-09-23T09:29:06.372260+0000","last_undegraded":"2023-09-23T09:29:06.372260+0000","last_fullsized":"2023-09-23T09:29:06.372260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:21:48.876752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855441+0000","last_change":"2023-09-23T09:27:50.278620+0000","last_active":"2023-09-23T09:28:54.855441+0000","last_peered":"2023-09-23T09:28:54.855441+0000","last_clean":"2023-09-23T09:28:54.855441+0000","last_became_active":"2023-09-23T09:27:50.278289+0000","last_became_peered":"2023-09-23T09:27:50.278289+0000","last_unstale":"2023-09-23T09:28:54.855441+0000","last_undegraded":"2023-09-23T09:28:54.855441+0000","last_fullsized":"2023-09-23T09:28:54.855441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:20:44.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854898+0000","last_change":"2023-09-23T09:27:49.662970+0000","last_active":"2023-09-23T09:28:54.854898+0000","last_peered":"2023-09-23T09:28:54.854898+0000","last_clean":"2023-09-23T09:28:54.854898+0000","last_became_active":"2023-09-23T09:27:49.662664+0000","last_became_peered":"2023-09-23T09:27:49.662664+0000","last_unstale":"2023-09-23T09:28:54.854898+0000","last_undegraded":"2023-09-23T09:28:54.854898+0000","last_fullsized":"2023-09-23T09:28:54.854898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:13.189686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852307+0000","last_change":"2023-09-23T09:27:50.278475+0000","last_active":"2023-09-23T09:28:54.852307+0000","last_peered":"2023-09-23T09:28:54.852307+0000","last_clean":"2023-09-23T09:28:54.852307+0000","last_became_active":"2023-09-23T09:27:50.277802+0000","last_became_peered":"2023-09-23T09:27:50.277802+0000","last_unstale":"2023-09-23T09:28:54.852307+0000","last_undegraded":"2023-09-23T09:28:54.852307+0000","last_fullsized":"2023-09-23T09:28:54.852307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:44.608172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855051+0000","last_change":"2023-09-23T09:27:49.657191+0000","last_active":"2023-09-23T09:28:54.855051+0000","last_peered":"2023-09-23T09:28:54.855051+0000","last_clean":"2023-09-23T09:28:54.855051+0000","last_became_active":"2023-09-23T09:27:49.656863+0000","last_became_peered":"2023-09-23T09:27:49.656863+0000","last_unstale":"2023-09-23T09:28:54.855051+0000","last_undegraded":"2023-09-23T09:28:54.855051+0000","last_fullsized":"2023-09-23T09:28:54.855051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:19:10.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856697+0000","last_change":"2023-09-23T09:27:50.274609+0000","last_active":"2023-09-23T09:28:54.856697+0000","last_peered":"2023-09-23T09:28:54.856697+0000","last_clean":"2023-09-23T09:28:54.856697+0000","last_became_active":"2023-09-23T09:27:50.274316+0000","last_became_peered":"2023-09-23T09:27:50.274316+0000","last_unstale":"2023-09-23T09:28:54.856697+0000","last_undegraded":"2023-09-23T09:28:54.856697+0000","last_fullsized":"2023-09-23T09:28:54.856697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:10:51.284306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:56.023463+0000","last_change":"2023-09-23T09:28:56.023463+0000","last_active":"2023-09-23T09:28:56.023463+0000","last_peered":"2023-09-23T09:28:56.023463+0000","last_clean":"2023-09-23T09:28:56.023463+0000","last_became_active":"2023-09-23T09:28:56.023001+0000","last_became_peered":"2023-09-23T09:28:56.023001+0000","last_unstale":"2023-09-23T09:28:56.023463+0000","last_undegraded":"2023-09-23T09:28:56.023463+0000","last_fullsized":"2023-09-23T09:28:56.023463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T17:48:56.424613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851865+0000","last_change":"2023-09-23T09:27:50.268568+0000","last_active":"2023-09-23T09:28:54.851865+0000","last_peered":"2023-09-23T09:28:54.851865+0000","last_clean":"2023-09-23T09:28:54.851865+0000","last_became_active":"2023-09-23T09:27:50.268268+0000","last_became_peered":"2023-09-23T09:27:50.268268+0000","last_unstale":"2023-09-23T09:28:54.851865+0000","last_undegraded":"2023-09-23T09:28:54.851865+0000","last_fullsized":"2023-09-23T09:28:54.851865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:59:22.516794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856046+0000","last_change":"2023-09-23T09:27:49.657160+0000","last_active":"2023-09-23T09:28:54.856046+0000","last_peered":"2023-09-23T09:28:54.856046+0000","last_clean":"2023-09-23T09:28:54.856046+0000","last_became_active":"2023-09-23T09:27:49.656853+0000","last_became_peered":"2023-09-23T09:27:49.656853+0000","last_unstale":"2023-09-23T09:28:54.856046+0000","last_undegraded":"2023-09-23T09:28:54.856046+0000","last_fullsized":"2023-09-23T09:28:54.856046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:17.489250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.851769+0000","last_change":"2023-09-23T09:27:50.272495+0000","last_active":"2023-09-23T09:28:54.851769+0000","last_peered":"2023-09-23T09:28:54.851769+0000","last_clean":"2023-09-23T09:28:54.851769+0000","last_became_active":"2023-09-23T09:27:50.272207+0000","last_became_peered":"2023-09-23T09:27:50.272207+0000","last_unstale":"2023-09-23T09:28:54.851769+0000","last_undegraded":"2023-09-23T09:28:54.851769+0000","last_fullsized":"2023-09-23T09:28:54.851769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:24:24.952253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852430+0000","last_change":"2023-09-23T09:27:50.275551+0000","last_active":"2023-09-23T09:28:54.852430+0000","last_peered":"2023-09-23T09:28:54.852430+0000","last_clean":"2023-09-23T09:28:54.852430+0000","last_became_active":"2023-09-23T09:27:50.275235+0000","last_became_peered":"2023-09-23T09:27:50.275235+0000","last_unstale":"2023-09-23T09:28:54.852430+0000","last_undegraded":"2023-09-23T09:28:54.852430+0000","last_fullsized":"2023-09-23T09:28:54.852430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:41.009566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":93,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:29:07.373280+0000","last_change":"2023-09-23T09:27:49.652889+0000","last_active":"2023-09-23T09:29:07.373280+0000","last_peered":"2023-09-23T09:29:07.373280+0000","last_clean":"2023-09-23T09:29:07.373280+0000","last_became_active":"2023-09-23T09:27:49.652571+0000","last_became_peered":"2023-09-23T09:27:49.652571+0000","last_unstale":"2023-09-23T09:29:07.373280+0000","last_undegraded":"2023-09-23T09:29:07.373280+0000","last_fullsized":"2023-09-23T09:29:07.373280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:22.484052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856368+0000","last_change":"2023-09-23T09:27:50.278419+0000","last_active":"2023-09-23T09:28:54.856368+0000","last_peered":"2023-09-23T09:28:54.856368+0000","last_clean":"2023-09-23T09:28:54.856368+0000","last_became_active":"2023-09-23T09:27:50.278071+0000","last_became_peered":"2023-09-23T09:27:50.278071+0000","last_unstale":"2023-09-23T09:28:54.856368+0000","last_undegraded":"2023-09-23T09:28:54.856368+0000","last_fullsized":"2023-09-23T09:28:54.856368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:28:45.896479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854153+0000","last_change":"2023-09-23T09:27:49.651251+0000","last_active":"2023-09-23T09:28:54.854153+0000","last_peered":"2023-09-23T09:28:54.854153+0000","last_clean":"2023-09-23T09:28:54.854153+0000","last_became_active":"2023-09-23T09:27:49.650933+0000","last_became_peered":"2023-09-23T09:27:49.650933+0000","last_unstale":"2023-09-23T09:28:54.854153+0000","last_undegraded":"2023-09-23T09:28:54.854153+0000","last_fullsized":"2023-09-23T09:28:54.854153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:00:54.100400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":321,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855483+0000","last_change":"2023-09-23T09:26:51.296669+0000","last_active":"2023-09-23T09:28:54.855483+0000","last_peered":"2023-09-23T09:28:54.855483+0000","last_clean":"2023-09-23T09:28:54.855483+0000","last_became_active":"2023-09-23T09:26:51.296229+0000","last_became_peered":"2023-09-23T09:26:51.296229+0000","last_unstale":"2023-09-23T09:28:54.855483+0000","last_undegraded":"2023-09-23T09:28:54.855483+0000","last_fullsized":"2023-09-23T09:28:54.855483+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:24:01.204422+0000","last_clean_scrub_stamp":"2023-09-23T09:24:01.204422+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:31:24.888152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.860426+0000","last_change":"2023-09-23T09:28:55.860426+0000","last_active":"2023-09-23T09:28:55.860426+0000","last_peered":"2023-09-23T09:28:55.860426+0000","last_clean":"2023-09-23T09:28:55.860426+0000","last_became_active":"2023-09-23T09:28:55.860066+0000","last_became_peered":"2023-09-23T09:28:55.860066+0000","last_unstale":"2023-09-23T09:28:55.860426+0000","last_undegraded":"2023-09-23T09:28:55.860426+0000","last_fullsized":"2023-09-23T09:28:55.860426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:36:03.549733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855636+0000","last_change":"2023-09-23T09:27:49.659773+0000","last_active":"2023-09-23T09:28:54.855636+0000","last_peered":"2023-09-23T09:28:54.855636+0000","last_clean":"2023-09-23T09:28:54.855636+0000","last_became_active":"2023-09-23T09:27:49.659526+0000","last_became_peered":"2023-09-23T09:27:49.659526+0000","last_unstale":"2023-09-23T09:28:54.855636+0000","last_undegraded":"2023-09-23T09:28:54.855636+0000","last_fullsized":"2023-09-23T09:28:54.855636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:27.423680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855501+0000","last_change":"2023-09-23T09:27:49.661336+0000","last_active":"2023-09-23T09:28:54.855501+0000","last_peered":"2023-09-23T09:28:54.855501+0000","last_clean":"2023-09-23T09:28:54.855501+0000","last_became_active":"2023-09-23T09:27:49.661038+0000","last_became_peered":"2023-09-23T09:27:49.661038+0000","last_unstale":"2023-09-23T09:28:54.855501+0000","last_undegraded":"2023-09-23T09:28:54.855501+0000","last_fullsized":"2023-09-23T09:28:54.855501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:20.634410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852470+0000","last_change":"2023-09-23T09:27:50.272485+0000","last_active":"2023-09-23T09:28:54.852470+0000","last_peered":"2023-09-23T09:28:54.852470+0000","last_clean":"2023-09-23T09:28:54.852470+0000","last_became_active":"2023-09-23T09:27:50.272164+0000","last_became_peered":"2023-09-23T09:27:50.272164+0000","last_unstale":"2023-09-23T09:28:54.852470+0000","last_undegraded":"2023-09-23T09:28:54.852470+0000","last_fullsized":"2023-09-23T09:28:54.852470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:48:00.596034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:56.027111+0000","last_change":"2023-09-23T09:28:56.027111+0000","last_active":"2023-09-23T09:28:56.027111+0000","last_peered":"2023-09-23T09:28:56.027111+0000","last_clean":"2023-09-23T09:28:56.027111+0000","last_became_active":"2023-09-23T09:28:56.023192+0000","last_became_peered":"2023-09-23T09:28:56.023192+0000","last_unstale":"2023-09-23T09:28:56.027111+0000","last_undegraded":"2023-09-23T09:28:56.027111+0000","last_fullsized":"2023-09-23T09:28:56.027111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:53.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857537+0000","last_change":"2023-09-23T09:27:50.274959+0000","last_active":"2023-09-23T09:28:54.857537+0000","last_peered":"2023-09-23T09:28:54.857537+0000","last_clean":"2023-09-23T09:28:54.857537+0000","last_became_active":"2023-09-23T09:27:50.274614+0000","last_became_peered":"2023-09-23T09:27:50.274614+0000","last_unstale":"2023-09-23T09:28:54.857537+0000","last_undegraded":"2023-09-23T09:28:54.857537+0000","last_fullsized":"2023-09-23T09:28:54.857537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:00:09.831103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855933+0000","last_change":"2023-09-23T09:27:49.654673+0000","last_active":"2023-09-23T09:28:54.855933+0000","last_peered":"2023-09-23T09:28:54.855933+0000","last_clean":"2023-09-23T09:28:54.855933+0000","last_became_active":"2023-09-23T09:27:49.654356+0000","last_became_peered":"2023-09-23T09:27:49.654356+0000","last_unstale":"2023-09-23T09:28:54.855933+0000","last_undegraded":"2023-09-23T09:28:54.855933+0000","last_fullsized":"2023-09-23T09:28:54.855933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:39:50.628346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.858948+0000","last_change":"2023-09-23T09:27:50.267368+0000","last_active":"2023-09-23T09:28:54.858948+0000","last_peered":"2023-09-23T09:28:54.858948+0000","last_clean":"2023-09-23T09:28:54.858948+0000","last_became_active":"2023-09-23T09:27:50.266962+0000","last_became_peered":"2023-09-23T09:27:50.266962+0000","last_unstale":"2023-09-23T09:28:54.858948+0000","last_undegraded":"2023-09-23T09:28:54.858948+0000","last_fullsized":"2023-09-23T09:28:54.858948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:24.660228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854472+0000","last_change":"2023-09-23T09:27:49.661200+0000","last_active":"2023-09-23T09:28:54.854472+0000","last_peered":"2023-09-23T09:28:54.854472+0000","last_clean":"2023-09-23T09:28:54.854472+0000","last_became_active":"2023-09-23T09:27:49.660902+0000","last_became_peered":"2023-09-23T09:27:49.660902+0000","last_unstale":"2023-09-23T09:28:54.854472+0000","last_undegraded":"2023-09-23T09:28:54.854472+0000","last_fullsized":"2023-09-23T09:28:54.854472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:41.036216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854827+0000","last_change":"2023-09-23T09:27:50.271834+0000","last_active":"2023-09-23T09:28:54.854827+0000","last_peered":"2023-09-23T09:28:54.854827+0000","last_clean":"2023-09-23T09:28:54.854827+0000","last_became_active":"2023-09-23T09:27:50.271519+0000","last_became_peered":"2023-09-23T09:27:50.271519+0000","last_unstale":"2023-09-23T09:28:54.854827+0000","last_undegraded":"2023-09-23T09:28:54.854827+0000","last_fullsized":"2023-09-23T09:28:54.854827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:09:08.036615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857519+0000","last_change":"2023-09-23T09:27:49.656794+0000","last_active":"2023-09-23T09:28:54.857519+0000","last_peered":"2023-09-23T09:28:54.857519+0000","last_clean":"2023-09-23T09:28:54.857519+0000","last_became_active":"2023-09-23T09:27:49.656480+0000","last_became_peered":"2023-09-23T09:27:49.656480+0000","last_unstale":"2023-09-23T09:28:54.857519+0000","last_undegraded":"2023-09-23T09:28:54.857519+0000","last_fullsized":"2023-09-23T09:28:54.857519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:35:46.948703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856490+0000","last_change":"2023-09-23T09:27:50.271805+0000","last_active":"2023-09-23T09:28:54.856490+0000","last_peered":"2023-09-23T09:28:54.856490+0000","last_clean":"2023-09-23T09:28:54.856490+0000","last_became_active":"2023-09-23T09:27:50.271463+0000","last_became_peered":"2023-09-23T09:27:50.271463+0000","last_unstale":"2023-09-23T09:28:54.856490+0000","last_undegraded":"2023-09-23T09:28:54.856490+0000","last_fullsized":"2023-09-23T09:28:54.856490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:00:03.608504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.854053+0000","last_change":"2023-09-23T09:27:49.656988+0000","last_active":"2023-09-23T09:28:55.854053+0000","last_peered":"2023-09-23T09:28:55.854053+0000","last_clean":"2023-09-23T09:28:55.854053+0000","last_became_active":"2023-09-23T09:27:49.656655+0000","last_became_peered":"2023-09-23T09:27:49.656655+0000","last_unstale":"2023-09-23T09:28:55.854053+0000","last_undegraded":"2023-09-23T09:28:55.854053+0000","last_fullsized":"2023-09-23T09:28:55.854053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:07:33.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856634+0000","last_change":"2023-09-23T09:27:50.275021+0000","last_active":"2023-09-23T09:28:54.856634+0000","last_peered":"2023-09-23T09:28:54.856634+0000","last_clean":"2023-09-23T09:28:54.856634+0000","last_became_active":"2023-09-23T09:27:50.274551+0000","last_became_peered":"2023-09-23T09:27:50.274551+0000","last_unstale":"2023-09-23T09:28:54.856634+0000","last_undegraded":"2023-09-23T09:28:54.856634+0000","last_fullsized":"2023-09-23T09:28:54.856634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:50:45.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854922+0000","last_change":"2023-09-23T09:27:49.659603+0000","last_active":"2023-09-23T09:28:54.854922+0000","last_peered":"2023-09-23T09:28:54.854922+0000","last_clean":"2023-09-23T09:28:54.854922+0000","last_became_active":"2023-09-23T09:27:49.659293+0000","last_became_peered":"2023-09-23T09:27:49.659293+0000","last_unstale":"2023-09-23T09:28:54.854922+0000","last_undegraded":"2023-09-23T09:28:54.854922+0000","last_fullsized":"2023-09-23T09:28:54.854922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:31:25.165888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856201+0000","last_change":"2023-09-23T09:27:50.275069+0000","last_active":"2023-09-23T09:28:54.856201+0000","last_peered":"2023-09-23T09:28:54.856201+0000","last_clean":"2023-09-23T09:28:54.856201+0000","last_became_active":"2023-09-23T09:27:50.274731+0000","last_became_peered":"2023-09-23T09:27:50.274731+0000","last_unstale":"2023-09-23T09:28:54.856201+0000","last_undegraded":"2023-09-23T09:28:54.856201+0000","last_fullsized":"2023-09-23T09:28:54.856201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:45:23.794774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854094+0000","last_change":"2023-09-23T09:27:49.659914+0000","last_active":"2023-09-23T09:28:54.854094+0000","last_peered":"2023-09-23T09:28:54.854094+0000","last_clean":"2023-09-23T09:28:54.854094+0000","last_became_active":"2023-09-23T09:27:49.659604+0000","last_became_peered":"2023-09-23T09:27:49.659604+0000","last_unstale":"2023-09-23T09:28:54.854094+0000","last_undegraded":"2023-09-23T09:28:54.854094+0000","last_fullsized":"2023-09-23T09:28:54.854094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:01:20.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855822+0000","last_change":"2023-09-23T09:27:50.271309+0000","last_active":"2023-09-23T09:28:54.855822+0000","last_peered":"2023-09-23T09:28:54.855822+0000","last_clean":"2023-09-23T09:28:54.855822+0000","last_became_active":"2023-09-23T09:27:50.270951+0000","last_became_peered":"2023-09-23T09:27:50.270951+0000","last_unstale":"2023-09-23T09:28:54.855822+0000","last_undegraded":"2023-09-23T09:28:54.855822+0000","last_fullsized":"2023-09-23T09:28:54.855822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:18.811428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853059+0000","last_change":"2023-09-23T09:27:49.663849+0000","last_active":"2023-09-23T09:28:54.853059+0000","last_peered":"2023-09-23T09:28:54.853059+0000","last_clean":"2023-09-23T09:28:54.853059+0000","last_became_active":"2023-09-23T09:27:49.663499+0000","last_became_peered":"2023-09-23T09:27:49.663499+0000","last_unstale":"2023-09-23T09:28:54.853059+0000","last_undegraded":"2023-09-23T09:28:54.853059+0000","last_fullsized":"2023-09-23T09:28:54.853059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:31:08.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.864630+0000","last_change":"2023-09-23T09:28:55.864630+0000","last_active":"2023-09-23T09:28:55.864630+0000","last_peered":"2023-09-23T09:28:55.864630+0000","last_clean":"2023-09-23T09:28:55.864630+0000","last_became_active":"2023-09-23T09:28:55.860402+0000","last_became_peered":"2023-09-23T09:28:55.860402+0000","last_unstale":"2023-09-23T09:28:55.864630+0000","last_undegraded":"2023-09-23T09:28:55.864630+0000","last_fullsized":"2023-09-23T09:28:55.864630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:58:52.288357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.910095+0000","last_change":"2023-09-23T09:28:55.910095+0000","last_active":"2023-09-23T09:28:55.910095+0000","last_peered":"2023-09-23T09:28:55.910095+0000","last_clean":"2023-09-23T09:28:55.910095+0000","last_became_active":"2023-09-23T09:28:55.907334+0000","last_became_peered":"2023-09-23T09:28:55.907334+0000","last_unstale":"2023-09-23T09:28:55.910095+0000","last_undegraded":"2023-09-23T09:28:55.910095+0000","last_fullsized":"2023-09-23T09:28:55.910095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:48.635606+0000","last_clean_scrub_stamp":"2023-09-23T09:27:48.635606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:11:04.045821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.855746+0000","last_change":"2023-09-23T09:27:50.271063+0000","last_active":"2023-09-23T09:28:54.855746+0000","last_peered":"2023-09-23T09:28:54.855746+0000","last_clean":"2023-09-23T09:28:54.855746+0000","last_became_active":"2023-09-23T09:27:50.270487+0000","last_became_peered":"2023-09-23T09:27:50.270487+0000","last_unstale":"2023-09-23T09:28:54.855746+0000","last_undegraded":"2023-09-23T09:28:54.855746+0000","last_fullsized":"2023-09-23T09:28:54.855746+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:11:05.621855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856550+0000","last_change":"2023-09-23T09:27:50.275900+0000","last_active":"2023-09-23T09:28:54.856550+0000","last_peered":"2023-09-23T09:28:54.856550+0000","last_clean":"2023-09-23T09:28:54.856550+0000","last_became_active":"2023-09-23T09:27:50.275565+0000","last_became_peered":"2023-09-23T09:27:50.275565+0000","last_unstale":"2023-09-23T09:28:54.856550+0000","last_undegraded":"2023-09-23T09:28:54.856550+0000","last_fullsized":"2023-09-23T09:28:54.856550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T19:57:37.004604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854247+0000","last_change":"2023-09-23T09:27:50.275911+0000","last_active":"2023-09-23T09:28:54.854247+0000","last_peered":"2023-09-23T09:28:54.854247+0000","last_clean":"2023-09-23T09:28:54.854247+0000","last_became_active":"2023-09-23T09:27:50.275626+0000","last_became_peered":"2023-09-23T09:27:50.275626+0000","last_unstale":"2023-09-23T09:28:54.854247+0000","last_undegraded":"2023-09-23T09:28:54.854247+0000","last_fullsized":"2023-09-23T09:28:54.854247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:46.790012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.853758+0000","last_change":"2023-09-23T09:27:50.269483+0000","last_active":"2023-09-23T09:28:54.853758+0000","last_peered":"2023-09-23T09:28:54.853758+0000","last_clean":"2023-09-23T09:28:54.853758+0000","last_became_active":"2023-09-23T09:27:50.269199+0000","last_became_peered":"2023-09-23T09:27:50.269199+0000","last_unstale":"2023-09-23T09:28:54.853758+0000","last_undegraded":"2023-09-23T09:28:54.853758+0000","last_fullsized":"2023-09-23T09:28:54.853758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:58:34.304439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-23T09:28:55.859887+0000","last_change":"2023-09-23T09:28:55.859887+0000","last_active":"2023-09-23T09:28:55.859887+0000","last_peered":"2023-09-23T09:28:55.859887+0000","last_clean":"2023-09-23T09:28:55.859887+0000","last_became_active":"2023-09-23T09:28:55.859497+0000","last_became_peered":"2023-09-23T09:28:55.859497+0000","last_unstale":"2023-09-23T09:28:55.859887+0000","last_undegraded":"2023-09-23T09:28:55.859887+0000","last_fullsized":"2023-09-23T09:28:55.859887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:05:52.153365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852551+0000","last_change":"2023-09-23T09:27:50.269934+0000","last_active":"2023-09-23T09:28:54.852551+0000","last_peered":"2023-09-23T09:28:54.852551+0000","last_clean":"2023-09-23T09:28:54.852551+0000","last_became_active":"2023-09-23T09:27:50.269060+0000","last_became_peered":"2023-09-23T09:27:50.269060+0000","last_unstale":"2023-09-23T09:28:54.852551+0000","last_undegraded":"2023-09-23T09:28:54.852551+0000","last_fullsized":"2023-09-23T09:28:54.852551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:47:31.191479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856576+0000","last_change":"2023-09-23T09:27:50.277885+0000","last_active":"2023-09-23T09:28:54.856576+0000","last_peered":"2023-09-23T09:28:54.856576+0000","last_clean":"2023-09-23T09:28:54.856576+0000","last_became_active":"2023-09-23T09:27:50.277576+0000","last_became_peered":"2023-09-23T09:27:50.277576+0000","last_unstale":"2023-09-23T09:28:54.856576+0000","last_undegraded":"2023-09-23T09:28:54.856576+0000","last_fullsized":"2023-09-23T09:28:54.856576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:03:43.338185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854099+0000","last_change":"2023-09-23T09:27:50.278225+0000","last_active":"2023-09-23T09:28:54.854099+0000","last_peered":"2023-09-23T09:28:54.854099+0000","last_clean":"2023-09-23T09:28:54.854099+0000","last_became_active":"2023-09-23T09:27:50.277891+0000","last_became_peered":"2023-09-23T09:27:50.277891+0000","last_unstale":"2023-09-23T09:28:54.854099+0000","last_undegraded":"2023-09-23T09:28:54.854099+0000","last_fullsized":"2023-09-23T09:28:54.854099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T20:27:08.961439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857440+0000","last_change":"2023-09-23T09:27:50.273445+0000","last_active":"2023-09-23T09:28:54.857440+0000","last_peered":"2023-09-23T09:28:54.857440+0000","last_clean":"2023-09-23T09:28:54.857440+0000","last_became_active":"2023-09-23T09:27:50.273124+0000","last_became_peered":"2023-09-23T09:27:50.273124+0000","last_unstale":"2023-09-23T09:28:54.857440+0000","last_undegraded":"2023-09-23T09:28:54.857440+0000","last_fullsized":"2023-09-23T09:28:54.857440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:46.030212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852344+0000","last_change":"2023-09-23T09:27:50.276164+0000","last_active":"2023-09-23T09:28:54.852344+0000","last_peered":"2023-09-23T09:28:54.852344+0000","last_clean":"2023-09-23T09:28:54.852344+0000","last_became_active":"2023-09-23T09:27:50.275860+0000","last_became_peered":"2023-09-23T09:27:50.275860+0000","last_unstale":"2023-09-23T09:28:54.852344+0000","last_undegraded":"2023-09-23T09:28:54.852344+0000","last_fullsized":"2023-09-23T09:28:54.852344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:53:50.521872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.857428+0000","last_change":"2023-09-23T09:27:50.275221+0000","last_active":"2023-09-23T09:28:54.857428+0000","last_peered":"2023-09-23T09:28:54.857428+0000","last_clean":"2023-09-23T09:28:54.857428+0000","last_became_active":"2023-09-23T09:27:50.274920+0000","last_became_peered":"2023-09-23T09:27:50.274920+0000","last_unstale":"2023-09-23T09:28:54.857428+0000","last_undegraded":"2023-09-23T09:28:54.857428+0000","last_fullsized":"2023-09-23T09:28:54.857428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:45:35.023433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.854715+0000","last_change":"2023-09-23T09:27:50.277776+0000","last_active":"2023-09-23T09:28:54.854715+0000","last_peered":"2023-09-23T09:28:54.854715+0000","last_clean":"2023-09-23T09:28:54.854715+0000","last_became_active":"2023-09-23T09:27:50.277291+0000","last_became_peered":"2023-09-23T09:27:50.277291+0000","last_unstale":"2023-09-23T09:28:54.854715+0000","last_undegraded":"2023-09-23T09:28:54.854715+0000","last_fullsized":"2023-09-23T09:28:54.854715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T21:23:46.984450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.852197+0000","last_change":"2023-09-23T09:27:50.271219+0000","last_active":"2023-09-23T09:28:54.852197+0000","last_peered":"2023-09-23T09:28:54.852197+0000","last_clean":"2023-09-23T09:28:54.852197+0000","last_became_active":"2023-09-23T09:27:50.270747+0000","last_became_peered":"2023-09-23T09:27:50.270747+0000","last_unstale":"2023-09-23T09:28:54.852197+0000","last_undegraded":"2023-09-23T09:28:54.852197+0000","last_fullsized":"2023-09-23T09:28:54.852197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:19.237260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-23T09:28:54.856573+0000","last_change":"2023-09-23T09:27:50.263794+0000","last_active":"2023-09-23T09:28:54.856573+0000","last_peered":"2023-09-23T09:28:54.856573+0000","last_clean":"2023-09-23T09:28:54.856573+0000","last_became_active":"2023-09-23T09:27:50.263459+0000","last_became_peered":"2023-09-23T09:27:50.263459+0000","last_unstale":"2023-09-23T09:28:54.856573+0000","last_undegraded":"2023-09-23T09:28:54.856573+0000","last_fullsized":"2023-09-23T09:28:54.856573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T09:27:49.637589+0000","last_clean_scrub_stamp":"2023-09-23T09:27:49.637589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T18:05:37.618959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250071,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61384,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":470,"num_read_kb":300,"num_write":151,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":185424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":444714,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79700000000000004}]},{"osd":1,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.748}]},{"osd":6,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71799999999999997}]},{"osd":8,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88500000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91600000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.81899999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1093585,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42199999999999999},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38100000000000001}]},{"osd":2,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.74099999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.497},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60699999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.46200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72799999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.623},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84899999999999998,"5min":1.006,"15min":1.006},"last":0.65900000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81000000000000005,"5min":0.873,"15min":0.873},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.40100000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.39600000000000002}]},{"osd":5,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.50800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.29,"15min":1.29},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.198,"15min":1.198},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91500000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91800000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57199999999999995}]},{"osd":9,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93200000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.63900000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93700000000000006,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61899999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59499999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":445073,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51400000000000001,"15min":0.51800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.67700000000000005,"15min":0.66100000000000003},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.139,"5min":0.79800000000000004,"15min":0.74099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.616,"5min":11.616,"15min":11.616},"last":0.443}]},{"osd":2,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":0.83999999999999997,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":11.647,"5min":11.647,"15min":11.647},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.9889999999999999,"5min":3.9889999999999999,"15min":3.9889999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.78800000000000003,"15min":0.751},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":11.702,"5min":11.702,"15min":11.702},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.6789999999999998,"5min":3.6789999999999998,"15min":3.6789999999999998},"last":0.376}]},{"osd":5,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":0.71299999999999997}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82099999999999995,"15min":0.83199999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88,"15min":0.88},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":0.999,"15min":0.96699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":11.557,"5min":11.557,"15min":11.557},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.004,"15min":0.96699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":11.675000000000001,"5min":11.675000000000001,"15min":11.675000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.89700000000000002,"15min":0.91600000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.65300000000000002}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.86699999999999999,"15min":0.86299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.67000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444870,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64800000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.55700000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.55900000000000005,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.45300000000000001},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63300000000000001,"5min":0.745,"15min":0.745},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.375,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60399999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80200000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.316,"15min":0.316},"max":{"1min":0.89100000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.76600000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.58099999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.999}]},{"osd":7,"last update":"Sat Sep 23 09:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.751,"15min":0.73099999999999998},"min":{"1min":0.60999999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89800000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.0800000000000001}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.749,"15min":0.73299999999999998},"min":{"1min":0.624,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":1.0169999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.042}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1036922,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.30299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.36799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67700000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.55200000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.83099999999999996}]},{"osd":3,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.497,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.879,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64400000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69499999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75800000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.53800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78800000000000003}]},{"osd":6,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.52800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83899999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67800000000000005,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Sep 23 09:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.53700000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99299999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.89300000000000002}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.871,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.874},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94699999999999995,"5min":1.022,"15min":1.022},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.90800000000000003}]},{"osd":11,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93100000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":444345,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.48199999999999998,"15min":0.432},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.442,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.432}]},{"osd":1,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.65900000000000003,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.626}]},{"osd":2,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.48099999999999998},"min":{"1min":0.33000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.66700000000000004,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.312,"15min":0.312},"max":{"1min":11.151999999999999,"5min":11.151999999999999,"15min":11.151999999999999},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.749}]},{"osd":5,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.749,"15min":0.69799999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":6.952,"5min":6.952,"15min":6.952},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2230000000000001,"5min":1.294,"15min":1.294},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70799999999999996}]},{"osd":8,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":6.9850000000000003,"5min":6.9850000000000003,"15min":6.9850000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0600000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.72199999999999998}]},{"osd":9,"last update":"Sat Sep 23 09:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1,"5min":0.85599999999999998,"15min":0.83199999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":7.016,"5min":7.016,"15min":7.016},"last":0.78000000000000003}]},{"osd":10,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.878,"15min":0.85399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81399999999999995,"15min":0.82999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95899999999999996,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":501126,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.505,"15min":0.46500000000000002},"min":{"1min":0.34899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.753,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.58099999999999996,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.54200000000000004,"15min":0.497},"min":{"1min":0.34699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77000000000000002}]},{"osd":3,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47199999999999998,"15min":0.44},"min":{"1min":0.33400000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.44900000000000001},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.85499999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.51600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.97699999999999998}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.64900000000000002,"15min":0.59699999999999998},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.7679999999999998,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":1.032},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":1.0109999999999999}]},{"osd":7,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55600000000000005,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.1220000000000001}]},{"osd":8,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.69499999999999995,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.73899999999999999,"15min":0.70599999999999996},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":1.107}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.94799999999999995}]},{"osd":10,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.75600000000000001,"15min":0.74099999999999999},"min":{"1min":0.64400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.78800000000000003,"15min":0.77300000000000002},"min":{"1min":0.69999999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.0600000000000001}]},{"osd":11,"last update":"Sat Sep 23 09:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.048},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.083}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":445610,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 09:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.435},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.32800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.7149999999999999,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58399999999999996,"15min":0.52800000000000002},"min":{"1min":0.47899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2230000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.60699999999999998,"15min":0.52900000000000003},"min":{"1min":0.42099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95499999999999996,"5min":2.653,"15min":2.653},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.38}]},{"osd":4,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78800000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Sep 23 09:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.51300000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.97299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.504}]},{"osd":7,"last update":"Sat Sep 23 09:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.718,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.718},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":8,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0649999999999999,"5min":1.101,"15min":1.101},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Sep 23 09:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.81599999999999995,"15min":0.83699999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60099999999999998}]},{"osd":10,"last update":"Sat Sep 23 09:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.88800000000000001,"15min":0.91400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93600000000000005,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.48999999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.877,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.58399999999999996}]},{"osd":11,"last update":"Sat Sep 23 09:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.72699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503502,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.78200000000000003,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.68999999999999995,"5min":1.655,"15min":1.655},"last":0.39800000000000002}]},{"osd":1,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.40100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85999999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Sep 23 09:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.45100000000000001},"min":{"1min":0.35999999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.50700000000000001,"15min":0.47199999999999998},"min":{"1min":0.34499999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.503}]},{"osd":6,"last update":"Sat Sep 23 09:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85699999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Sep 23 09:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.014,"15min":1.014},"last":0.47299999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95899999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.623}]},{"osd":9,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93000000000000005,"5min":0.995,"15min":0.995},"last":0.755},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67900000000000005}]},{"osd":10,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.61799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.61799999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73399999999999999}]},{"osd":11,"last update":"Sat Sep 23 09:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70899999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444208,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.39100000000000001,"15min":0.35299999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.372},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.28199999999999997,"5min":0.191,"15min":0.191},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.307}]},{"osd":1,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55000000000000004,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96599999999999997,"5min":1.036,"15min":1.036},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79500000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64300000000000002}]},{"osd":4,"last update":"Sat Sep 23 09:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83799999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86399999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Sep 23 09:28:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.55300000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":1,"15min":1},"last":0.48499999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.53500000000000003},"min":{"1min":0.438,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4990000000000001,"5min":2.4990000000000001,"15min":2.4990000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.437}]},{"osd":7,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0620000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58899999999999997}]},{"osd":8,"last update":"Sat Sep 23 09:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Sep 23 09:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.46899999999999997}]},{"osd":10,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.232,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.999,"5min":1.115,"15min":1.115},"last":0.629}]},{"osd":11,"last update":"Sat Sep 23 09:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66700000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":445166,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.998,"15min":0.998},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.36099999999999999},"min":{"1min":0.28699999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.41199999999999998,"15min":0.38500000000000001},"min":{"1min":0.32600000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.60699999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.56200000000000006},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60399999999999998,"15min":0.56599999999999995},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76600000000000001}]},{"osd":3,"last update":"Sat Sep 23 09:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.89700000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63,"15min":0.59099999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.753},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63500000000000001,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80500000000000005}]},{"osd":6,"last update":"Sat Sep 23 09:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86699999999999999,"5min":1.071,"15min":1.071},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59099999999999997,"15min":0.56000000000000005},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92500000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":8,"last update":"Sat Sep 23 09:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69999999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.626}]},{"osd":9,"last update":"Sat Sep 23 09:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":1.089,"15min":1.089},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.86499999999999999}]},{"osd":10,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96499999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70699999999999996}]},{"osd":11,"last update":"Sat Sep 23 09:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73999999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757301,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1036489,"data_compressed":42002,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.55300000000000005,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.52000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.51000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.56399999999999995,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.163,"5min":1.538,"15min":1.538},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Sep 23 09:28:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.72199999999999998,"15min":0.67700000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.84799999999999998}]},{"osd":7,"last update":"Sat Sep 23 09:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.004,"5min":1.238,"15min":1.238},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67300000000000004,"15min":0.65400000000000003},"min":{"1min":0.61399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":1.153,"15min":1.153},"last":0.70299999999999996}]},{"osd":8,"last update":"Sat Sep 23 09:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88100000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86199999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005}]},{"osd":9,"last update":"Sat Sep 23 09:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1020000000000003,"5min":4.1020000000000003,"15min":4.1020000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.83099999999999996,"15min":0.80700000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Sep 23 09:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.65700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89600000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.72499999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.88800000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34553,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T09:29:14.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:29:14.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 -- 172.21.15.45:0/4191752633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec4100f00 msgr2=0x7efec4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 --2- 172.21.15.45:0/4191752633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec4100f00 0x7efec4101360 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efeb00099c0 tx=0x7efeb0009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 -- 172.21.15.45:0/4191752633 shutdown_connections 2023-09-23T09:29:14.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 --2- 172.21.15.45:0/4191752633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec41018a0 0x7efec4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 --2- 172.21.15.45:0/4191752633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec40ffd10 0x7efec4100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 --2- 172.21.15.45:0/4191752633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec4100f00 0x7efec4101360 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 -- 172.21.15.45:0/4191752633 >> 172.21.15.45:0/4191752633 conn(0x7efec40fb240 msgr2=0x7efec40fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:14.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 -- 172.21.15.45:0/4191752633 shutdown_connections 2023-09-23T09:29:14.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.451+0000 7efecad35700 1 -- 172.21.15.45:0/4191752633 wait complete. 2023-09-23T09:29:14.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 Processor -- start 2023-09-23T09:29:14.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 -- start start 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec40ffd10 0x7efec41958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4100f00 0x7efec4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec41018a0 0x7efec419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec419d460 con 0x7efec41018a0 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec419d5a0 con 0x7efec40ffd10 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.452+0000 7efecad35700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec419d730 con 0x7efec4100f00 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec41018a0 0x7efec419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec41018a0 0x7efec419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60320/0 (socket says 172.21.15.45:60320) 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 -- 172.21.15.45:0/1139469038 learned_addr learned my addr 172.21.15.45:0/1139469038 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:14.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec3fff700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4100f00 0x7efec4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec8ad1700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec40ffd10 0x7efec41958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 -- 172.21.15.45:0/1139469038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4100f00 msgr2=0x7efec4195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4100f00 0x7efec4195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 -- 172.21.15.45:0/1139469038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec40ffd10 msgr2=0x7efec41958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec40ffd10 0x7efec41958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 -- 172.21.15.45:0/1139469038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00095f0 con 0x7efec41018a0 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec92d2700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec41018a0 0x7efec419ce90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efebc007ee0 tx=0x7efebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec1ffb700 1 -- 172.21.15.45:0/1139469038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc017070 con 0x7efec41018a0 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec1ffb700 1 -- 172.21.15.45:0/1139469038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efebc00f040 con 0x7efec41018a0 2023-09-23T09:29:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec419da10 con 0x7efec41018a0 2023-09-23T09:29:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efec1ffb700 1 -- 172.21.15.45:0/1139469038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc005580 con 0x7efec41018a0 2023-09-23T09:29:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.453+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec419ded0 con 0x7efec41018a0 2023-09-23T09:29:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.454+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec410b520 con 0x7efec41018a0 2023-09-23T09:29:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.455+0000 7efec1ffb700 1 -- 172.21.15.45:0/1139469038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7efebc00db20 con 0x7efec41018a0 2023-09-23T09:29:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.456+0000 7efec1ffb700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efeb40a1670 0x7efeb40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.456+0000 7efec8ad1700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efeb40a1670 0x7efeb40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.456+0000 7efec1ffb700 1 -- 172.21.15.45:0/1139469038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efebc0cc820 con 0x7efec41018a0 2023-09-23T09:29:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.457+0000 7efec8ad1700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efeb40a1670 0x7efeb40a3b30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efec4100d60 tx=0x7efeb800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:14.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.461+0000 7efec1ffb700 1 -- 172.21.15.45:0/1139469038 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7efebc095cb0 con 0x7efec41018a0 2023-09-23T09:29:14.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:14 smithi045 ceph-mon[98627]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:14.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:14 smithi045 ceph-mon[98627]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:14.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.664+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efec410b710 con 0x7efec41018a0 2023-09-23T09:29:14.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.666+0000 7efec1ffb700 1 -- 172.21.15.45:0/1139469038 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22481 (secure 0 0 0) 0x7efebc0992d0 con 0x7efec41018a0 2023-09-23T09:29:14.668 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:14.668 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:28:55.849323+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:29:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efeb40a1670 msgr2=0x7efeb40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efeb40a1670 0x7efeb40a3b30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efec4100d60 tx=0x7efeb800a3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec41018a0 msgr2=0x7efec419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec41018a0 0x7efec419ce90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efebc007ee0 tx=0x7efebc00d3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 shutdown_connections 2023-09-23T09:29:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec41018a0 0x7efec419ce90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec40ffd10 0x7efec41958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7efeb40a1670 0x7efeb40a3b30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 --2- 172.21.15.45:0/1139469038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec4100f00 0x7efec4195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 >> 172.21.15.45:0/1139469038 conn(0x7efec40fb240 msgr2=0x7efec40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 shutdown_connections 2023-09-23T09:29:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.668+0000 7efecad35700 1 -- 172.21.15.45:0/1139469038 wait complete. 2023-09-23T09:29:14.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:29:14.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:29:14.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:14 smithi098 ceph-mon[102386]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:14.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:14 smithi098 ceph-mon[102386]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 -- 172.21.15.45:0/2926352756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 msgr2=0x7f5380103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 --2- 172.21.15.45:0/2926352756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380103ce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f537c009990 tx=0x7f537c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 -- 172.21.15.45:0/2926352756 shutdown_connections 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 --2- 172.21.15.45:0/2926352756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5380106b50 0x7f5380108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 --2- 172.21.15.45:0/2926352756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380103ce0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 --2- 172.21.15.45:0/2926352756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5380104220 0x7f5380106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 -- 172.21.15.45:0/2926352756 >> 172.21.15.45:0/2926352756 conn(0x7f53800fb280 msgr2=0x7f53800fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 -- 172.21.15.45:0/2926352756 shutdown_connections 2023-09-23T09:29:14.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.810+0000 7f5387b51700 1 -- 172.21.15.45:0/2926352756 wait complete. 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 Processor -- start 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 -- start start 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5380104220 0x7f53800ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5380106b50 0x7f53800ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5380100580 con 0x7f53801018f0 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53801006f0 con 0x7f5380104220 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f5387b51700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53801a0ac0 con 0x7f5380106b50 2023-09-23T09:29:14.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60330/0 (socket says 172.21.15.45:60330) 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 -- 172.21.15.45:0/3692863617 learned_addr learned my addr 172.21.15.45:0/3692863617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53850ec700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5380104220 0x7f53800ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53860ee700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5380106b50 0x7f53800ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 -- 172.21.15.45:0/3692863617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5380106b50 msgr2=0x7f53800ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5380106b50 0x7f53800ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 -- 172.21.15.45:0/3692863617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5380104220 msgr2=0x7f53800ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5380104220 0x7f53800ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.811+0000 7f53858ed700 1 -- 172.21.15.45:0/3692863617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0095f0 con 0x7f53801018f0 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.812+0000 7f53858ed700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380101260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f537c000c00 tx=0x7f537c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:14.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.812+0000 7f5372ffd700 1 -- 172.21.15.45:0/3692863617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c01d070 con 0x7f53801018f0 2023-09-23T09:29:14.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.812+0000 7f5372ffd700 1 -- 172.21.15.45:0/3692863617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537c00fcb0 con 0x7f53801018f0 2023-09-23T09:29:14.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.812+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53801a0d40 con 0x7f53801018f0 2023-09-23T09:29:14.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.812+0000 7f5372ffd700 1 -- 172.21.15.45:0/3692863617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c017650 con 0x7f53801018f0 2023-09-23T09:29:14.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.812+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53801a1190 con 0x7f53801018f0 2023-09-23T09:29:14.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.813+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53800fce00 con 0x7f53801018f0 2023-09-23T09:29:14.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.814+0000 7f5372ffd700 1 -- 172.21.15.45:0/3692863617 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f537c021410 con 0x7f53801018f0 2023-09-23T09:29:14.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.814+0000 7f5372ffd700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f536c0a1730 0x7f536c0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:14.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.815+0000 7f53850ec700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f536c0a1730 0x7f536c0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:14.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.815+0000 7f5372ffd700 1 -- 172.21.15.45:0/3692863617 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f537c0ce5b0 con 0x7f53801018f0 2023-09-23T09:29:14.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.815+0000 7f53850ec700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f536c0a1730 0x7f536c0a3bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5374009c60 tx=0x7f5374009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:14.819+0000 7f5372ffd700 1 -- 172.21.15.45:0/3692863617 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f537c097a90 con 0x7f53801018f0 2023-09-23T09:29:14.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:14 smithi123 ceph-mon[103577]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:14.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:14 smithi123 ceph-mon[103577]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.019+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f538002cf30 con 0x7f53801018f0 2023-09-23T09:29:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.019+0000 7f5372ffd700 1 -- 172.21.15.45:0/3692863617 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f537c09b0b0 con 0x7f53801018f0 2023-09-23T09:29:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:29:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f536c0a1730 msgr2=0x7f536c0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f536c0a1730 0x7f536c0a3bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5374009c60 tx=0x7f5374009260 comp rx=0 tx=0).stop 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 msgr2=0x7f5380101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380101260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f537c000c00 tx=0x7f537c00f460 comp rx=0 tx=0).stop 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 shutdown_connections 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f536c0a1730 0x7f536c0a3bf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5380106b50 0x7f53800ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53801018f0 0x7f5380101260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 --2- 172.21.15.45:0/3692863617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5380104220 0x7f53800ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.021+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 >> 172.21.15.45:0/3692863617 conn(0x7f53800fb280 msgr2=0x7f5380102580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.022+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 shutdown_connections 2023-09-23T09:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.022+0000 7f5387b51700 1 -- 172.21.15.45:0/3692863617 wait complete. 2023-09-23T09:29:15.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:29:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 -- 172.21.15.45:0/3345520699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c0ffd40 msgr2=0x7f779c100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/3345520699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c0ffd40 0x7f779c100160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7790009990 tx=0x7f7790009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 -- 172.21.15.45:0/3345520699 shutdown_connections 2023-09-23T09:29:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/3345520699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c1018d0 0x7f779c108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/3345520699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c0ffd40 0x7f779c100160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/3345520699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f779c100f30 0x7f779c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 -- 172.21.15.45:0/3345520699 >> 172.21.15.45:0/3345520699 conn(0x7f779c0fb2b0 msgr2=0x7f779c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:15.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 -- 172.21.15.45:0/3345520699 shutdown_connections 2023-09-23T09:29:15.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.163+0000 7f77a0bbf700 1 -- 172.21.15.45:0/3345520699 wait complete. 2023-09-23T09:29:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 Processor -- start 2023-09-23T09:29:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 -- start start 2023-09-23T09:29:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c0ffd40 0x7f779c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c100f30 0x7f779c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f779c1018d0 0x7f779c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19e670 con 0x7f779c0ffd40 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19e7e0 con 0x7f779c1018d0 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f77a0bbf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19e9a0 con 0x7f779c100f30 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f779a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c0ffd40 0x7f779c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f779a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c0ffd40 0x7f779c069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60344/0 (socket says 172.21.15.45:60344) 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f779a59c700 1 -- 172.21.15.45:0/2934298538 learned_addr learned my addr 172.21.15.45:0/2934298538 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f779ad9d700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f779c1018d0 0x7f779c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.164+0000 7f7799d9b700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c100f30 0x7f779c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f779a59c700 1 -- 172.21.15.45:0/2934298538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c100f30 msgr2=0x7f779c19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f779a59c700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c100f30 0x7f779c19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f779a59c700 1 -- 172.21.15.45:0/2934298538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f779c1018d0 msgr2=0x7f779c19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f779a59c700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f779c1018d0 0x7f779c19dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f779a59c700 1 -- 172.21.15.45:0/2934298538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77900095f0 con 0x7f779c0ffd40 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f7799d9b700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c100f30 0x7f779c19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f779a59c700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c0ffd40 0x7f779c069ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f779000b680 tx=0x7f7790004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f778b7fe700 1 -- 172.21.15.45:0/2934298538 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779001d070 con 0x7f779c0ffd40 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c19ec20 con 0x7f779c0ffd40 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f778b7fe700 1 -- 172.21.15.45:0/2934298538 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77900054f0 con 0x7f779c0ffd40 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f778b7fe700 1 -- 172.21.15.45:0/2934298538 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790004ca0 con 0x7f779c0ffd40 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.165+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c19f040 con 0x7f779c0ffd40 2023-09-23T09:29:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.166+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c106c70 con 0x7f779c0ffd40 2023-09-23T09:29:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.167+0000 7f778b7fe700 1 -- 172.21.15.45:0/2934298538 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f779000fc70 con 0x7f779c0ffd40 2023-09-23T09:29:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.168+0000 7f778b7fe700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f77840a1630 0x7f77840a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.168+0000 7f7799d9b700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f77840a1630 0x7f77840a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.168+0000 7f778b7fe700 1 -- 172.21.15.45:0/2934298538 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f77900ce870 con 0x7f779c0ffd40 2023-09-23T09:29:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.168+0000 7f7799d9b700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f77840a1630 0x7f77840a3af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f779c069850 tx=0x7f778c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.171+0000 7f778b7fe700 1 -- 172.21.15.45:0/2934298538 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7790097d90 con 0x7f779c0ffd40 2023-09-23T09:29:15.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.373+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f779c10b570 con 0x7f779c0ffd40 2023-09-23T09:29:15.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.374+0000 7f778b7fe700 1 -- 172.21.15.45:0/2934298538 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f779009b3b0 con 0x7f779c0ffd40 2023-09-23T09:29:15.376 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:29:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f77840a1630 msgr2=0x7f77840a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f77840a1630 0x7f77840a3af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f779c069850 tx=0x7f778c009260 comp rx=0 tx=0).stop 2023-09-23T09:29:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c0ffd40 msgr2=0x7f779c069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c0ffd40 0x7f779c069ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f779000b680 tx=0x7f7790004730 comp rx=0 tx=0).stop 2023-09-23T09:29:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 shutdown_connections 2023-09-23T09:29:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f779c1018d0 0x7f779c19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f779c0ffd40 0x7f779c069ae0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f77840a1630 0x7f77840a3af0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 --2- 172.21.15.45:0/2934298538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779c100f30 0x7f779c19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 >> 172.21.15.45:0/2934298538 conn(0x7f779c0fb2b0 msgr2=0x7f779c0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 shutdown_connections 2023-09-23T09:29:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.376+0000 7f77a0bbf700 1 -- 172.21.15.45:0/2934298538 wait complete. 2023-09-23T09:29:15.390 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 -- 172.21.15.45:0/4269419433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6141018f0 msgr2=0x7fc614103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4269419433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6141018f0 0x7fc614103ce0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc608009990 tx=0x7fc608009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 -- 172.21.15.45:0/4269419433 shutdown_connections 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4269419433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc614106b50 0x7fc614108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4269419433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6141018f0 0x7fc614103ce0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4269419433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc614104220 0x7fc614106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 -- 172.21.15.45:0/4269419433 >> 172.21.15.45:0/4269419433 conn(0x7fc6140fb280 msgr2=0x7fc6140fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 -- 172.21.15.45:0/4269419433 shutdown_connections 2023-09-23T09:29:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.517+0000 7fc618f9c700 1 -- 172.21.15.45:0/4269419433 wait complete. 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 Processor -- start 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 -- start start 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6141018f0 0x7fc614101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc614104220 0x7fc6140ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc614106b50 0x7fc6140ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc614100580 con 0x7fc614104220 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6141006f0 con 0x7fc6141018f0 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc618f9c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6141a0ab0 con 0x7fc614106b50 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc614104220 0x7fc6140ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc614104220 0x7fc6140ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60360/0 (socket says 172.21.15.45:60360) 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 -- 172.21.15.45:0/4130945170 learned_addr learned my addr 172.21.15.45:0/4130945170 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc61259c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6141018f0 0x7fc614101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc612d9d700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc614106b50 0x7fc6140ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 -- 172.21.15.45:0/4130945170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc614106b50 msgr2=0x7fc6140ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc614106b50 0x7fc6140ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 -- 172.21.15.45:0/4130945170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6141018f0 msgr2=0x7fc614101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6141018f0 0x7fc614101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.518+0000 7fc611d9b700 1 -- 172.21.15.45:0/4130945170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080095f0 con 0x7fc614104220 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.519+0000 7fc611d9b700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc614104220 0x7fc6140ff8a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c9b0 tx=0x7fc60400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.519+0000 7fc6037fe700 1 -- 172.21.15.45:0/4130945170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604007b00 con 0x7fc614104220 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.519+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6141a0d90 con 0x7fc614104220 2023-09-23T09:29:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.519+0000 7fc6037fe700 1 -- 172.21.15.45:0/4130945170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc604007c60 con 0x7fc614104220 2023-09-23T09:29:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.519+0000 7fc6037fe700 1 -- 172.21.15.45:0/4130945170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604018660 con 0x7fc614104220 2023-09-23T09:29:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.519+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6141a1260 con 0x7fc614104220 2023-09-23T09:29:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.520+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6140fce00 con 0x7fc614104220 2023-09-23T09:29:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.522+0000 7fc6037fe700 1 -- 172.21.15.45:0/4130945170 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fc60400f450 con 0x7fc614104220 2023-09-23T09:29:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.522+0000 7fc6037fe700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5fc0a1630 0x7fc5fc0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.522+0000 7fc61259c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5fc0a1630 0x7fc5fc0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.522+0000 7fc6037fe700 1 -- 172.21.15.45:0/4130945170 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc6040cd320 con 0x7fc614104220 2023-09-23T09:29:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.523+0000 7fc61259c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5fc0a1630 0x7fc5fc0a3af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b5c0 tx=0x7fc608005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:15.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.526+0000 7fc6037fe700 1 -- 172.21.15.45:0/4130945170 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc604096840 con 0x7fc614104220 2023-09-23T09:29:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.725+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc61402cfe0 con 0x7fc614104220 2023-09-23T09:29:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.725+0000 7fc6037fe700 1 -- 172.21.15.45:0/4130945170 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc604099e60 con 0x7fc614104220 2023-09-23T09:29:15.727 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:29:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5fc0a1630 msgr2=0x7fc5fc0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5fc0a1630 0x7fc5fc0a3af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b5c0 tx=0x7fc608005fd0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc614104220 msgr2=0x7fc6140ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc614104220 0x7fc6140ff8a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c9b0 tx=0x7fc60400ccc0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 shutdown_connections 2023-09-23T09:29:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fc5fc0a1630 0x7fc5fc0a3af0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc614106b50 0x7fc6140ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6141018f0 0x7fc614101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 --2- 172.21.15.45:0/4130945170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc614104220 0x7fc6140ff8a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 >> 172.21.15.45:0/4130945170 conn(0x7fc6140fb280 msgr2=0x7fc614102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:15.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 shutdown_connections 2023-09-23T09:29:15.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.727+0000 7fc618f9c700 1 -- 172.21.15.45:0/4130945170 wait complete. 2023-09-23T09:29:15.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:15 smithi045 ceph-mon[98627]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:15.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:15 smithi045 ceph-mon[98627]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-23T09:29:15.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1139469038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:15.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3692863617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:15.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2934298538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:15.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:29:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:15 smithi098 ceph-mon[102386]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:15 smithi098 ceph-mon[102386]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-23T09:29:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1139469038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3692863617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2934298538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:15 smithi123 ceph-mon[103577]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T09:29:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:15 smithi123 ceph-mon[103577]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-23T09:29:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1139469038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3692863617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:29:15.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2934298538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:29:15.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.865+0000 7f6a25705700 1 -- 172.21.15.45:0/1136619433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a201018d0 msgr2=0x7f6a20103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 --2- 172.21.15.45:0/1136619433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a201018d0 0x7f6a20103cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009990 tx=0x7f6a10009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 -- 172.21.15.45:0/1136619433 shutdown_connections 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 --2- 172.21.15.45:0/1136619433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a20106b30 0x7f6a20108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 --2- 172.21.15.45:0/1136619433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a201018d0 0x7f6a20103cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 --2- 172.21.15.45:0/1136619433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a20104200 0x7f6a201065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 -- 172.21.15.45:0/1136619433 >> 172.21.15.45:0/1136619433 conn(0x7f6a200fb2a0 msgr2=0x7f6a200fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 -- 172.21.15.45:0/1136619433 shutdown_connections 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 -- 172.21.15.45:0/1136619433 wait complete. 2023-09-23T09:29:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 Processor -- start 2023-09-23T09:29:15.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.866+0000 7f6a25705700 1 -- start start 2023-09-23T09:29:15.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a201018d0 0x7f6a20196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a20104200 0x7f6a20197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20106b30 0x7f6a2019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2019e640 con 0x7f6a20106b30 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2019e780 con 0x7f6a20104200 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2019e910 con 0x7f6a201018d0 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20106b30 0x7f6a2019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a20104200 0x7f6a20197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a201018d0 0x7f6a20196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20106b30 0x7f6a2019e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60376/0 (socket says 172.21.15.45:60376) 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a20104200 0x7f6a20197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:53492/0 (socket says 172.21.15.45:53492) 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 -- 172.21.15.45:0/706624690 learned_addr learned my addr 172.21.15.45:0/706624690 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a201018d0 0x7f6a20196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34246/0 (socket says 172.21.15.45:34246) 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 -- 172.21.15.45:0/706624690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a201018d0 msgr2=0x7f6a20196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a201018d0 0x7f6a20196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 -- 172.21.15.45:0/706624690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a20104200 msgr2=0x7f6a20197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a20104200 0x7f6a20197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 -- 172.21.15.45:0/706624690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a100095f0 con 0x7f6a20106b30 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a1f7fe700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20106b30 0x7f6a2019e070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800eb80 tx=0x7f6a1800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a07fff700 1 -- 172.21.15.45:0/706624690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18018070 con 0x7f6a20106b30 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a07fff700 1 -- 172.21.15.45:0/706624690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a1800f040 con 0x7f6a20106b30 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2019eb90 con 0x7f6a20106b30 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a07fff700 1 -- 172.21.15.45:0/706624690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800cde0 con 0x7f6a20106b30 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.867+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2019f040 con 0x7f6a20106b30 2023-09-23T09:29:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.868+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a200fce20 con 0x7f6a20106b30 2023-09-23T09:29:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.870+0000 7f6a07fff700 1 -- 172.21.15.45:0/706624690 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f6a18003a50 con 0x7f6a20106b30 2023-09-23T09:29:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.870+0000 7f6a07fff700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6a080a1670 0x7f6a080a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.871+0000 7f6a07fff700 1 -- 172.21.15.45:0/706624690 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a180cd610 con 0x7f6a20106b30 2023-09-23T09:29:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.871+0000 7f6a1effd700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6a080a1670 0x7f6a080a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.871+0000 7f6a1effd700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6a080a1670 0x7f6a080a3b30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b2e0 tx=0x7f6a10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:15.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:15.874+0000 7f6a07fff700 1 -- 172.21.15.45:0/706624690 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6a18096b30 con 0x7f6a20106b30 2023-09-23T09:29:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.118+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a2010c8d0 con 0x7f6a20106b30 2023-09-23T09:29:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.119+0000 7f6a07fff700 1 -- 172.21.15.45:0/706624690 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11306 (secure 0 0 0) 0x7f6a18025380 con 0x7f6a20106b30 2023-09-23T09:29:16.120 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:16.120 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:27:55.252601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:29:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.120+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6a080a1670 msgr2=0x7f6a080a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.120+0000 7f6a25705700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6a080a1670 0x7f6a080a3b30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b2e0 tx=0x7f6a10005fb0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.120+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20106b30 msgr2=0x7f6a2019e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20106b30 0x7f6a2019e070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800eb80 tx=0x7f6a1800ee90 comp rx=0 tx=0).stop 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 shutdown_connections 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20106b30 0x7f6a2019e070 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f6a080a1670 0x7f6a080a3b30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a201018d0 0x7f6a20196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 --2- 172.21.15.45:0/706624690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a20104200 0x7f6a20197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 >> 172.21.15.45:0/706624690 conn(0x7f6a200fb2a0 msgr2=0x7f6a20102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 shutdown_connections 2023-09-23T09:29:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.121+0000 7f6a25705700 1 -- 172.21.15.45:0/706624690 wait complete. 2023-09-23T09:29:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-09-23T09:29:16.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:29:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 -- 172.21.15.45:0/4278457282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 msgr2=0x7f8f68108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 --2- 172.21.15.45:0/4278457282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f68108140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b0d0 tx=0x7f8f6000b3e0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 -- 172.21.15.45:0/4278457282 shutdown_connections 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 --2- 172.21.15.45:0/4278457282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f68108140 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 --2- 172.21.15.45:0/4278457282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f680ffd00 0x7f8f68100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 --2- 172.21.15.45:0/4278457282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68100ef0 0x7f8f68101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 -- 172.21.15.45:0/4278457282 >> 172.21.15.45:0/4278457282 conn(0x7f8f680fb250 msgr2=0x7f8f680fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 -- 172.21.15.45:0/4278457282 shutdown_connections 2023-09-23T09:29:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 -- 172.21.15.45:0/4278457282 wait complete. 2023-09-23T09:29:16.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 Processor -- start 2023-09-23T09:29:16.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.260+0000 7f8f70108700 1 -- start start 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f680ffd00 0x7f8f68196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68100ef0 0x7f8f68197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f6819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819e890 con 0x7f8f68101890 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819ea00 con 0x7f8f680ffd00 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819ebc0 con 0x7f8f68100ef0 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f6819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f6819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60390/0 (socket says 172.21.15.45:60390) 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 -- 172.21.15.45:0/2090813668 learned_addr learned my addr 172.21.15.45:0/2090813668 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6dea4700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f680ffd00 0x7f8f68196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6d6a3700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68100ef0 0x7f8f68197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 -- 172.21.15.45:0/2090813668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68100ef0 msgr2=0x7f8f68197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68100ef0 0x7f8f68197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 -- 172.21.15.45:0/2090813668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f680ffd00 msgr2=0x7f8f68196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f680ffd00 0x7f8f68196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 -- 172.21.15.45:0/2090813668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60009d00 con 0x7f8f68101890 2023-09-23T09:29:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f6e6a5700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f6819e2c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8f60007a00 tx=0x7f8f60009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f5effd700 1 -- 172.21.15.45:0/2090813668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000e040 con 0x7f8f68101890 2023-09-23T09:29:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f5effd700 1 -- 172.21.15.45:0/2090813668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f60003e10 con 0x7f8f68101890 2023-09-23T09:29:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6819ee40 con 0x7f8f68101890 2023-09-23T09:29:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f5effd700 1 -- 172.21.15.45:0/2090813668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60005750 con 0x7f8f68101890 2023-09-23T09:29:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.261+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6819f230 con 0x7f8f68101890 2023-09-23T09:29:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.262+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6810b630 con 0x7f8f68101890 2023-09-23T09:29:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.264+0000 7f8f5effd700 1 -- 172.21.15.45:0/2090813668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f8f60012cb0 con 0x7f8f68101890 2023-09-23T09:29:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.264+0000 7f8f5effd700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f540a16f0 0x7f8f540a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.264+0000 7f8f5effd700 1 -- 172.21.15.45:0/2090813668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8f600ced30 con 0x7f8f68101890 2023-09-23T09:29:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.264+0000 7f8f6dea4700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f540a16f0 0x7f8f540a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.265+0000 7f8f6dea4700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f540a16f0 0x7f8f540a3bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006fd0 tx=0x7f8f58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.268+0000 7f8f5effd700 1 -- 172.21.15.45:0/2090813668 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8f60098250 con 0x7f8f68101890 2023-09-23T09:29:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.469+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f6819f420 con 0x7f8f68101890 2023-09-23T09:29:16.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.471+0000 7f8f5effd700 1 -- 172.21.15.45:0/2090813668 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f8f6009b870 con 0x7f8f68101890 2023-09-23T09:29:16.473 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:16.473 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:28:55.849323+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:29:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f540a16f0 msgr2=0x7f8f540a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f540a16f0 0x7f8f540a3bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006fd0 tx=0x7f8f58008040 comp rx=0 tx=0).stop 2023-09-23T09:29:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 msgr2=0x7f8f6819e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f6819e2c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8f60007a00 tx=0x7f8f60009570 comp rx=0 tx=0).stop 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 shutdown_connections 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f68101890 0x7f8f6819e2c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f680ffd00 0x7f8f68196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f540a16f0 0x7f8f540a3bb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 --2- 172.21.15.45:0/2090813668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68100ef0 0x7f8f68197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.473+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 >> 172.21.15.45:0/2090813668 conn(0x7f8f680fb250 msgr2=0x7f8f68067fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.474+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 shutdown_connections 2023-09-23T09:29:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.474+0000 7f8f70108700 1 -- 172.21.15.45:0/2090813668 wait complete. 2023-09-23T09:29:16.488 DEBUG:tasks.fwd_scrub:Yielding 2023-09-23T09:29:16.488 INFO:teuthology.run_tasks:Running task exec... 2023-09-23T09:29:16.499 INFO:teuthology.task.exec:Executing custom commands... 2023-09-23T09:29:16.499 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:29:16.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-09-23T09:29:16.500 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-23T09:29:16.500 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T09:29:16.500 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T09:29:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 -- 172.21.15.45:0/1292957832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148106bc0 msgr2=0x7f9148108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 --2- 172.21.15.45:0/1292957832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148106bc0 0x7f9148108fb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f914000b0a0 tx=0x7f914000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 -- 172.21.15.45:0/1292957832 shutdown_connections 2023-09-23T09:29:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 --2- 172.21.15.45:0/1292957832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148106bc0 0x7f9148108fb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 --2- 172.21.15.45:0/1292957832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9148101960 0x7f9148103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 --2- 172.21.15.45:0/1292957832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9148104290 0x7f9148106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 -- 172.21.15.45:0/1292957832 >> 172.21.15.45:0/1292957832 conn(0x7f91480fb270 msgr2=0x7f91480fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 -- 172.21.15.45:0/1292957832 shutdown_connections 2023-09-23T09:29:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.626+0000 7f914d619700 1 -- 172.21.15.45:0/1292957832 wait complete. 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.627+0000 7f914d619700 1 Processor -- start 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.627+0000 7f914d619700 1 -- start start 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.627+0000 7f914d619700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9148101960 0x7f91480ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.627+0000 7f914d619700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148104290 0x7f91480ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f914d619700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9148106bc0 0x7f9148100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f914d619700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481a0c10 con 0x7f9148104290 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f914d619700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481a0d50 con 0x7f9148101960 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f914d619700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481a0e90 con 0x7f9148106bc0 2023-09-23T09:29:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148104290 0x7f91480ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f9146ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9148101960 0x7f91480ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148104290 0x7f91480ffe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60408/0 (socket says 172.21.15.45:60408) 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 -- 172.21.15.45:0/2533028113 learned_addr learned my addr 172.21.15.45:0/2533028113 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91477fe700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9148106bc0 0x7f9148100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 -- 172.21.15.45:0/2533028113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9148106bc0 msgr2=0x7f9148100390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9148106bc0 0x7f9148100390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 -- 172.21.15.45:0/2533028113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9148101960 msgr2=0x7f91480ff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9148101960 0x7f91480ff910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 -- 172.21.15.45:0/2533028113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140009d00 con 0x7f9148104290 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f9146ffd700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9148101960 0x7f91480ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.628+0000 7f91467fc700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148104290 0x7f91480ffe50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f913c00d950 tx=0x7f913c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.629+0000 7f912ffff700 1 -- 172.21.15.45:0/2533028113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00f890 con 0x7f9148104290 2023-09-23T09:29:16.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.629+0000 7f912ffff700 1 -- 172.21.15.45:0/2533028113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913c010460 con 0x7f9148104290 2023-09-23T09:29:16.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.629+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91481a1120 con 0x7f9148104290 2023-09-23T09:29:16.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.629+0000 7f912ffff700 1 -- 172.21.15.45:0/2533028113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00e610 con 0x7f9148104290 2023-09-23T09:29:16.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.629+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91481a1630 con 0x7f9148104290 2023-09-23T09:29:16.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.629+0000 7fbcaff71700 1 -- 172.21.15.45:0/4275519988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8102940 msgr2=0x7fbca8109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.629+0000 7fbcaff71700 1 --2- 172.21.15.45:0/4275519988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8102940 0x7fbca8109260 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009f90 tx=0x7fbca000b330 comp rx=0 tx=0).stop 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 -- 172.21.15.45:0/4275519988 shutdown_connections 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 --2- 172.21.15.45:0/4275519988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8102940 0x7fbca8109260 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 --2- 172.21.15.45:0/4275519988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca8100e30 0x7fbca8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 --2- 172.21.15.45:0/4275519988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbca8101f80 0x7fbca8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 -- 172.21.15.45:0/4275519988 >> 172.21.15.45:0/4275519988 conn(0x7fbca80fc3a0 msgr2=0x7fbca80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 -- 172.21.15.45:0/4275519988 shutdown_connections 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 -- 172.21.15.45:0/4275519988 wait complete. 2023-09-23T09:29:16.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91480fcdf0 con 0x7f9148104290 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.630+0000 7fbcaff71700 1 Processor -- start 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcaff71700 1 -- start start 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcaff71700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca8100e30 0x7fbca8196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcaff71700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8101f80 0x7fbca8197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcaff71700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbca8102940 0x7fbca819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcaff71700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca819e780 con 0x7fbca8101f80 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcaff71700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca819e8f0 con 0x7fbca8100e30 2023-09-23T09:29:16.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcaff71700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca819eab0 con 0x7fbca8102940 2023-09-23T09:29:16.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8101f80 0x7fbca8197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8101f80 0x7fbca8197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60414/0 (socket says 172.21.15.45:60414) 2023-09-23T09:29:16.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 -- 172.21.15.45:0/84185710 learned_addr learned my addr 172.21.15.45:0/84185710 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcae50e700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbca8102940 0x7fbca819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca8100e30 0x7fbca8196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7f912ffff700 1 -- 172.21.15.45:0/2533028113 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f913c0108d0 con 0x7f9148104290 2023-09-23T09:29:16.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 -- 172.21.15.45:0/84185710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbca8102940 msgr2=0x7fbca819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbca8102940 0x7fbca819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 -- 172.21.15.45:0/84185710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca8100e30 msgr2=0x7fbca8196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca8100e30 0x7fbca8196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.631+0000 7fbcad50c700 1 -- 172.21.15.45:0/84185710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40095f0 con 0x7fbca8101f80 2023-09-23T09:29:16.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.632+0000 7fbcad50c700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8101f80 0x7fbca8197180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800e9a0 tx=0x7fbc9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.632+0000 7f912ffff700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91300a1630 0x7f91300a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.632+0000 7f912ffff700 1 -- 172.21.15.45:0/2533028113 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f913c0cce50 con 0x7f9148104290 2023-09-23T09:29:16.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.632+0000 7f9146ffd700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91300a1630 0x7f91300a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.632+0000 7f9146ffd700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91300a1630 0x7f91300a3af0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9148101220 tx=0x7f91380073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.635+0000 7fbc9effd700 1 -- 172.21.15.45:0/84185710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98009790 con 0x7fbca8101f80 2023-09-23T09:29:16.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.635+0000 7fbc9effd700 1 -- 172.21.15.45:0/84185710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc98010430 con 0x7fbca8101f80 2023-09-23T09:29:16.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.635+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca0009c40 con 0x7fbca8101f80 2023-09-23T09:29:16.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.636+0000 7fbc9effd700 1 -- 172.21.15.45:0/84185710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98018680 con 0x7fbca8101f80 2023-09-23T09:29:16.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.636+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca8073480 con 0x7fbca8101f80 2023-09-23T09:29:16.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.636+0000 7f912ffff700 1 -- 172.21.15.45:0/2533028113 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f913c0962a0 con 0x7f9148104290 2023-09-23T09:29:16.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.638+0000 7fbc9effd700 1 -- 172.21.15.45:0/84185710 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7fbc980105a0 con 0x7fbca8101f80 2023-09-23T09:29:16.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.638+0000 7fbc9effd700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbc940a16b0 0x7fbc940a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.638+0000 7fbc9effd700 1 -- 172.21.15.45:0/84185710 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc98014070 con 0x7fbca8101f80 2023-09-23T09:29:16.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.638+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbc940a16b0 0x7fbc940a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.639+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbc940a16b0 0x7fbc940a3b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbca8198110 tx=0x7fbca4014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.640+0000 7fbcbc15bb80 1 Processor -- start 2023-09-23T09:29:16.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcbc15bb80 1 -- start start 2023-09-23T09:29:16.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb5620 0x557f2cfb5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfb6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb6e20 0x557f2cfb72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cf1d4e0 con 0x557f2cfb6e20 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cf0eca0 con 0x557f2cfb5620 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cf0c5c0 con 0x557f2cfb5f80 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb6e20 0x557f2cfb72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb6e20 0x557f2cfb72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60426/0 (socket says 172.21.15.45:60426) 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 -- 172.21.15.45:0/2123080847 learned_addr learned my addr 172.21.15.45:0/2123080847 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb5620 0x557f2cfb5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcad50c700 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfb6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 -- 172.21.15.45:0/2123080847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 msgr2=0x557f2cfb6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfb6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 -- 172.21.15.45:0/2123080847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb5620 msgr2=0x557f2cfb5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb5620 0x557f2cfb5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.641+0000 7fbcae50e700 1 -- 172.21.15.45:0/2123080847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f2cf6a950 con 0x557f2cfb6e20 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.642+0000 7fbcae50e700 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb6e20 0x557f2cfb72a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbca0015000 tx=0x7fbca00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c6aff661965de server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.642+0000 7fbc93fff700 1 -- 172.21.15.45:0/2123080847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000e040 con 0x557f2cfb6e20 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.642+0000 7fbc93fff700 1 -- 172.21.15.45:0/2123080847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca00076a0 con 0x557f2cfb6e20 2023-09-23T09:29:16.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.642+0000 7fbc93fff700 1 -- 172.21.15.45:0/2123080847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000bd50 con 0x557f2cfb6e20 2023-09-23T09:29:16.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2123080847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb6e20 msgr2=0x557f2cfb72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb6e20 0x557f2cfb72a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbca0015000 tx=0x7fbca00093c0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2123080847 shutdown_connections 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb6e20 0x557f2cfb72a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb5620 0x557f2cfb5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2123080847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfb6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2123080847 >> 172.21.15.45:0/2123080847 conn(0x557f2cf292e0 msgr2=0x557f2cfb0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2123080847 shutdown_connections 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2123080847 wait complete. 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 -- start start 2023-09-23T09:29:16.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.643+0000 7fbcbc15bb80 1 Processor -- start 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5620 0x557f2cfbf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfbf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cf07eb0 con 0x557f2cfb5620 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cea4790 con 0x557f2cfb6e20 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cea4430 con 0x557f2cfb5f80 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcbc15bb80 10 client.0 _ll_register_callbacks cb 0x557f2cd01880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5620 0x557f2cfbf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5620 0x557f2cfbf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60428/0 (socket says 172.21.15.45:60428) 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 -- 172.21.15.45:0/2430694869 learned_addr learned my addr 172.21.15.45:0/2430694869 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcae50e700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcad50c700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfbf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 -- 172.21.15.45:0/2430694869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 msgr2=0x557f2cfbf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfbf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 -- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 msgr2=0x557f2cfc6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 -- 172.21.15.45:0/2430694869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca8109b90 con 0x557f2cfb5620 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcae50e700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5620 0x557f2cfbf380 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbca4003930 tx=0x7fbca4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1130cbdef24573 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.644+0000 7fbc917fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbc917fa700 1 -- 172.21.15.45:0/2430694869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4017040 con 0x557f2cfb5620 2023-09-23T09:29:16.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbc917fa700 1 -- 172.21.15.45:0/2430694869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbca401dd50 con 0x557f2cfb5620 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f2cfa82d0 con 0x557f2cfb5620 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbc917fa700 1 -- 172.21.15.45:0/2430694869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca4003e60 con 0x557f2cfb5620 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 10 client.15369 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557f2cd32e60 con 0x557f2cfb5620 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbc917fa700 1 -- 172.21.15.45:0/2430694869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca4038020 con 0x557f2cfb5620 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbc917fa700 1 -- 172.21.15.45:0/2430694869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbca4032520 con 0x557f2cfb5620 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 10 client.15369 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 20 client.15369 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2cfcb280 0x557f2cfcd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 4 client.15369 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.645+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f2cf00e90 con 0x557f2cfcb280 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.646+0000 7fbcad50c700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2cfcb280 0x557f2cfcd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.646+0000 7fbcad50c700 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2cfcb280 0x557f2cfcd6a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8989c2514076ff2 server_cookie=876cc4ffce4206cf in_seq=0 out_seq=0 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.646+0000 7fbc917fa700 10 client.15369 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbc917fa700 1 -- 172.21.15.45:0/2430694869 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f2cf00e90 con 0x557f2cfcb280 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbc917fa700 10 client.15369 handle_command_reply: tid=0 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 client.15369 shutdown 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5620 msgr2=0x557f2cfbf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5620 0x557f2cfbf380 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbca4003930 tx=0x7fbca4004750 comp rx=0 tx=0).stop 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 shutdown_connections 2023-09-23T09:29:16.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2cfcb280 0x557f2cfcd6a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5620 0x557f2cfbf380 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbc917fa700 0 client.15369 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/2430694869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5f80 0x557f2cfbf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 >> 172.21.15.45:0/2430694869 conn(0x557f2cf292e0 msgr2=0x557f2cfb0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 shutdown_connections 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/2430694869 wait complete. 2023-09-23T09:29:16.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.647+0000 7fbcbc15bb80 20 client.15369 trim_cache size 0 max 16384 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 Processor -- start 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 -- start start 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfb9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cd889a0 con 0x557f2cfb5f80 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cfbeef0 con 0x557f2cfb6e20 2023-09-23T09:29:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2cfb6630 con 0x557f2cfb5620 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcad50c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcad50c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60436/0 (socket says 172.21.15.45:60436) 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.664+0000 7fbcad50c700 1 -- 172.21.15.45:0/771637758 learned_addr learned my addr 172.21.15.45:0/771637758 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcae50e700 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfb9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcad50c700 1 -- 172.21.15.45:0/771637758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 msgr2=0x557f2cfb9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcad50c700 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfb9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcad50c700 1 -- 172.21.15.45:0/771637758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 msgr2=0x557f2cfc68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcad50c700 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcad50c700 1 -- 172.21.15.45:0/771637758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f2cfd22f0 con 0x557f2cfb5f80 2023-09-23T09:29:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbcad50c700 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfba240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cac10 tx=0x7fbc98010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91de4f78ed2453d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbc9cff9700 1 -- 172.21.15.45:0/771637758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc980b1070 con 0x557f2cfb5f80 2023-09-23T09:29:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbc9cff9700 1 -- 172.21.15.45:0/771637758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9809a360 con 0x557f2cfb5f80 2023-09-23T09:29:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.665+0000 7fbc9cff9700 1 -- 172.21.15.45:0/771637758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc980cae10 con 0x557f2cfb5f80 2023-09-23T09:29:16.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/771637758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 msgr2=0x557f2cfba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfba240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cac10 tx=0x7fbc98010c80 comp rx=0 tx=0).stop 2023-09-23T09:29:16.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/771637758 shutdown_connections 2023-09-23T09:29:16.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfb9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/771637758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfba240 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/771637758 >> 172.21.15.45:0/771637758 conn(0x557f2cf292e0 msgr2=0x557f2cfbd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/771637758 shutdown_connections 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/771637758 wait complete. 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 -- start start 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.666+0000 7fbcbc15bb80 1 Processor -- start 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfc3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcbc15bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2d01f0c0 con 0x557f2cfb5f80 2023-09-23T09:29:16.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2d01f200 con 0x557f2cfb6e20 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcbc15bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2d01f340 con 0x557f2cfb5620 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcbc15bb80 10 client.0 _ll_register_callbacks cb 0x7fbca4004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfc3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfc3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60442/0 (socket says 172.21.15.45:60442) 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 -- 172.21.15.45:0/1527441471 learned_addr learned my addr 172.21.15.45:0/1527441471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcae50e700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 -- 172.21.15.45:0/1527441471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 msgr2=0x557f2cfc68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfc68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 -- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 msgr2=0x557f2cfc3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcad50c700 1 -- 172.21.15.45:0/1527441471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca8073480 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.667+0000 7fbcae50e700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcad50c700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfc3360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbc9809a060 tx=0x7fbc98005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c3d26cf74a0563 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbc917fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbc917fa700 1 -- 172.21.15.45:0/1527441471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc980b8070 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbc917fa700 1 -- 172.21.15.45:0/1527441471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc9809e4b0 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbca00076a0 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbc917fa700 1 -- 172.21.15.45:0/1527441471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc980b1050 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 10 client.15375 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557f2cd1bce0 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbc917fa700 1 -- 172.21.15.45:0/1527441471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca00076a0 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbc917fa700 1 -- 172.21.15.45:0/1527441471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbc98099850 con 0x557f2cfb5f80 2023-09-23T09:29:16.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 10 client.15375 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 10 client.15375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 20 client.15375 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2d020b20 0x557f2d022f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 4 client.15375 mds_command: new command op to 24530 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.668+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557f2cf00bb0 con 0x557f2d020b20 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.669+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2d020b20 0x557f2d022f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.669+0000 7fbc917fa700 10 client.15375 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.669+0000 7fbcadd0d700 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2d020b20 0x557f2d022f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251d6eab4a2427cc server_cookie=7b09ebaba478ea2c in_seq=0 out_seq=0 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbc917fa700 1 -- 172.21.15.45:0/1527441471 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557f2cf00bb0 con 0x557f2d020b20 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbc917fa700 10 client.15375 handle_command_reply: tid=0 2023-09-23T09:29:16.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbcbc15bb80 1 client.15375 shutdown 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 msgr2=0x557f2cfc3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfc3360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbc9809a060 tx=0x7fbc98005ce0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 shutdown_connections 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f2d020b20 0x557f2d022f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbc917fa700 0 client.15375 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f2cfb6e20 0x557f2cfc3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.670+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f2cfb5620 0x557f2cfc68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcbc15bb80 1 --2- 172.21.15.45:0/1527441471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2cfb5f80 0x557f2cfc3360 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 >> 172.21.15.45:0/1527441471 conn(0x557f2cf292e0 msgr2=0x557f2cfb9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 shutdown_connections 2023-09-23T09:29:16.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcbc15bb80 1 -- 172.21.15.45:0/1527441471 wait complete. 2023-09-23T09:29:16.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcbc15bb80 20 client.15375 trim_cache size 0 max 16384 2023-09-23T09:29:16.679 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:16.679 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2023-09-23T09:29:16.679 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "285aa56b-1494-47ef-a8e9-003b2d7d6146", 2023-09-23T09:29:16.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2023-09-23T09:29:16.679 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:16.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbc940a16b0 msgr2=0x7fbc940a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbc940a16b0 0x7fbc940a3b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbca8198110 tx=0x7fbca4014040 comp rx=0 tx=0).stop 2023-09-23T09:29:16.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8101f80 msgr2=0x7fbca8197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8101f80 0x7fbca8197180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800e9a0 tx=0x7fbc9800ecb0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 shutdown_connections 2023-09-23T09:29:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7fbc940a16b0 0x7fbc940a3b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbca8102940 0x7fbca819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca8100e30 0x7fbca8196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 --2- 172.21.15.45:0/84185710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbca8101f80 0x7fbca8197180 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 >> 172.21.15.45:0/84185710 conn(0x7fbca80fc3a0 msgr2=0x7fbca81054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 shutdown_connections 2023-09-23T09:29:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.671+0000 7fbcaff71700 1 -- 172.21.15.45:0/84185710 wait complete. 2023-09-23T09:29:16.684 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:16 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/4130945170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:16.684 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:16 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/706624690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:16.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:29:16.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:16 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/4130945170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:16.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:16 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/706624690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 -- 172.21.15.45:0/3181594647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06140ffd10 msgr2=0x7f0614100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 --2- 172.21.15.45:0/3181594647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06140ffd10 0x7f0614100130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f06040099c0 tx=0x7f0604009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 -- 172.21.15.45:0/3181594647 shutdown_connections 2023-09-23T09:29:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 --2- 172.21.15.45:0/3181594647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06141018a0 0x7f0614108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 --2- 172.21.15.45:0/3181594647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06140ffd10 0x7f0614100130 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 --2- 172.21.15.45:0/3181594647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0614100f00 0x7f0614101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 -- 172.21.15.45:0/3181594647 >> 172.21.15.45:0/3181594647 conn(0x7f06140fb280 msgr2=0x7f06140fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 -- 172.21.15.45:0/3181594647 shutdown_connections 2023-09-23T09:29:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.811+0000 7f06189a2700 1 -- 172.21.15.45:0/3181594647 wait complete. 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.812+0000 7f06189a2700 1 Processor -- start 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.812+0000 7f06189a2700 1 -- start start 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.812+0000 7f06189a2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06140ffd10 0x7f0614068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.812+0000 7f06189a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0614100f00 0x7f06141995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.812+0000 7f06189a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06141018a0 0x7f0614199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.812+0000 7f06189a2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061419a410 con 0x7f06141018a0 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f06189a2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061419a580 con 0x7f06140ffd10 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f06189a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061419a740 con 0x7f0614100f00 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06141018a0 0x7f0614199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06141018a0 0x7f0614199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60458/0 (socket says 172.21.15.45:60458) 2023-09-23T09:29:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 -- 172.21.15.45:0/538466181 learned_addr learned my addr 172.21.15.45:0/538466181 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0611d9b700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0614100f00 0x7f06141995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f061259c700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06140ffd10 0x7f0614068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 -- 172.21.15.45:0/538466181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0614100f00 msgr2=0x7f06141995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0614100f00 0x7f06141995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 -- 172.21.15.45:0/538466181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06140ffd10 msgr2=0x7f0614068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06140ffd10 0x7f0614068b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 -- 172.21.15.45:0/538466181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040095f0 con 0x7f06141018a0 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0611d9b700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0614100f00 0x7f06141995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f0612d9d700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06141018a0 0x7f0614199c70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f060c007ee0 tx=0x7f060c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f060b7fe700 1 -- 172.21.15.45:0/538466181 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c017070 con 0x7f06141018a0 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061419aa20 con 0x7f06141018a0 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f060b7fe700 1 -- 172.21.15.45:0/538466181 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f060c00f040 con 0x7f06141018a0 2023-09-23T09:29:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f060b7fe700 1 -- 172.21.15.45:0/538466181 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c005580 con 0x7f06141018a0 2023-09-23T09:29:16.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.813+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061419af30 con 0x7f06141018a0 2023-09-23T09:29:16.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f060b7fe700 1 -- 172.21.15.45:0/538466181 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 136243+0+0 (secure 0 0 0) 0x7f060c005780 con 0x7f06141018a0 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 Processor -- start 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 -- start start 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f060b7fe700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06000a1630 0x7f06000a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c85a0 0x560cfd3c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c8f00 0x560cfd3c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd330460 con 0x560cfd3c9da0 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd321c20 con 0x560cfd3c8f00 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd31f540 con 0x560cfd3c85a0 2023-09-23T09:29:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f0611d9b700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06000a1630 0x7f06000a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.817+0000 7f060b7fe700 1 -- 172.21.15.45:0/538466181 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f060c0cc840 con 0x7f06141018a0 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c85a0 0x560cfd3c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c85a0 0x560cfd3c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34336/0 (socket says 172.21.15.45:34336) 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 -- 172.21.15.45:0/1942166360 learned_addr learned my addr 172.21.15.45:0/1942166360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f0611d9b700 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c8f00 0x560cfd3c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f0612d9d700 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 -- 172.21.15.45:0/1942166360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c8f00 msgr2=0x560cfd3c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c8f00 0x560cfd3c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 -- 172.21.15.45:0/1942166360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 msgr2=0x560cfd3ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 -- 172.21.15.45:0/1942166360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cfd37d8d0 con 0x560cfd3c85a0 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f0611d9b700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06000a1630 0x7f06000a3af0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005d90 tx=0x7f05fc0123f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f061259c700 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c85a0 0x560cfd3c89c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06040099c0 tx=0x7f0604005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ef9ef29d9dbb48 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.818+0000 7f0608ff9700 1 -- 172.21.15.45:0/1942166360 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060401d070 con 0x560cfd3c85a0 2023-09-23T09:29:16.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0608ff9700 1 -- 172.21.15.45:0/1942166360 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f060401e040 con 0x560cfd3c85a0 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0608ff9700 1 -- 172.21.15.45:0/1942166360 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400be60 con 0x560cfd3c85a0 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1942166360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c85a0 msgr2=0x560cfd3c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c85a0 0x560cfd3c89c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06040099c0 tx=0x7f0604005f10 comp rx=0 tx=0).stop 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1942166360 shutdown_connections 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c85a0 0x560cfd3c89c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1942166360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c8f00 0x560cfd3c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1942166360 >> 172.21.15.45:0/1942166360 conn(0x560cfd33c260 msgr2=0x560cfd3c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1942166360 shutdown_connections 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.819+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1942166360 wait complete. 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 -- start start 2023-09-23T09:29:16.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 Processor -- start 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c85a0 0x560cfd3d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c9da0 0x560cfd3d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd31ae30 con 0x560cfd3c85a0 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd2b7710 con 0x560cfd3c9da0 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd2b73b0 con 0x560cfd3c8f00 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0624b8cb80 10 client.0 _ll_register_callbacks cb 0x560cfd114800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f061259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c85a0 0x560cfd3d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0612d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c9da0 0x560cfd3d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0611d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f061259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c85a0 0x560cfd3d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60468/0 (socket says 172.21.15.45:60468) 2023-09-23T09:29:16.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f0612d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c9da0 0x560cfd3d9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:53590/0 (socket says 172.21.15.45:53590) 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.820+0000 7f061259c700 1 -- 172.21.15.45:0/1506616944 learned_addr learned my addr 172.21.15.45:0/1506616944 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0611d9b700 1 -- 172.21.15.45:0/1506616944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c9da0 msgr2=0x560cfd3d9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0611d9b700 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c9da0 0x560cfd3d9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0611d9b700 1 -- 172.21.15.45:0/1506616944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c85a0 msgr2=0x560cfd3d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0611d9b700 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c85a0 0x560cfd3d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0611d9b700 1 -- 172.21.15.45:0/1506616944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0614108aa0 con 0x560cfd3c8f00 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0611d9b700 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d27d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005950 tx=0x7f05fc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=409935e692dfd0fd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f05f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1506616944 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc01ec10 con 0x560cfd3c8f00 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cfd3bb250 con 0x560cfd3c8f00 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1506616944 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f05fc01ed70 con 0x560cfd3c8f00 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1506616944 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05fc004020 con 0x560cfd3c8f00 2023-09-23T09:29:16.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0624b8cb80 10 client.25084 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x560cfd145de0 con 0x560cfd3c8f00 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.821+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1506616944 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cfd3bb250 con 0x560cfd3c8f00 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1506616944 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f05fc003c30 con 0x560cfd3c8f00 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f0624b8cb80 10 client.25084 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f0624b8cb80 10 client.25084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f0624b8cb80 20 client.25084 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd3de0b0 0x560cfd3e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f0624b8cb80 4 client.25084 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cfd313e10 con 0x560cfd3de0b0 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f061259c700 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd3de0b0 0x560cfd3e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f05f9ffb700 10 client.25084 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.822+0000 7f061259c700 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd3de0b0 0x560cfd3e04d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7622b8b6ea90cb52 server_cookie=7c7783ff2319ae46 in_seq=0 out_seq=0 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1506616944 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560cfd313e10 con 0x560cfd3de0b0 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f05f9ffb700 10 client.25084 handle_command_reply: tid=0 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 client.25084 shutdown 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 msgr2=0x560cfd3d27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d27d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005950 tx=0x7f05fc01c000 comp rx=0 tx=0).stop 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 shutdown_connections 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd3de0b0 0x560cfd3e04d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c9da0 0x560cfd3d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c85a0 0x560cfd3d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1506616944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d27d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f05f9ffb700 0 client.25084 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 >> 172.21.15.45:0/1506616944 conn(0x560cfd33c260 msgr2=0x560cfd3c3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 shutdown_connections 2023-09-23T09:29:16.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1506616944 wait complete. 2023-09-23T09:29:16.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.823+0000 7f0624b8cb80 20 client.25084 trim_cache size 0 max 16384 2023-09-23T09:29:16.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:16 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/4130945170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:29:16.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:16 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/706624690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:16.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.840+0000 7f0624b8cb80 1 Processor -- start 2023-09-23T09:29:16.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.840+0000 7f0624b8cb80 1 -- start start 2023-09-23T09:29:16.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd19b920 con 0x560cfd3c9da0 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd3c9a60 con 0x560cfd3c85a0 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd3bae10 con 0x560cfd3c8f00 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f061259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3d97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60480/0 (socket says 172.21.15.45:60480) 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 -- 172.21.15.45:0/4046730411 learned_addr learned my addr 172.21.15.45:0/4046730411 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0611d9b700 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 -- 172.21.15.45:0/4046730411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 msgr2=0x560cfd3d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 -- 172.21.15.45:0/4046730411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 msgr2=0x560cfd3d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 -- 172.21.15.45:0/4046730411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cfd12ec60 con 0x560cfd3c9da0 2023-09-23T09:29:16.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f061259c700 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:16.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f0612d9d700 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3d97d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f060c0963c0 tx=0x7f060c0b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a1684be30ad916 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f06097fa700 1 -- 172.21.15.45:0/4046730411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c099950 con 0x560cfd3c9da0 2023-09-23T09:29:16.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f06097fa700 1 -- 172.21.15.45:0/4046730411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f060c09daa0 con 0x560cfd3c9da0 2023-09-23T09:29:16.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.841+0000 7f06097fa700 1 -- 172.21.15.45:0/4046730411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c022250 con 0x560cfd3c9da0 2023-09-23T09:29:16.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 -- 172.21.15.45:0/4046730411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 msgr2=0x560cfd3d97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3d97d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f060c0963c0 tx=0x7f060c0b9040 comp rx=0 tx=0).stop 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 -- 172.21.15.45:0/4046730411 shutdown_connections 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c9da0 0x560cfd3d97d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/4046730411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c8f00 0x560cfd3d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 -- 172.21.15.45:0/4046730411 >> 172.21.15.45:0/4046730411 conn(0x560cfd33c260 msgr2=0x560cfd3c1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 -- 172.21.15.45:0/4046730411 shutdown_connections 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 -- 172.21.15.45:0/4046730411 wait complete. 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 -- start start 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.842+0000 7f0624b8cb80 1 Processor -- start 2023-09-23T09:29:16.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c8f00 0x560cfd3d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0624b8cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c9da0 0x560cfd3d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd3d9410 con 0x560cfd3c8f00 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd3d9580 con 0x560cfd3c85a0 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0624b8cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cfd3d9740 con 0x560cfd3c9da0 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0624b8cb80 10 client.0 _ll_register_callbacks cb 0x560cfd114800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c8f00 0x560cfd3d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c8f00 0x560cfd3d5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60484/0 (socket says 172.21.15.45:60484) 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 -- 172.21.15.45:0/1564303733 learned_addr learned my addr 172.21.15.45:0/1564303733 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f061259c700 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0612d9d700 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c9da0 0x560cfd3d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 -- 172.21.15.45:0/1564303733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c9da0 msgr2=0x560cfd3d8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c9da0 0x560cfd3d8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 -- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 msgr2=0x560cfd3d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 -- 172.21.15.45:0/1564303733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cfd37d8d0 con 0x560cfd3c8f00 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f0611d9b700 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c8f00 0x560cfd3d5eb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f05fc014010 tx=0x7f05fc02a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f321405c4b98f72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.843+0000 7f05f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1564303733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc02a770 con 0x560cfd3c8f00 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f060401e040 con 0x560cfd3c8f00 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1564303733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f05fc02ac40 con 0x560cfd3c8f00 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1564303733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05fc02ae00 con 0x560cfd3c8f00 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 10 client.15393 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f061419af30 con 0x560cfd3c8f00 2023-09-23T09:29:16.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1564303733 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f060401e040 con 0x560cfd3c8f00 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1564303733 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f05fc005030 con 0x560cfd3c8f00 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 10 client.15393 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 20 client.15393 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd4331e0 0x560cfd435600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 4 client.15393 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.844+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560cfd313b30 con 0x560cfd4331e0 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f061259c700 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd4331e0 0x560cfd435600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f05f9ffb700 10 client.15393 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f061259c700 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd4331e0 0x560cfd435600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4454acb094d98935 server_cookie=97a7ba5eaadfb5c in_seq=0 out_seq=0 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f05f9ffb700 1 -- 172.21.15.45:0/1564303733 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560cfd313b30 con 0x560cfd4331e0 2023-09-23T09:29:16.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f05f9ffb700 10 client.15393 handle_command_reply: tid=0 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f0624b8cb80 1 client.15393 shutdown 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c8f00 msgr2=0x560cfd3d5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c8f00 0x560cfd3d5eb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f05fc014010 tx=0x7f05fc02a600 comp rx=0 tx=0).stop 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 shutdown_connections 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560cfd4331e0 0x560cfd435600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cfd3c9da0 0x560cfd3d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.845+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560cfd3c85a0 0x560cfd3d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f0624b8cb80 1 --2- 172.21.15.45:0/1564303733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560cfd3c8f00 0x560cfd3d5eb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f05f9ffb700 0 client.15393 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 >> 172.21.15.45:0/1564303733 conn(0x560cfd33c260 msgr2=0x560cfd3d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 shutdown_connections 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f0624b8cb80 1 -- 172.21.15.45:0/1564303733 wait complete. 2023-09-23T09:29:16.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f0624b8cb80 20 client.15393 trim_cache size 0 max 16384 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: "285aa56b-1494-47ef-a8e9-003b2d7d6146": { 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "285aa56b-1494-47ef-a8e9-003b2d7d6146", 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:29:16.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:29:16.855 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:16.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06000a1630 msgr2=0x7f06000a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06000a1630 0x7f06000a3af0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005d90 tx=0x7f05fc0123f0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06141018a0 msgr2=0x7f0614199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06141018a0 0x7f0614199c70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f060c007ee0 tx=0x7f060c00d3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 shutdown_connections 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06141018a0 0x7f0614199c70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06140ffd10 0x7f0614068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f06000a1630 0x7f06000a3af0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 --2- 172.21.15.45:0/538466181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0614100f00 0x7f06141995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 >> 172.21.15.45:0/538466181 conn(0x7f06140fb280 msgr2=0x7f06140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 shutdown_connections 2023-09-23T09:29:16.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.846+0000 7f06189a2700 1 -- 172.21.15.45:0/538466181 wait complete. 2023-09-23T09:29:16.860 INFO:tasks.cephfs.filesystem:scrub status for tag:285aa56b-1494-47ef-a8e9-003b2d7d6146 - {'path': '/', 'tag': '285aa56b-1494-47ef-a8e9-003b2d7d6146', 'options': 'recursive,force'} 2023-09-23T09:29:16.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:16.917+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f914810c600 con 0x7f9148104290 2023-09-23T09:29:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.430+0000 7f912ffff700 1 -- 172.21.15.45:0/2533028113 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7f913c0160a0 con 0x7f9148104290 2023-09-23T09:29:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91300a1630 msgr2=0x7f91300a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91300a1630 0x7f91300a3af0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9148101220 tx=0x7f91380073d0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148104290 msgr2=0x7f91480ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148104290 0x7f91480ffe50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f913c00d950 tx=0x7f913c00dc60 comp rx=0 tx=0).stop 2023-09-23T09:29:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 shutdown_connections 2023-09-23T09:29:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f91300a1630 0x7f91300a3af0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9148106bc0 0x7f9148100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9148101960 0x7f91480ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 --2- 172.21.15.45:0/2533028113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148104290 0x7f91480ffe50 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f913c00d950 tx=0x7f913c00dc60 comp rx=0 tx=0).stop 2023-09-23T09:29:17.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 >> 172.21.15.45:0/2533028113 conn(0x7f91480fb270 msgr2=0x7f91481025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:17.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 shutdown_connections 2023-09-23T09:29:17.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.433+0000 7f914d619700 1 -- 172.21.15.45:0/2533028113 wait complete. 2023-09-23T09:29:17.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 -- 172.21.15.45:0/1448732324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 msgr2=0x7f8f5c0ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 --2- 172.21.15.45:0/1448732324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c0ffb80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009990 tx=0x7f8f58009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 -- 172.21.15.45:0/1448732324 shutdown_connections 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 --2- 172.21.15.45:0/1448732324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f5c100a80 0x7f8f5c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 --2- 172.21.15.45:0/1448732324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c0ffb80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 --2- 172.21.15.45:0/1448732324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c1000c0 0x7f8f5c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 -- 172.21.15.45:0/1448732324 >> 172.21.15.45:0/1448732324 conn(0x7f8f5c0fb2e0 msgr2=0x7f8f5c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 -- 172.21.15.45:0/1448732324 shutdown_connections 2023-09-23T09:29:17.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.596+0000 7f8f64765700 1 -- 172.21.15.45:0/1448732324 wait complete. 2023-09-23T09:29:17.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 Processor -- start 2023-09-23T09:29:17.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 -- start start 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f5c1000c0 0x7f8f5c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c100a80 0x7f8f5c19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c19a840 con 0x7f8f5c0ff760 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c19a9b0 con 0x7f8f5c1000c0 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f64765700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c19ab70 con 0x7f8f5c100a80 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60522/0 (socket says 172.21.15.45:60522) 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 -- 172.21.15.45:0/346049852 learned_addr learned my addr 172.21.15.45:0/346049852 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f61d00700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f5c1000c0 0x7f8f5c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62d02700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c100a80 0x7f8f5c19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:17.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 -- 172.21.15.45:0/346049852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c100a80 msgr2=0x7f8f5c19a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c100a80 0x7f8f5c19a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 -- 172.21.15.45:0/346049852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f5c1000c0 msgr2=0x7f8f5c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f5c1000c0 0x7f8f5c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.597+0000 7f8f62501700 1 -- 172.21.15.45:0/346049852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f580095f0 con 0x7f8f5c0ff760 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.598+0000 7f8f62501700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c198ec0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8f58004760 tx=0x7f8f58004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.598+0000 7f8f537fe700 1 -- 172.21.15.45:0/346049852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5801d070 con 0x7f8f5c0ff760 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.598+0000 7f8f537fe700 1 -- 172.21.15.45:0/346049852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f58004830 con 0x7f8f5c0ff760 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.598+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c19adf0 con 0x7f8f5c0ff760 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.598+0000 7f8f537fe700 1 -- 172.21.15.45:0/346049852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58005350 con 0x7f8f5c0ff760 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.598+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c1a1570 con 0x7f8f5c0ff760 2023-09-23T09:29:17.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.599+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5c10a540 con 0x7f8f5c0ff760 2023-09-23T09:29:17.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.600+0000 7f8f537fe700 1 -- 172.21.15.45:0/346049852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 136260+0+0 (secure 0 0 0) 0x7f8f580054d0 con 0x7f8f5c0ff760 2023-09-23T09:29:17.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.601+0000 7f8f537fe700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 0x7f8f480a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.601+0000 7f8f61d00700 1 -- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 msgr2=0x7f8f480a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:17.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.601+0000 7f8f537fe700 1 -- 172.21.15.45:0/346049852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8f580ce720 con 0x7f8f5c0ff760 2023-09-23T09:29:17.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.601+0000 7f8f61d00700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 0x7f8f480a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:29:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.609+0000 7f8f537fe700 1 -- 172.21.15.45:0/346049852 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8f58097c40 con 0x7f8f5c0ff760 2023-09-23T09:29:17.691 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: ignoring --setuser ceph since I am not root 2023-09-23T09:29:17.691 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: ignoring --setgroup ceph since I am not root 2023-09-23T09:29:17.691 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mgr[105520]: -- 172.21.15.123:0/1556054491 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d6330ab1e0 con 0x55d633081000 2023-09-23T09:29:17.691 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:17.608+0000 7f5aeb76a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:29:17.691 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:17.689+0000 7f5aeb76a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:29:17.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mon[103577]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2023-09-23T09:29:17.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2090813668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:17.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mon[103577]: scrub queued for path: / 2023-09-23T09:29:17.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:17.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:29:17.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:17.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2533028113' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T09:29:17.693 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: ignoring --setuser ceph since I am not root 2023-09-23T09:29:17.693 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: ignoring --setgroup ceph since I am not root 2023-09-23T09:29:17.693 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mgr[103860]: -- 172.21.15.98:0/4279005936 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e552e71e0 con 0x558e552bd000 2023-09-23T09:29:17.693 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:17.610+0000 7f9e223c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:29:17.693 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:17.691+0000 7f9e223c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:29:17.693 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mon[102386]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2023-09-23T09:29:17.694 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2090813668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:17.694 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mon[102386]: scrub queued for path: / 2023-09-23T09:29:17.694 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:17.694 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:29:17.694 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:17.694 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2533028113' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T09:29:17.745 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-mon[98627]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2090813668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-mon[98627]: scrub queued for path: / 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2533028113' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: ignoring --setuser ceph since I am not root 2023-09-23T09:29:17.746 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: ignoring --setgroup ceph since I am not root 2023-09-23T09:29:17.747 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:17.644+0000 7f88999d5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T09:29:17.747 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:17.744+0000 7f88999d5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T09:29:17.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.802+0000 7f8f61d00700 1 -- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 msgr2=0x7f8f480a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:17.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.802+0000 7f8f61d00700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 0x7f8f480a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T09:29:17.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.816+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f8f5c04ec00 con 0x7f8f5c0ff760 2023-09-23T09:29:17.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.820+0000 7f8f537fe700 1 -- 172.21.15.45:0/346049852 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8f5809b260 con 0x7f8f5c0ff760 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 msgr2=0x7f8f480a3b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 0x7f8f480a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 msgr2=0x7f8f5c198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c198ec0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8f58004760 tx=0x7f8f58004790 comp rx=0 tx=0).stop 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 shutdown_connections 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f8f480a1680 0x7f8f480a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c100a80 0x7f8f5c19a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f5c0ff760 0x7f8f5c198ec0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 --2- 172.21.15.45:0/346049852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f5c1000c0 0x7f8f5c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 >> 172.21.15.45:0/346049852 conn(0x7f8f5c0fb2e0 msgr2=0x7f8f5c0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:17.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 shutdown_connections 2023-09-23T09:29:17.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.822+0000 7f8f64765700 1 -- 172.21.15.45:0/346049852 wait complete. 2023-09-23T09:29:17.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-09-23T09:29:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 -- 172.21.15.45:0/386930702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc100e30 msgr2=0x7f3afc101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 --2- 172.21.15.45:0/386930702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc100e30 0x7f3afc101250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009990 tx=0x7f3aec009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 -- 172.21.15.45:0/386930702 shutdown_connections 2023-09-23T09:29:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 --2- 172.21.15.45:0/386930702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3afc102940 0x7f3afc109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 --2- 172.21.15.45:0/386930702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc100e30 0x7f3afc101250 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 --2- 172.21.15.45:0/386930702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3afc101f80 0x7f3afc102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 -- 172.21.15.45:0/386930702 >> 172.21.15.45:0/386930702 conn(0x7f3afc0fc3a0 msgr2=0x7f3afc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 -- 172.21.15.45:0/386930702 shutdown_connections 2023-09-23T09:29:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.964+0000 7f3b0243c700 1 -- 172.21.15.45:0/386930702 wait complete. 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 Processor -- start 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 -- start start 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3afc100e30 0x7f3afc073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc101f80 0x7f3afc073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3afc102940 0x7f3afc073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1a0bc0 con 0x7f3afc101f80 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1a0d00 con 0x7f3afc102940 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b0243c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1a0e40 con 0x7f3afc100e30 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc101f80 0x7f3afc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc101f80 0x7f3afc073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60578/0 (socket says 172.21.15.45:60578) 2023-09-23T09:29:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3af3fff700 1 -- 172.21.15.45:0/2664275046 learned_addr learned my addr 172.21.15.45:0/2664275046 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3b009d9700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3afc102940 0x7f3afc073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.965+0000 7f3afbfff700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3afc100e30 0x7f3afc073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af3fff700 1 -- 172.21.15.45:0/2664275046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3afc100e30 msgr2=0x7f3afc073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af3fff700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3afc100e30 0x7f3afc073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af3fff700 1 -- 172.21.15.45:0/2664275046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3afc102940 msgr2=0x7f3afc073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af3fff700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3afc102940 0x7f3afc073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af3fff700 1 -- 172.21.15.45:0/2664275046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0095f0 con 0x7f3afc101f80 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af3fff700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc101f80 0x7f3afc073750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d980 tx=0x7f3ae400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af9ffb700 1 -- 172.21.15.45:0/2664275046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4004220 con 0x7f3afc101f80 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc1a10d0 con 0x7f3afc101f80 2023-09-23T09:29:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af9ffb700 1 -- 172.21.15.45:0/2664275046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ae4007640 con 0x7f3afc101f80 2023-09-23T09:29:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3af9ffb700 1 -- 172.21.15.45:0/2664275046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4005690 con 0x7f3afc101f80 2023-09-23T09:29:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.966+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc1a1630 con 0x7f3afc101f80 2023-09-23T09:29:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.967+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc107da0 con 0x7f3afc101f80 2023-09-23T09:29:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.968+0000 7f3af9ffb700 1 -- 172.21.15.45:0/2664275046 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 136260+0+0 (secure 0 0 0) 0x7f3ae4004750 con 0x7f3afc101f80 2023-09-23T09:29:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.969+0000 7f3af9ffb700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 0x7f3ae80a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.969+0000 7f3afbfff700 1 -- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 msgr2=0x7f3ae80a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.969+0000 7f3af9ffb700 1 -- 172.21.15.45:0/2664275046 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3ae40cd3f0 con 0x7f3afc101f80 2023-09-23T09:29:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.969+0000 7f3afbfff700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 0x7f3ae80a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:29:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:17.972+0000 7f3af9ffb700 1 -- 172.21.15.45:0/2664275046 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3ae4096880 con 0x7f3afc101f80 2023-09-23T09:29:18.043 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:17.780+0000 7f9e223c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:29:18.043 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:17.873+0000 7f9e223c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:29:18.048 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:17.779+0000 7f5aeb76a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:29:18.048 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:17.874+0000 7f5aeb76a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:29:18.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:17.858+0000 7f88999d5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T09:29:18.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:17.978+0000 7f88999d5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T09:29:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.170+0000 7f3afbfff700 1 -- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 msgr2=0x7f3ae80a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.170+0000 7f3afbfff700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 0x7f3ae80a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T09:29:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.174+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f3afc10c680 con 0x7f3afc101f80 2023-09-23T09:29:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.178+0000 7f3af9ffb700 1 -- 172.21.15.45:0/2664275046 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3ae4099ea0 con 0x7f3afc101f80 2023-09-23T09:29:18.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 msgr2=0x7f3ae80a3b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:29:18.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 0x7f3ae80a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc101f80 msgr2=0x7f3afc073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc101f80 0x7f3afc073750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d980 tx=0x7f3ae400dc90 comp rx=0 tx=0).stop 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 shutdown_connections 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3afc102940 0x7f3afc073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f3ae80a1680 0x7f3ae80a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3afc100e30 0x7f3afc073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 --2- 172.21.15.45:0/2664275046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc101f80 0x7f3afc073750 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 >> 172.21.15.45:0/2664275046 conn(0x7f3afc0fc3a0 msgr2=0x7f3afc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.180+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 shutdown_connections 2023-09-23T09:29:18.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.181+0000 7f3b0243c700 1 -- 172.21.15.45:0/2664275046 wait complete. 2023-09-23T09:29:18.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-09-23T09:29:18.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 -- 172.21.15.45:0/4081215037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 msgr2=0x7f3320109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:18.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 --2- 172.21.15.45:0/4081215037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 0x7f3320109260 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f331800b0a0 tx=0x7f331800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 -- 172.21.15.45:0/4081215037 shutdown_connections 2023-09-23T09:29:18.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 --2- 172.21.15.45:0/4081215037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 0x7f3320109260 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 --2- 172.21.15.45:0/4081215037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3320100e30 0x7f3320101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 --2- 172.21.15.45:0/4081215037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3320101f80 0x7f3320102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 -- 172.21.15.45:0/4081215037 >> 172.21.15.45:0/4081215037 conn(0x7f33200fc3a0 msgr2=0x7f33200fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 -- 172.21.15.45:0/4081215037 shutdown_connections 2023-09-23T09:29:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.326+0000 7f33276b4700 1 -- 172.21.15.45:0/4081215037 wait complete. 2023-09-23T09:29:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 Processor -- start 2023-09-23T09:29:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 -- start start 2023-09-23T09:29:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3320100e30 0x7f3320198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3320101f80 0x7f33201994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 0x7f332019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332019a980 con 0x7f3320102940 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332019aaf0 con 0x7f3320101f80 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f33276b4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332019acb0 con 0x7f3320100e30 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f3325c51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 0x7f332019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f3325450700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3320100e30 0x7f3320198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f3325450700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3320100e30 0x7f3320198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34516/0 (socket says 172.21.15.45:34516) 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f3325c51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 0x7f332019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60646/0 (socket says 172.21.15.45:60646) 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f3325450700 1 -- 172.21.15.45:0/3153340958 learned_addr learned my addr 172.21.15.45:0/3153340958 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.327+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3320101f80 0x7f33201994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f3325450700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3320101f80 msgr2=0x7f33201994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f3325450700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3320101f80 0x7f33201994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f3325450700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 msgr2=0x7f332019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f3325450700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102940 0x7f332019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f3325450700 1 -- 172.21.15.45:0/3153340958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318009d00 con 0x7f3320100e30 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f3325450700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3320100e30 0x7f3320198f60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e9a0 tx=0x7f331c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c009790 con 0x7f3320100e30 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332019af90 con 0x7f3320100e30 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331c010430 con 0x7f3320100e30 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c018680 con 0x7f3320100e30 2023-09-23T09:29:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.328+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33201a1750 con 0x7f3320100e30 2023-09-23T09:29:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.329+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320193470 con 0x7f3320100e30 2023-09-23T09:29:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.330+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) v1 ==== 136260+0+0 (secure 0 0 0) 0x7f331c0105a0 con 0x7f3320100e30 2023-09-23T09:29:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.331+0000 7f33167fc700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.331+0000 7f3324c4f700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 msgr2=0x7f330c0a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.331+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f331c014070 con 0x7f3320100e30 2023-09-23T09:29:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.331+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T09:29:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.335+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f331c051ec0 con 0x7f3320100e30 2023-09-23T09:29:18.338 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:18.046+0000 7f5aeb76a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:29:18.411 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:18.043+0000 7f9e223c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:29:18.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.532+0000 7f3324c4f700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 msgr2=0x7f330c0a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:18.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.532+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T09:29:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.566+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 --> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f332002cfa0 con 0x7f330c0a1680 2023-09-23T09:29:18.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:18.192+0000 7f88999d5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T09:29:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2533028113' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-23T09:29:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-mon[98627]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-23T09:29:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/346049852' entity='client.admin' 2023-09-23T09:29:18.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2664275046' entity='client.admin' 2023-09-23T09:29:18.726 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2533028113' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-23T09:29:18.726 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-mon[102386]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-23T09:29:18.726 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/346049852' entity='client.admin' 2023-09-23T09:29:18.726 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2664275046' entity='client.admin' 2023-09-23T09:29:18.726 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:18.410+0000 7f9e223c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:29:18.727 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:18.541+0000 7f9e223c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:29:18.727 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:18.630+0000 7f9e223c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:29:18.728 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:18.414+0000 7f5aeb76a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:29:18.728 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:18.545+0000 7f5aeb76a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:29:18.728 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:18.633+0000 7f5aeb76a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:29:18.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2533028113' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-23T09:29:18.729 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-mon[103577]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-23T09:29:18.729 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/346049852' entity='client.admin' 2023-09-23T09:29:18.729 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2664275046' entity='client.admin' 2023-09-23T09:29:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.933+0000 7f3324c4f700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 msgr2=0x7f330c0a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:18.933+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-23T09:29:18.947 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:18.682+0000 7f88999d5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T09:29:18.947 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:18.836+0000 7f88999d5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T09:29:19.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:18.727+0000 7f5aeb76a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:29:19.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:18.946+0000 7f88999d5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T09:29:19.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:19.062+0000 7f88999d5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:29:19.433 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:19.146+0000 7f5aeb76a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:29:19.434 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:18.725+0000 7f9e223c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T09:29:19.434 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:19.142+0000 7f9e223c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:29:19.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:19 smithi123 ceph-mon[103577]: scrub summary: idle 2023-09-23T09:29:19.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:19.734+0000 7f3324c4f700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 msgr2=0x7f330c0a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:19.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:19.734+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-23T09:29:19.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:19 smithi098 ceph-mon[102386]: scrub summary: idle 2023-09-23T09:29:19.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:19 smithi045 ceph-mon[98627]: scrub summary: idle 2023-09-23T09:29:19.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:19.578+0000 7f88999d5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T09:29:20.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:19.796+0000 7f5aeb76a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:29:20.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:19.878+0000 7f5aeb76a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:29:20.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:19.792+0000 7f9e223c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:29:20.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:19.873+0000 7f9e223c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:29:20.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:20.371+0000 7f88999d5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T09:29:20.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:20.472+0000 7f88999d5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T09:29:20.906 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:20.642+0000 7f9e223c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:29:20.906 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:20.733+0000 7f9e223c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:29:20.909 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:20.646+0000 7f5aeb76a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:29:20.909 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:20.736+0000 7f5aeb76a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:29:21.183 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:20.904+0000 7f9e223c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:29:21.187 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:20.908+0000 7f5aeb76a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:29:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:21.335+0000 7f3324c4f700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 msgr2=0x7f330c0a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:21.335+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-23T09:29:21.543 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:21.182+0000 7f9e223c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:29:21.543 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:21.407+0000 7f9e223c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:29:21.559 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:21.186+0000 7f5aeb76a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:29:21.559 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:21.411+0000 7f5aeb76a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:29:21.796 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:21.456+0000 7f88999d5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:29:21.796 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:21.569+0000 7f88999d5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T09:29:21.820 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:21.556+0000 7f9e223c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:29:21.821 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:21.557+0000 7f5aeb76a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:29:22.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:21.794+0000 7f88999d5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T09:29:22.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:21.819+0000 7f5aeb76a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:29:22.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:21.818+0000 7f9e223c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:29:22.410 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:22.152+0000 7f88999d5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T09:29:22.793 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:22.513+0000 7f9e223c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:29:22.793 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:22.644+0000 7f9e223c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:29:22.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:22.408+0000 7f88999d5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T09:29:22.830 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:22.594+0000 7f88999d5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T09:29:22.838 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:22.514+0000 7f5aeb76a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:29:22.838 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:22.644+0000 7f5aeb76a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:29:23.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:22.836+0000 7f5aeb76a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:29:23.088 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:22.949+0000 7f5aeb76a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:29:23.170 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:22.838+0000 7f9e223c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:29:23.170 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:22.950+0000 7f9e223c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:29:23.170 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:23.168+0000 7f9e223c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:29:23.330 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:22.934+0000 7f88999d5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T09:29:23.588 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:23 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:23.166+0000 7f5aeb76a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:29:24.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:23.775+0000 7f88999d5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T09:29:24.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:23.937+0000 7f88999d5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T09:29:24.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:24.096+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 19) v1 ==== 136260+0+0 (secure 0 0 0) 0x7f331c010880 con 0x7f3320100e30 2023-09-23T09:29:24.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y[103836]: 2023-09-23T09:29:24.039+0000 7f9e223c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:29:24.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: Standby manager daemon z restarted 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: Standby manager daemon z started 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: Standby manager daemon y restarted 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: Standby manager daemon y started 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T09:29:24.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:24 smithi098 ceph-mon[102386]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:29:24.338 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-z[105437]: 2023-09-23T09:29:24.022+0000 7f5aeb76a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:29:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: Standby manager daemon z restarted 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: Standby manager daemon z started 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: Standby manager daemon y restarted 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: Standby manager daemon y started 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T09:29:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:24 smithi123 ceph-mon[103577]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:29:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:24.538+0000 7f3324c4f700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 msgr2=0x7f330c0a3b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/944196506 2023-09-23T09:29:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:24.538+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:24.192+0000 7f88999d5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:24.333+0000 7f88999d5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: Standby manager daemon z restarted 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: Standby manager daemon z started 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T09:29:24.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.123:0/3775850639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:29:24.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: Standby manager daemon y restarted 2023-09-23T09:29:24.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: Standby manager daemon y started 2023-09-23T09:29:24.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T09:29:24.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T09:29:24.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T09:29:24.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-mon[98627]: from='mgr.? 172.21.15.98:0/1457977688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T09:29:25.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:24.613+0000 7f88999d5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T09:29:25.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:25 smithi098 ceph-mon[102386]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-09-23T09:29:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-mon[98627]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-09-23T09:29:25.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:25 smithi123 ceph-mon[103577]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-09-23T09:29:25.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:25.713+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 20) v1 ==== 135877+0+0 (secure 0 0 0) 0x7f331c096b10 con 0x7f3320100e30 2023-09-23T09:29:25.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:25.713+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 msgr2=0x7f330c0a3b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:29:25.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:25.713+0000 7f33167fc700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/944196506,v1:172.21.15.45:6801/944196506] conn(0x7f330c0a1680 0x7f330c0a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:25.959 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.685+0000 7f88999d5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T09:29:25.959 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.956+0000 7f87f9ae3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:25.959 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.956+0000 7f87f9ae3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:25.959 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.956+0000 7f87f9ae3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:25.959 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.956+0000 7f87f9ae3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:25.959 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.956+0000 7f87f9ae3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:26.281 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.957+0000 7f87fc2e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:26.281 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.957+0000 7f87fc2e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:26.281 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.957+0000 7f87fc2e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:26.282 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.957+0000 7f87fc2e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:26.282 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:29:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:29:25.957+0000 7f87fc2e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:29:26.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: Active manager daemon x restarted 2023-09-23T09:29:26.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: Activating manager daemon x 2023-09-23T09:29:26.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: osdmap e86: 12 total, 12 up, 12 in 2023-09-23T09:29:26.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: mgrmap e20: x(active, starting, since 0.0115612s), standbys: z, y 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:29:26.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:29:26.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T09:29:26.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: Manager daemon x is now available 2023-09-23T09:29:26.548 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:26.548 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:26.548 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T09:29:26.548 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:26 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T09:29:26.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: Active manager daemon x restarted 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: Activating manager daemon x 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: osdmap e86: 12 total, 12 up, 12 in 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: mgrmap e20: x(active, starting, since 0.0115612s), standbys: z, y 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T09:29:26.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T09:29:26.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:29:26.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: Manager daemon x is now available 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T09:29:26.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:26 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T09:29:26.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: Active manager daemon x restarted 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: Activating manager daemon x 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: osdmap e86: 12 total, 12 up, 12 in 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: mgrmap e20: x(active, starting, since 0.0115612s), standbys: z, y 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T09:29:26.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T09:29:26.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T09:29:26.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T09:29:26.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T09:29:26.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: Manager daemon x is now available 2023-09-23T09:29:26.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:26.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:26.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T09:29:26.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:26 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T09:29:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.714+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mon.1 v2:172.21.15.123:3300/0 9 ==== mgrmap(e 21) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f331c0b8590 con 0x7f3320100e30 2023-09-23T09:29:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.715+0000 7f33167fc700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f330c01b3b0 0x7f330c01b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.715+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 --> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f332002cfa0 con 0x7f330c01b3b0 2023-09-23T09:29:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.715+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f330c01b3b0 0x7f330c01b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:26.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.722+0000 7f3324c4f700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f330c01b3b0 0x7f330c01b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3310009880 tx=0x7f3310009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:26.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.786+0000 7f33167fc700 1 -- 172.21.15.45:0/3153340958 <== mgr.15405 v2:172.21.15.45:6800/441481757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f332002cfa0 con 0x7f330c01b3b0 2023-09-23T09:29:26.788 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path / 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f330c01b3b0 msgr2=0x7f330c01b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f330c01b3b0 0x7f330c01b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3310009880 tx=0x7f3310009260 comp rx=0 tx=0).stop 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3320100e30 msgr2=0x7f3320198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 --2- 172.21.15.45:0/3153340958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3320100e30 0x7f3320198f60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e9a0 tx=0x7f331c00ecb0 comp rx=0 tx=0).stop 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f3325c51700 1 -- 172.21.15.45:0/3153340958 reap_dead start 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 shutdown_connections 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 >> 172.21.15.45:0/3153340958 conn(0x7f33200fc3a0 msgr2=0x7f3320107910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 shutdown_connections 2023-09-23T09:29:26.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.788+0000 7f33276b4700 1 -- 172.21.15.45:0/3153340958 wait complete. 2023-09-23T09:29:26.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-09-23T09:29:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 -- 172.21.15.45:0/1721147567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48100f60 msgr2=0x7fea481013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/1721147567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48100f60 0x7fea481013c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0099c0 tx=0x7fea3c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:29:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 -- 172.21.15.45:0/1721147567 shutdown_connections 2023-09-23T09:29:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/1721147567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea48101900 0x7fea481081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/1721147567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea480ffd70 0x7fea48100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/1721147567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48100f60 0x7fea481013c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 -- 172.21.15.45:0/1721147567 >> 172.21.15.45:0/1721147567 conn(0x7fea480fb2e0 msgr2=0x7fea480fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 -- 172.21.15.45:0/1721147567 shutdown_connections 2023-09-23T09:29:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.941+0000 7fea4e5dd700 1 -- 172.21.15.45:0/1721147567 wait complete. 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.942+0000 7fea4e5dd700 1 Processor -- start 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.942+0000 7fea4e5dd700 1 -- start start 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4e5dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea480ffd70 0x7fea48195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4e5dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea48100f60 0x7fea48196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4e5dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48101900 0x7fea48196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4e5dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48197540 con 0x7fea48101900 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4e5dd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea481976b0 con 0x7fea48100f60 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4e5dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48197870 con 0x7fea480ffd70 2023-09-23T09:29:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48101900 0x7fea48196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48101900 0x7fea48196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60886/0 (socket says 172.21.15.45:60886) 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 -- 172.21.15.45:0/804448679 learned_addr learned my addr 172.21.15.45:0/804448679 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea477fe700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea48100f60 0x7fea48196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea47fff700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea480ffd70 0x7fea48195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 -- 172.21.15.45:0/804448679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea480ffd70 msgr2=0x7fea48195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea480ffd70 0x7fea48195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 -- 172.21.15.45:0/804448679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea48100f60 msgr2=0x7fea48196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea48100f60 0x7fea48196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 -- 172.21.15.45:0/804448679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0095f0 con 0x7fea48101900 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4cb7a700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48101900 0x7fea48196ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea4000ef30 tx=0x7fea4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea457fa700 1 -- 172.21.15.45:0/804448679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40018070 con 0x7fea48101900 2023-09-23T09:29:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea48197af0 con 0x7fea48101900 2023-09-23T09:29:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea457fa700 1 -- 172.21.15.45:0/804448679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea4000f040 con 0x7fea48101900 2023-09-23T09:29:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.943+0000 7fea457fa700 1 -- 172.21.15.45:0/804448679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cd00 con 0x7fea48101900 2023-09-23T09:29:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.944+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4819e310 con 0x7fea48101900 2023-09-23T09:29:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.945+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea48107330 con 0x7fea48101900 2023-09-23T09:29:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.946+0000 7fea457fa700 1 -- 172.21.15.45:0/804448679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fea40003a00 con 0x7fea48101900 2023-09-23T09:29:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.946+0000 7fea457fa700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fea300a13a0 0x7fea300a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.947+0000 7fea47fff700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fea300a13a0 0x7fea300a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.947+0000 7fea457fa700 1 -- 172.21.15.45:0/804448679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fea400cc7c0 con 0x7fea48101900 2023-09-23T09:29:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.947+0000 7fea47fff700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fea300a13a0 0x7fea300a3860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea48100dc0 tx=0x7fea38009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:26.951+0000 7fea457fa700 1 -- 172.21.15.45:0/804448679 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fea40096000 con 0x7fea48101900 2023-09-23T09:29:27.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.185+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 --> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fea4802d040 con 0x7fea300a13a0 2023-09-23T09:29:27.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.239+0000 7fea457fa700 1 -- 172.21.15.45:0/804448679 <== mgr.15405 v2:172.21.15.45:6800/441481757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fea4802d040 con 0x7fea300a13a0 2023-09-23T09:29:27.241 INFO:teuthology.orchestra.run.smithi045.stdout:Retention added to path / 2023-09-23T09:29:27.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fea300a13a0 msgr2=0x7fea300a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fea300a13a0 0x7fea300a3860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea48100dc0 tx=0x7fea38009260 comp rx=0 tx=0).stop 2023-09-23T09:29:27.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48101900 msgr2=0x7fea48196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48101900 0x7fea48196ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea4000ef30 tx=0x7fea4000c3a0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 shutdown_connections 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fea300a13a0 0x7fea300a3860 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea48101900 0x7fea48196ee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea480ffd70 0x7fea48195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 --2- 172.21.15.45:0/804448679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea48100f60 0x7fea48196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.241+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 >> 172.21.15.45:0/804448679 conn(0x7fea480fb2e0 msgr2=0x7fea48104470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.242+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 shutdown_connections 2023-09-23T09:29:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.242+0000 7fea4e5dd700 1 -- 172.21.15.45:0/804448679 wait complete. 2023-09-23T09:29:27.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-23T09:29:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 -- 172.21.15.45:0/2734463565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 msgr2=0x7f4770109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 --2- 172.21.15.45:0/2734463565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 0x7f4770109260 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4768009f90 tx=0x7f476800b330 comp rx=0 tx=0).stop 2023-09-23T09:29:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 -- 172.21.15.45:0/2734463565 shutdown_connections 2023-09-23T09:29:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 --2- 172.21.15.45:0/2734463565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 0x7f4770109260 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 --2- 172.21.15.45:0/2734463565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770100e30 0x7f4770101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 --2- 172.21.15.45:0/2734463565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770101f80 0x7f4770102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 -- 172.21.15.45:0/2734463565 >> 172.21.15.45:0/2734463565 conn(0x7f47700fc3a0 msgr2=0x7f47700fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:27.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 -- 172.21.15.45:0/2734463565 shutdown_connections 2023-09-23T09:29:27.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.400+0000 7f477751d700 1 -- 172.21.15.45:0/2734463565 wait complete. 2023-09-23T09:29:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 Processor -- start 2023-09-23T09:29:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 -- start start 2023-09-23T09:29:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770100e30 0x7f4770073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770101f80 0x7f4770073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 0x7f4770073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47701a0ca0 con 0x7f4770102940 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47701a0de0 con 0x7f4770100e30 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.401+0000 7f477751d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47701a0f20 con 0x7f4770101f80 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4774ab8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770101f80 0x7f4770073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4775aba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 0x7f4770073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f47752b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770100e30 0x7f4770073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f47752b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770100e30 0x7f4770073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54014/0 (socket says 172.21.15.45:54014) 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4774ab8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770101f80 0x7f4770073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34772/0 (socket says 172.21.15.45:34772) 2023-09-23T09:29:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f47752b9700 1 -- 172.21.15.45:0/3154314208 learned_addr learned my addr 172.21.15.45:0/3154314208 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4775aba700 1 -- 172.21.15.45:0/3154314208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770101f80 msgr2=0x7f4770073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4775aba700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770101f80 0x7f4770073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4775aba700 1 -- 172.21.15.45:0/3154314208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770100e30 msgr2=0x7f4770073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4775aba700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770100e30 0x7f4770073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4775aba700 1 -- 172.21.15.45:0/3154314208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768009c40 con 0x7f4770102940 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4774ab8700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770101f80 0x7f4770073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f4775aba700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 0x7f4770073c90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4768007930 tx=0x7f4768004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f47667fc700 1 -- 172.21.15.45:0/3154314208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476800e040 con 0x7f4770102940 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47701a1150 con 0x7f4770102940 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f47667fc700 1 -- 172.21.15.45:0/3154314208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4768008ef0 con 0x7f4770102940 2023-09-23T09:29:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47701a1620 con 0x7f4770102940 2023-09-23T09:29:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.403+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47701083c0 con 0x7f4770102940 2023-09-23T09:29:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.402+0000 7f47667fc700 1 -- 172.21.15.45:0/3154314208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680052a0 con 0x7f4770102940 2023-09-23T09:29:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.405+0000 7f47667fc700 1 -- 172.21.15.45:0/3154314208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f476801d470 con 0x7f4770102940 2023-09-23T09:29:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.406+0000 7f47667fc700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f475c0a13a0 0x7f475c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.406+0000 7f47752b9700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f475c0a13a0 0x7f475c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.406+0000 7f47667fc700 1 -- 172.21.15.45:0/3154314208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f47680ce8d0 con 0x7f4770102940 2023-09-23T09:29:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.407+0000 7f47752b9700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f475c0a13a0 0x7f475c0a3860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4770074b20 tx=0x7f476c009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.409+0000 7f47667fc700 1 -- 172.21.15.45:0/3154314208 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4768098230 con 0x7f4770102940 2023-09-23T09:29:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:27 smithi098 ceph-mon[102386]: [23/Sep/2023:09:29:26] ENGINE Bus STARTING 2023-09-23T09:29:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:27 smithi098 ceph-mon[102386]: [23/Sep/2023:09:29:26] ENGINE Serving on https://172.21.15.45:7150 2023-09-23T09:29:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:27 smithi098 ceph-mon[102386]: [23/Sep/2023:09:29:26] ENGINE Serving on http://172.21.15.45:8765 2023-09-23T09:29:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:27 smithi098 ceph-mon[102386]: [23/Sep/2023:09:29:26] ENGINE Bus STARTED 2023-09-23T09:29:27.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:27 smithi098 ceph-mon[102386]: mgrmap e21: x(active, since 1.01936s), standbys: z, y 2023-09-23T09:29:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:27 smithi045 ceph-mon[98627]: [23/Sep/2023:09:29:26] ENGINE Bus STARTING 2023-09-23T09:29:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:27 smithi045 ceph-mon[98627]: [23/Sep/2023:09:29:26] ENGINE Serving on https://172.21.15.45:7150 2023-09-23T09:29:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:27 smithi045 ceph-mon[98627]: [23/Sep/2023:09:29:26] ENGINE Serving on http://172.21.15.45:8765 2023-09-23T09:29:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:27 smithi045 ceph-mon[98627]: [23/Sep/2023:09:29:26] ENGINE Bus STARTED 2023-09-23T09:29:27.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:27 smithi045 ceph-mon[98627]: mgrmap e21: x(active, since 1.01936s), standbys: z, y 2023-09-23T09:29:27.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:27 smithi123 ceph-mon[103577]: [23/Sep/2023:09:29:26] ENGINE Bus STARTING 2023-09-23T09:29:27.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:27 smithi123 ceph-mon[103577]: [23/Sep/2023:09:29:26] ENGINE Serving on https://172.21.15.45:7150 2023-09-23T09:29:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:27 smithi123 ceph-mon[103577]: [23/Sep/2023:09:29:26] ENGINE Serving on http://172.21.15.45:8765 2023-09-23T09:29:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:27 smithi123 ceph-mon[103577]: [23/Sep/2023:09:29:26] ENGINE Bus STARTED 2023-09-23T09:29:27.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:27 smithi123 ceph-mon[103577]: mgrmap e21: x(active, since 1.01936s), standbys: z, y 2023-09-23T09:29:27.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.649+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 --> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f4770061260 con 0x7f475c0a13a0 2023-09-23T09:29:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.657+0000 7f47667fc700 1 -- 172.21.15.45:0/3154314208 <== mgr.15405 v2:172.21.15.45:6800/441481757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f4770061260 con 0x7f475c0a13a0 2023-09-23T09:29:27.659 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-09-23T00:00:00", "created": "2023-09-23T09:29:26", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f475c0a13a0 msgr2=0x7f475c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f475c0a13a0 0x7f475c0a3860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4770074b20 tx=0x7f476c009260 comp rx=0 tx=0).stop 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 msgr2=0x7f4770073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 0x7f4770073c90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4768007930 tx=0x7f4768004d00 comp rx=0 tx=0).stop 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 shutdown_connections 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f475c0a13a0 0x7f475c0a3860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4770102940 0x7f4770073c90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770100e30 0x7f4770073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 --2- 172.21.15.45:0/3154314208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4770101f80 0x7f4770073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 >> 172.21.15.45:0/3154314208 conn(0x7f47700fc3a0 msgr2=0x7f4770105500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 shutdown_connections 2023-09-23T09:29:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.660+0000 7f477751d700 1 -- 172.21.15.45:0/3154314208 wait complete. 2023-09-23T09:29:27.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 -- 172.21.15.45:0/2720856427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c102990 msgr2=0x7fec9c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 --2- 172.21.15.45:0/2720856427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c102990 0x7fec9c1092b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fec94009990 tx=0x7fec94009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 -- 172.21.15.45:0/2720856427 shutdown_connections 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 --2- 172.21.15.45:0/2720856427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c102990 0x7fec9c1092b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 --2- 172.21.15.45:0/2720856427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9c100e80 0x7fec9c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 --2- 172.21.15.45:0/2720856427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec9c101fd0 0x7fec9c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 -- 172.21.15.45:0/2720856427 >> 172.21.15.45:0/2720856427 conn(0x7fec9c0fc3d0 msgr2=0x7fec9c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 -- 172.21.15.45:0/2720856427 shutdown_connections 2023-09-23T09:29:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.816+0000 7feca2cbc700 1 -- 172.21.15.45:0/2720856427 wait complete. 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 Processor -- start 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 -- start start 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9c100e80 0x7fec9c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c101fd0 0x7fec9c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec9c102990 0x7fec9c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c1985e0 con 0x7fec9c101fd0 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c198750 con 0x7fec9c102990 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca2cbc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c198910 con 0x7fec9c100e80 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c101fd0 0x7fec9c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c101fd0 0x7fec9c197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60914/0 (socket says 172.21.15.45:60914) 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 -- 172.21.15.45:0/3513562367 learned_addr learned my addr 172.21.15.45:0/3513562367 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca0a58700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9c100e80 0x7fec9c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7feca1259700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec9c102990 0x7fec9c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 -- 172.21.15.45:0/3513562367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9c100e80 msgr2=0x7fec9c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9c100e80 0x7fec9c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 -- 172.21.15.45:0/3513562367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec9c102990 msgr2=0x7fec9c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec9c102990 0x7fec9c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.817+0000 7fec9bfff700 1 -- 172.21.15.45:0/3513562367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec940095f0 con 0x7fec9c101fd0 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.818+0000 7fec9bfff700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c101fd0 0x7fec9c197100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e9a0 tx=0x7fec8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.818+0000 7fec99ffb700 1 -- 172.21.15.45:0/3513562367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84009790 con 0x7fec9c101fd0 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.818+0000 7fec99ffb700 1 -- 172.21.15.45:0/3513562367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec84010430 con 0x7fec9c101fd0 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.818+0000 7fec99ffb700 1 -- 172.21.15.45:0/3513562367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84018680 con 0x7fec9c101fd0 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.818+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c198bf0 con 0x7fec9c101fd0 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.818+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c19f3b0 con 0x7fec9c101fd0 2023-09-23T09:29:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.819+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c108360 con 0x7fec9c101fd0 2023-09-23T09:29:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.820+0000 7fec99ffb700 1 -- 172.21.15.45:0/3513562367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fec840105a0 con 0x7fec9c101fd0 2023-09-23T09:29:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.821+0000 7fec99ffb700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fec880a14f0 0x7fec880a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.821+0000 7feca0a58700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fec880a14f0 0x7fec880a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.821+0000 7fec99ffb700 1 -- 172.21.15.45:0/3513562367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fec84014070 con 0x7fec9c101fd0 2023-09-23T09:29:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.821+0000 7feca0a58700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fec880a14f0 0x7fec880a39b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009550 tx=0x7fec8c0091b0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:27.825+0000 7fec99ffb700 1 -- 172.21.15.45:0/3513562367 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fec84096780 con 0x7fec9c101fd0 2023-09-23T09:29:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.087+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 --> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fec9c02cfd0 con 0x7fec880a14f0 2023-09-23T09:29:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.095+0000 7fec99ffb700 1 -- 172.21.15.45:0/3513562367 <== mgr.15405 v2:172.21.15.45:6800/441481757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fec9c02cfd0 con 0x7fec880a14f0 2023-09-23T09:29:28.096 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2023-09-23T09:29:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fec880a14f0 msgr2=0x7fec880a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fec880a14f0 0x7fec880a39b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009550 tx=0x7fec8c0091b0 comp rx=0 tx=0).stop 2023-09-23T09:29:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c101fd0 msgr2=0x7fec9c197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c101fd0 0x7fec9c197100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e9a0 tx=0x7fec8400ecb0 comp rx=0 tx=0).stop 2023-09-23T09:29:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 shutdown_connections 2023-09-23T09:29:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec9c102990 0x7fec9c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9c100e80 0x7fec9c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fec880a14f0 0x7fec880a39b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 --2- 172.21.15.45:0/3513562367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9c101fd0 0x7fec9c197100 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 >> 172.21.15.45:0/3513562367 conn(0x7fec9c0fc3d0 msgr2=0x7fec9c1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 shutdown_connections 2023-09-23T09:29:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:28.097+0000 7feca2cbc700 1 -- 172.21.15.45:0/3513562367 wait complete. 2023-09-23T09:29:28.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-09-23T09:29:28.140 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-09-23T09:29:28.157 INFO:teuthology.run_tasks:Running task background_exec... 2023-09-23T09:29:28.219 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:29:28.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-09-23T09:29:28.240 INFO:teuthology.run_tasks:Running task exec... 2023-09-23T09:29:28.249 INFO:teuthology.task.exec:Executing custom commands... 2023-09-23T09:29:28.249 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:29:28.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-09-23T09:29:28.292 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:28.329 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem Size Used Avail Use% Mounted on 2023-09-23T09:29:28.329 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:3300,172.21.15.98:3300,172.21.15.123:3300:/volumes/_nogroup/sv_1/03909e3e-9314-4e2a-91e9-07669c01496d 340G 0 340G 0% /tmp/cephfs 2023-09-23T09:29:28.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-09-23T09:29:28.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:28 smithi045 ceph-mon[98627]: from='client.25117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:28.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:28 smithi045 ceph-mon[98627]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:28.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:28 smithi045 ceph-mon[98627]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:28.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-09-23T09:29:28.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-09-23T09:29:28.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:28 smithi098 ceph-mon[102386]: from='client.25117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:28.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:28 smithi098 ceph-mon[102386]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:28.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:28 smithi098 ceph-mon[102386]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:28.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:28 smithi123 ceph-mon[103577]: from='client.25117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:28.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:28 smithi123 ceph-mon[103577]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:28.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:28 smithi123 ceph-mon[103577]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:29.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:29.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:29.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:29.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-09-23T09:29:29.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:29 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:29.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-09-23T09:29:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:29 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:29.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:29.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:29 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.129 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:31.129 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.129 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi045 to 197.2M 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Unable to set osd_memory_target on smithi045 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-23T09:29:31.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Updating smithi045:/etc/ceph/ceph.conf 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:30 smithi045 ceph-mon[98627]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-09-23T09:29:31.266 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-09-23T09:29:31.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-09-23T09:29:31.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi045 to 197.2M 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Unable to set osd_memory_target on smithi045 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Updating smithi045:/etc/ceph/ceph.conf 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:30 smithi098 ceph-mon[102386]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi045 to 197.2M 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Unable to set osd_memory_target on smithi045 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Updating smithi045:/etc/ceph/ceph.conf 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:30 smithi123 ceph-mon[103577]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/: 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 .. 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 data 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data: 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:31.389 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 .. 2023-09-23T09:29:31.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-09-23T09:29:31.479 INFO:teuthology.orchestra.run.smithi045.stdout:[Service] 2023-09-23T09:29:31.479 INFO:teuthology.orchestra.run.smithi045.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-09-23T09:29:31.479 INFO:teuthology.orchestra.run.smithi045.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-09-23T09:29:31.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-09-23T09:29:31.565 INFO:teuthology.orchestra.run.smithi045.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-09-23T09:29:32.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:32.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:29:32.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:29:32.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:29:32.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: Updating smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: Updating smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: Updating smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:29:32.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:29:32.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:29:33.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:33 smithi098 ceph-mon[102386]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-23T09:29:33.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:33 smithi045 ceph-mon[98627]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-23T09:29:33.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:33 smithi123 ceph-mon[103577]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-23T09:29:35.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:35 smithi098 ceph-mon[102386]: pgmap v7: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 148 KiB/s wr, 69 op/s 2023-09-23T09:29:35.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:35 smithi045 ceph-mon[98627]: pgmap v7: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 148 KiB/s wr, 69 op/s 2023-09-23T09:29:35.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:35 smithi123 ceph-mon[103577]: pgmap v7: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 148 KiB/s wr, 69 op/s 2023-09-23T09:29:37.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:37 smithi098 ceph-mon[102386]: pgmap v8: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 115 KiB/s wr, 54 op/s 2023-09-23T09:29:37.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:37 smithi045 ceph-mon[98627]: pgmap v8: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 115 KiB/s wr, 54 op/s 2023-09-23T09:29:37.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:37 smithi123 ceph-mon[103577]: pgmap v8: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 115 KiB/s wr, 54 op/s 2023-09-23T09:29:39.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:39 smithi098 ceph-mon[102386]: pgmap v9: 97 pgs: 97 active+clean; 30 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 2.8 MiB/s wr, 92 op/s 2023-09-23T09:29:39.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:39 smithi045 ceph-mon[98627]: pgmap v9: 97 pgs: 97 active+clean; 30 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 2.8 MiB/s wr, 92 op/s 2023-09-23T09:29:39.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:39 smithi123 ceph-mon[103577]: pgmap v9: 97 pgs: 97 active+clean; 30 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 2.8 MiB/s wr, 92 op/s 2023-09-23T09:29:39.599 INFO:teuthology.orchestra.run.smithi045.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-09-23T09:29:39.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-23T09:29:39.649 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/: 2023-09-23T09:29:39.649 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.649 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:29:39.649 INFO:teuthology.orchestra.run.smithi045.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 .. 2023-09-23T09:29:39.650 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 data 2023-09-23T09:29:39.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.650 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data: 2023-09-23T09:29:39.650 INFO:teuthology.orchestra.run.smithi045.stdout:total 30 2023-09-23T09:29:39.650 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 . 2023-09-23T09:29:39.650 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 .. 2023-09-23T09:29:39.650 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 base 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 23 09:29 global 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 log 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_commit_ts 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_dynshmem 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 23 09:29 pg_hba.conf 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 23 09:29 pg_ident.conf 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 pg_logical 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 pg_multixact 2023-09-23T09:29:39.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 pg_notify 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_replslot 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_serial 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_snapshots 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_stat 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_stat_tmp 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 pg_subtrans 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_tblspc 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_twophase 2023-09-23T09:29:39.652 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 pg_wal 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 pg_xact 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 23 09:29 postgresql.auto.conf 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 23 09:29 postgresql.conf 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base: 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 . 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 1 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 12346 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 12347 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:total 7201 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 . 2023-09-23T09:29:39.654 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 .. 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 112 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 113 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 23 09:29 12182 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12182_fsm 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12182_vm 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12184 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12186 2023-09-23T09:29:39.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12187_fsm 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187_vm 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12189 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12191 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12192_fsm 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192_vm 2023-09-23T09:29:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12194 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12196 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12197_fsm 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197_vm 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12199 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12201 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12202_fsm 2023-09-23T09:29:39.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202_vm 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12204 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12206 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12207_fsm 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207_vm 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12209 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12211 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12212 2023-09-23T09:29:39.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12214 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12216 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 23 09:29 1247 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1247_fsm 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1247_vm 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 23 09:29 1249 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1249_fsm 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1249_vm 2023-09-23T09:29:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 23 09:29 1255 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1255_fsm 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1255_vm 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 23 09:29 1259 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1259_fsm 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1259_vm 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417_vm 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418 2023-09-23T09:29:39.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418_vm 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 174 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 175 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2187 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224_vm 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328_vm 2023-09-23T09:29:39.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336_vm 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2337 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2600 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2600_fsm 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2600_vm 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2601_fsm 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601_vm 2023-09-23T09:29:39.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2602 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2602_fsm 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2602_vm 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2603 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2603_fsm 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2603_vm 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604_vm 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2605 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2605_fsm 2023-09-23T09:29:39.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2605_vm 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2606_fsm 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606_vm 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607_fsm 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2607_vm 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 23 09:29 2608 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2608_fsm 2023-09-23T09:29:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2608_vm 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 23 09:29 2609 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2609_fsm 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2609_vm 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610_fsm 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2610_vm 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611 2023-09-23T09:29:39.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611_vm 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2612_fsm 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612_vm 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613_vm 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2615_fsm 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615_vm 2023-09-23T09:29:39.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616_fsm 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2616_vm 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 23 09:29 2617 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2617_fsm 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2617_vm 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 23 09:29 2618 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2618_fsm 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2618_vm 2023-09-23T09:29:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 23 09:29 2619 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2619_fsm 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2619_vm 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620_vm 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2650 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2651 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2652 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2653 2023-09-23T09:29:39.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2654 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2655 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2656 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2657 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 23 09:29 2658 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2659 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2660 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2661 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2662 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2663 2023-09-23T09:29:39.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2664 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2665 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2666 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2667 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2668 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2669 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2670 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Sep 23 09:29 2673 2023-09-23T09:29:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 23 09:29 2674 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 23 09:29 2675 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2678 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2679 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2680 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2681 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2682 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2683 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2684 2023-09-23T09:29:39.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2685 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2686 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2687 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2688 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2689 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2690 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 23 09:29 2691 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2692 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2693 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2696 2023-09-23T09:29:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2699 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2701 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2702 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2703 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2704 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2753 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2753_fsm 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2753_vm 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2754 2023-09-23T09:29:39.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2755 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2756 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2757 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830_vm 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2831 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832_vm 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2833 2023-09-23T09:29:39.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834_vm 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2835 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836_vm 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2837 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 23 09:29 2838 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2838_fsm 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2838_vm 2023-09-23T09:29:39.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2839 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840_fsm 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2840_vm 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2841 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995_vm 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2996 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079 2023-09-23T09:29:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3079_fsm 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079_vm 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3080 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3081 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3085 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118_vm 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3119 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3164 2023-09-23T09:29:39.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256_vm 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3257 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3258 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350_vm 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3351 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3379 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3380 2023-09-23T09:29:39.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381_vm 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3394 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3394_fsm 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3394_vm 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3395 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439_vm 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3440 2023-09-23T09:29:39.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 3455 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3456 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3456_fsm 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3456_vm 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466_vm 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3467 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3468 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501 2023-09-23T09:29:39.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501_vm 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3502 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3503 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3534 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3541_fsm 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541_vm 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3542 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3574 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3575 2023-09-23T09:29:39.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576_vm 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596_vm 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3597 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598_vm 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3599 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600 2023-09-23T09:29:39.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3600_fsm 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600_vm 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3601_fsm 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601_vm 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3602_fsm 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602_vm 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3603 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3603_fsm 2023-09-23T09:29:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3603_vm 2023-09-23T09:29:39.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3604 2023-09-23T09:29:39.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3605 2023-09-23T09:29:39.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3606 2023-09-23T09:29:39.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3607 2023-09-23T09:29:39.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3608 2023-09-23T09:29:39.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 3609 2023-09-23T09:29:39.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3712 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3764_fsm 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764_vm 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3766 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3767 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3997 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 5002 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 548 2023-09-23T09:29:39.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 549 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102_vm 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104_vm 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106_vm 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6110 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6111 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6112 2023-09-23T09:29:39.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6113 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6117 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826_vm 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 827 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 828 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.690 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:total 7201 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 . 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 .. 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 112 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 113 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 23 09:29 12182 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12182_fsm 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12182_vm 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12184 2023-09-23T09:29:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12186 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12187_fsm 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187_vm 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12189 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12191 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12192_fsm 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192_vm 2023-09-23T09:29:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12194 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12196 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12197_fsm 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197_vm 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12199 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12201 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12202_fsm 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202_vm 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12204 2023-09-23T09:29:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12206 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12207_fsm 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207_vm 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12209 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12211 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12212 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12214 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12216 2023-09-23T09:29:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 23 09:29 1247 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1247_fsm 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1247_vm 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 23 09:29 1249 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1249_fsm 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1249_vm 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 23 09:29 1255 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1255_fsm 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1255_vm 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 23 09:29 1259 2023-09-23T09:29:39.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1259_fsm 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1259_vm 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417_vm 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418_vm 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 174 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 175 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2187 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224 2023-09-23T09:29:39.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224_vm 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328_vm 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336_vm 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2337 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2600 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2600_fsm 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2600_vm 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601 2023-09-23T09:29:39.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2601_fsm 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601_vm 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2602 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2602_fsm 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2602_vm 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2603 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2603_fsm 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2603_vm 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604 2023-09-23T09:29:39.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604_vm 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2605 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2605_fsm 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2605_vm 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2606_fsm 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606_vm 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607_fsm 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2607_vm 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 23 09:29 2608 2023-09-23T09:29:39.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2608_fsm 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2608_vm 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 23 09:29 2609 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2609_fsm 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2609_vm 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610_fsm 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2610_vm 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611_vm 2023-09-23T09:29:39.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2612_fsm 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612_vm 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613_vm 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2615_fsm 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615_vm 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616_fsm 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2616_vm 2023-09-23T09:29:39.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 23 09:29 2617 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2617_fsm 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2617_vm 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 23 09:29 2618 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2618_fsm 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2618_vm 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 23 09:29 2619 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2619_fsm 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2619_vm 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620 2023-09-23T09:29:39.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620_vm 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2650 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2651 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2652 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2653 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2654 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2655 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2656 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2657 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 23 09:29 2658 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2659 2023-09-23T09:29:39.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2660 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2661 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2662 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2663 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2664 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2665 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2666 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2667 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2668 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2669 2023-09-23T09:29:39.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2670 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Sep 23 09:29 2673 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 23 09:29 2674 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 23 09:29 2675 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2678 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2679 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2680 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2681 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2682 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2683 2023-09-23T09:29:39.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2684 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2685 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2686 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2687 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2688 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2689 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2690 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 23 09:29 2691 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2692 2023-09-23T09:29:39.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2693 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2696 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2699 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2701 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2702 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2703 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2704 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2753 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2753_fsm 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2753_vm 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2754 2023-09-23T09:29:39.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2755 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2756 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2757 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830_vm 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2831 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832_vm 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2833 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834_vm 2023-09-23T09:29:39.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2835 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836_vm 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2837 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 23 09:29 2838 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2838_fsm 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2838_vm 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2839 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840_fsm 2023-09-23T09:29:39.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2840_vm 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2841 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995_vm 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2996 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3079_fsm 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079_vm 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3080 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3081 2023-09-23T09:29:39.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3085 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118_vm 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3119 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3164 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256_vm 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3257 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3258 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350_vm 2023-09-23T09:29:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3351 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3379 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3380 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381_vm 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3394 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3394_fsm 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3394_vm 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3395 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439 2023-09-23T09:29:39.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439_vm 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3440 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 3455 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3456 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3456_fsm 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3456_vm 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466_vm 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3467 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3468 2023-09-23T09:29:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501_vm 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3502 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3503 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3534 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3541_fsm 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541_vm 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3542 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3574 2023-09-23T09:29:39.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3575 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576_vm 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596_vm 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3597 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598_vm 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3599 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3600_fsm 2023-09-23T09:29:39.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600_vm 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3601_fsm 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601_vm 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3602_fsm 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602_vm 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3603 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3603_fsm 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3603_vm 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3604 2023-09-23T09:29:39.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3605 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3606 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3607 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3608 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 3609 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3712 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3764_fsm 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764_vm 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3766 2023-09-23T09:29:39.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3767 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3997 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 5002 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 548 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 549 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102_vm 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104_vm 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106 2023-09-23T09:29:39.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106_vm 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6110 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6111 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6112 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6113 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6117 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826_vm 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 827 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 828 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:29:39.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:total 7201 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 . 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 .. 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 112 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 113 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 23 09:29 12182 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12182_fsm 2023-09-23T09:29:39.720 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12182_vm 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12184 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12186 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12187_fsm 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187_vm 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12189 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12191 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12192_fsm 2023-09-23T09:29:39.721 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192_vm 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12194 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12196 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12197_fsm 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197_vm 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12199 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12201 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12202_fsm 2023-09-23T09:29:39.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202_vm 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12204 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12206 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12207_fsm 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207_vm 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12209 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12211 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12212 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12214 2023-09-23T09:29:39.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12216 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 23 09:29 1247 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1247_fsm 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1247_vm 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 23 09:29 1249 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1249_fsm 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1249_vm 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 23 09:29 1255 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1255_fsm 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1255_vm 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 23 09:29 1259 2023-09-23T09:29:39.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1259_fsm 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1259_vm 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417_vm 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418_vm 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 174 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 175 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2187 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224 2023-09-23T09:29:39.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224_vm 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328_vm 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336_vm 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2337 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2600 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2600_fsm 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2600_vm 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601 2023-09-23T09:29:39.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2601_fsm 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601_vm 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2602 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2602_fsm 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2602_vm 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2603 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2603_fsm 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2603_vm 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604_vm 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2605 2023-09-23T09:29:39.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2605_fsm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2605_vm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2606_fsm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606_vm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607_fsm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2607_vm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 23 09:29 2608 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2608_fsm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2608_vm 2023-09-23T09:29:39.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 23 09:29 2609 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2609_fsm 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2609_vm 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610_fsm 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2610_vm 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611_vm 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2612_fsm 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612_vm 2023-09-23T09:29:39.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613_vm 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2615_fsm 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615_vm 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616_fsm 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2616_vm 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 23 09:29 2617 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2617_fsm 2023-09-23T09:29:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2617_vm 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 23 09:29 2618 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2618_fsm 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2618_vm 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 23 09:29 2619 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2619_fsm 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2619_vm 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620_vm 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2650 2023-09-23T09:29:39.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2651 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2652 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2653 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2654 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2655 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2656 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2657 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 23 09:29 2658 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2659 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2660 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2661 2023-09-23T09:29:39.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2662 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2663 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2664 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2665 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2666 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2667 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2668 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2669 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2670 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Sep 23 09:29 2673 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 23 09:29 2674 2023-09-23T09:29:39.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 23 09:29 2675 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2678 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2679 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2680 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2681 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2682 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2683 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2684 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2685 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2686 2023-09-23T09:29:39.734 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2687 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2688 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2689 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2690 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 23 09:29 2691 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2692 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2693 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2696 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2699 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2701 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2702 2023-09-23T09:29:39.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2703 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2704 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2753 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2753_fsm 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2753_vm 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2754 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2755 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2756 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2757 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830 2023-09-23T09:29:39.736 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830_vm 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2831 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832_vm 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2833 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834_vm 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2835 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836_vm 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2837 2023-09-23T09:29:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 23 09:29 2838 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2838_fsm 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2838_vm 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2839 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840_fsm 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2840_vm 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2841 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995_vm 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2996 2023-09-23T09:29:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3079_fsm 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079_vm 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3080 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3081 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3085 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118_vm 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3119 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3164 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256 2023-09-23T09:29:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256_vm 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3257 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3258 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350_vm 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3351 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3379 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3380 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381_vm 2023-09-23T09:29:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3394 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3394_fsm 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3394_vm 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3395 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439_vm 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3440 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 3455 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3456 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3456_fsm 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3456_vm 2023-09-23T09:29:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466_vm 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3467 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3468 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501_vm 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3502 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3503 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3534 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3541_fsm 2023-09-23T09:29:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541_vm 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3542 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3574 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3575 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576_vm 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596_vm 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3597 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598_vm 2023-09-23T09:29:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3599 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3600_fsm 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600_vm 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3601_fsm 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601_vm 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3602_fsm 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602_vm 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3603 2023-09-23T09:29:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3603_fsm 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3603_vm 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3604 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3605 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3606 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3607 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3608 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 3609 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3712 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3764_fsm 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764_vm 2023-09-23T09:29:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3766 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3767 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3997 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 5002 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 548 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 549 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102_vm 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104_vm 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106 2023-09-23T09:29:39.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106_vm 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6110 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6111 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6112 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6113 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6117 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826_vm 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 827 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 828 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:29:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/global: 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:total 553 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 23 09:29 . 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1136 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1136_fsm 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1136_vm 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 1137 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1213 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1213_fsm 2023-09-23T09:29:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1213_vm 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1214 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1214_fsm 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1214_vm 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 1232 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 1233 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1260 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1260_fsm 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1260_vm 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1261 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1261_fsm 2023-09-23T09:29:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1261_vm 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1262 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1262_fsm 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1262_vm 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2396 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2396_fsm 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2396_vm 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2397 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2671 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2672 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2676 2023-09-23T09:29:39.750 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2677 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2694 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2695 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2697 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2698 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2846 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2846_vm 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2847 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2964 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2964_vm 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2965 2023-09-23T09:29:39.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2966 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2966_vm 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2967 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3592 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3592_vm 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3593 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 4060 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 4060_vm 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 4061 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6000 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6000_vm 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6001 2023-09-23T09:29:39.752 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6002 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6100 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6100_vm 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6114 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6115 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 pg_control 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/log: 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.753 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:total 1 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 . 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 mappings 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 23 09:29 replorigin_checkpoint 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 snapshots 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.755 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 .. 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 .. 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 . 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 members 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 offsets 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 .. 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:29:39.757 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 .. 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.758 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.759 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-23T09:29:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-23T09:29:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:total 16384 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 . 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:29 000000010000000000000001 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 archive_status 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-23T09:29:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 .. 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 23 09:29 .. 2023-09-23T09:29:39.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:29:39.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-09-23T09:29:39.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2023-09-23T09:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-09-23T09:29:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-09-23T09:29:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-09-23T09:29:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-09-23T09:29:39.959 INFO:teuthology.orchestra.run.smithi045.stderr:creating tables... 2023-09-23T09:29:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.86 s) 2023-09-23T09:29:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.24 s, remaining 3.58 s) 2023-09-23T09:29:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.33 s, remaining 3.23 s) 2023-09-23T09:29:40.457 INFO:teuthology.orchestra.run.smithi045.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.50 s, remaining 3.47 s) 2023-09-23T09:29:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.68 s, remaining 3.65 s) 2023-09-23T09:29:40.727 INFO:teuthology.orchestra.run.smithi045.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.77 s, remaining 3.32 s) 2023-09-23T09:29:40.891 INFO:teuthology.orchestra.run.smithi045.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.93 s, remaining 3.32 s) 2023-09-23T09:29:41.061 INFO:teuthology.orchestra.run.smithi045.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.10 s, remaining 3.30 s) 2023-09-23T09:29:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.21 s, remaining 3.08 s) 2023-09-23T09:29:41.309 INFO:teuthology.orchestra.run.smithi045.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.35 s, remaining 2.96 s) 2023-09-23T09:29:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 -- 172.21.15.45:0/1115504480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594106ea0 msgr2=0x7fa594107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 --2- 172.21.15.45:0/1115504480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594106ea0 0x7fa594107280 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa584009990 tx=0x7fa584009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 -- 172.21.15.45:0/1115504480 shutdown_connections 2023-09-23T09:29:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 --2- 172.21.15.45:0/1115504480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa594069750 0x7fa59410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 --2- 172.21.15.45:0/1115504480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594106ea0 0x7fa594107280 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 --2- 172.21.15.45:0/1115504480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa594068d90 0x7fa594069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 -- 172.21.15.45:0/1115504480 >> 172.21.15.45:0/1115504480 conn(0x7fa594077bf0 msgr2=0x7fa594078000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 -- 172.21.15.45:0/1115504480 shutdown_connections 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.411+0000 7fa5998f1700 1 -- 172.21.15.45:0/1115504480 wait complete. 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 Processor -- start 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 -- start start 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594068d90 0x7fa594196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa594069750 0x7fa5941973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa594106ea0 0x7fa59419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59419c070 con 0x7fa594068d90 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59419c1e0 con 0x7fa594106ea0 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa5998f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59419c3a0 con 0x7fa594069750 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594068d90 0x7fa594196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594068d90 0x7fa594196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60968/0 (socket says 172.21.15.45:60968) 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.412+0000 7fa592ffd700 1 -- 172.21.15.45:0/3192816553 learned_addr learned my addr 172.21.15.45:0/3192816553 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa594106ea0 0x7fa59419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa5927fc700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa594069750 0x7fa5941973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa592ffd700 1 -- 172.21.15.45:0/3192816553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa594069750 msgr2=0x7fa5941973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa594069750 0x7fa5941973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa592ffd700 1 -- 172.21.15.45:0/3192816553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa594106ea0 msgr2=0x7fa59419ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa594106ea0 0x7fa59419ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa592ffd700 1 -- 172.21.15.45:0/3192816553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840095f0 con 0x7fa594068d90 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594068d90 0x7fa594196e80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098e0 tx=0x7fa58400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa5988ef700 1 -- 172.21.15.45:0/3192816553 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58401d070 con 0x7fa594068d90 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.413+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59419c620 con 0x7fa594068d90 2023-09-23T09:29:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.414+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941a3ec0 con 0x7fa594068d90 2023-09-23T09:29:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.414+0000 7fa5988ef700 1 -- 172.21.15.45:0/3192816553 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa584005510 con 0x7fa594068d90 2023-09-23T09:29:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.414+0000 7fa5988ef700 1 -- 172.21.15.45:0/3192816553 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584004cd0 con 0x7fa594068d90 2023-09-23T09:29:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.417+0000 7fa5988ef700 1 -- 172.21.15.45:0/3192816553 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7fa58400fc10 con 0x7fa594068d90 2023-09-23T09:29:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.417+0000 7fa5a5adbb80 1 Processor -- start 2023-09-23T09:29:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.417+0000 7fa5988ef700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa57c0a1820 0x7fa57c0a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5a5adbb80 1 -- start start 2023-09-23T09:29:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5988ef700 1 -- 172.21.15.45:0/3192816553 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa5840ce2e0 con 0x7fa594068d90 2023-09-23T09:29:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5927fc700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa57c0a1820 0x7fa57c0a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d715a400 con 0x5560d71f0f40 2023-09-23T09:29:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d714bbc0 con 0x5560d71fc710 2023-09-23T09:29:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d71494e0 con 0x5560d71f1860 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5937fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5937fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54096/0 (socket says 172.21.15.45:54096) 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5937fe700 1 -- 172.21.15.45:0/2623998152 learned_addr learned my addr 172.21.15.45:0/2623998152 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5927fc700 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa592ffd700 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.418+0000 7fa5927fc700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa57c0a1820 0x7fa57c0a3ce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa588005950 tx=0x7fa588009ee0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa5937fe700 1 -- 172.21.15.45:0/2623998152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 msgr2=0x5560d71fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa5937fe700 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa5937fe700 1 -- 172.21.15.45:0/2623998152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 msgr2=0x5560d71f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa5937fe700 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa5937fe700 1 -- 172.21.15.45:0/2623998152 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560d71a7870 con 0x5560d71fc710 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa592ffd700 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa5937fe700 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71feb00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0084c0 tx=0x7fa58c0087d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7e0487f8bfa8a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa579ffb700 1 -- 172.21.15.45:0/2623998152 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c003c00 con 0x5560d71fc710 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa579ffb700 1 -- 172.21.15.45:0/2623998152 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58c010040 con 0x5560d71fc710 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.419+0000 7fa579ffb700 1 -- 172.21.15.45:0/2623998152 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c012760 con 0x5560d71fc710 2023-09-23T09:29:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2623998152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 msgr2=0x5560d71feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71feb00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0084c0 tx=0x7fa58c0087d0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2623998152 shutdown_connections 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71feb00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2623998152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2623998152 >> 172.21.15.45:0/2623998152 conn(0x5560d7166200 msgr2=0x5560d71ed3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2623998152 shutdown_connections 2023-09-23T09:29:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.420+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2623998152 wait complete. 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 -- start start 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 Processor -- start 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71fc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d7144dd0 con 0x5560d71f0f40 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d70e16b0 con 0x5560d71fc710 2023-09-23T09:29:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d70e1350 con 0x5560d71f1860 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.421+0000 7fa5a5adbb80 10 client.0 _ll_register_callbacks cb 0x5560d6f3e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71fc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60982/0 (socket says 172.21.15.45:60982) 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 -- 172.21.15.45:0/3226931902 learned_addr learned my addr 172.21.15.45:0/3226931902 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa5927fc700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 -- 172.21.15.45:0/3226931902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 msgr2=0x5560d71f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 -- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 msgr2=0x5560d71f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 -- 172.21.15.45:0/3226931902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c007d00 con 0x5560d71f0f40 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71fc070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa584009350 tx=0x7fa5840cbef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9629a18c35b5fdb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa572ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.422+0000 7fa572ffd700 1 -- 172.21.15.45:0/3226931902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58409a970 con 0x5560d71f0f40 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa572ffd700 1 -- 172.21.15.45:0/3226931902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa584004e30 con 0x5560d71f0f40 2023-09-23T09:29:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560d71e5090 con 0x5560d71f0f40 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa572ffd700 1 -- 172.21.15.45:0/3226931902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa58409b980 con 0x5560d71f0f40 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 10 client.15495 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5560d6f6fd80 con 0x5560d71f0f40 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa572ffd700 1 -- 172.21.15.45:0/3226931902 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560d71e5090 con 0x5560d71f0f40 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa572ffd700 1 -- 172.21.15.45:0/3226931902 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5840b9310 con 0x5560d71f0f40 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 20 client.15495 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7210350 0x5560d7212770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 4 client.15495 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.423+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560d713ddb0 con 0x5560d7210350 2023-09-23T09:29:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.424+0000 7fa5927fc700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7210350 0x5560d7212770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.424+0000 7fa572ffd700 10 client.15495 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.425+0000 7fa5927fc700 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7210350 0x5560d7212770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9ad7663efedaec server_cookie=140f196f06617d50 in_seq=0 out_seq=0 2023-09-23T09:29:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.425+0000 7fa572ffd700 1 -- 172.21.15.45:0/3226931902 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5560d713ddb0 con 0x5560d7210350 2023-09-23T09:29:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.425+0000 7fa572ffd700 10 client.15495 handle_command_reply: tid=0 2023-09-23T09:29:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.425+0000 7fa5a5adbb80 1 client.15495 shutdown 2023-09-23T09:29:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.425+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 msgr2=0x5560d71fc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71fc070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa584009350 tx=0x7fa5840cbef0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 shutdown_connections 2023-09-23T09:29:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7210350 0x5560d7212770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71fc710 0x5560d71f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f0f40 0x5560d71fc070 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa572ffd700 0 client.15495 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/3226931902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 >> 172.21.15.45:0/3226931902 conn(0x5560d7166200 msgr2=0x5560d71ecfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 shutdown_connections 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/3226931902 wait complete. 2023-09-23T09:29:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.426+0000 7fa5a5adbb80 20 client.15495 trim_cache size 0 max 16384 2023-09-23T09:29:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 Processor -- start 2023-09-23T09:29:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 -- start start 2023-09-23T09:29:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d72032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71fc710 0x5560d71f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d6fc58c0 con 0x5560d71fc710 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d71f2160 con 0x5560d71f0f40 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d71f2a50 con 0x5560d71f1860 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71fc710 0x5560d71f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71fc710 0x5560d71f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60994/0 (socket says 172.21.15.45:60994) 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5937fe700 1 -- 172.21.15.45:0/2182629623 learned_addr learned my addr 172.21.15.45:0/2182629623 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa592ffd700 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d72032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5927fc700 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5937fe700 1 -- 172.21.15.45:0/2182629623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 msgr2=0x5560d71fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.444+0000 7fa5937fe700 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa5937fe700 1 -- 172.21.15.45:0/2182629623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 msgr2=0x5560d72032b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa5937fe700 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d72032b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa5937fe700 1 -- 172.21.15.45:0/2182629623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560d6f58c00 con 0x5560d71fc710 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa592ffd700 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d72032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa5927fc700 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa5937fe700 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71fc710 0x5560d71f8af0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa58c000f80 tx=0x7fa58c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f2ef729ec49bfd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa57a7fc700 1 -- 172.21.15.45:0/2182629623 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c007790 con 0x5560d71fc710 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa57a7fc700 1 -- 172.21.15.45:0/2182629623 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58c012cd0 con 0x5560d71fc710 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.445+0000 7fa57a7fc700 1 -- 172.21.15.45:0/2182629623 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c01fcb0 con 0x5560d71fc710 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2182629623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71fc710 msgr2=0x5560d71f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71fc710 0x5560d71f8af0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa58c000f80 tx=0x7fa58c006000 comp rx=0 tx=0).stop 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2182629623 shutdown_connections 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71fc710 0x5560d71f8af0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d72032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/2182629623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71f1860 0x5560d71fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2182629623 >> 172.21.15.45:0/2182629623 conn(0x5560d7166200 msgr2=0x5560d71f4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2182629623 shutdown_connections 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/2182629623 wait complete. 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 -- start start 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 Processor -- start 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d71f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f1860 0x5560d71f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.446+0000 7fa5a5adbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71fc710 0x5560d71f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d71f5af0 con 0x5560d71f1860 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d71f5c60 con 0x5560d71f0f40 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5a5adbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d71f5e20 con 0x5560d71fc710 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5a5adbb80 10 client.0 _ll_register_callbacks cb 0x7fa58409b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f1860 0x5560d71f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f1860 0x5560d71f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60998/0 (socket says 172.21.15.45:60998) 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 -- 172.21.15.45:0/542288886 learned_addr learned my addr 172.21.15.45:0/542288886 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa592ffd700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d71f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5937fe700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71fc710 0x5560d71f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 -- 172.21.15.45:0/542288886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71fc710 msgr2=0x5560d71f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71fc710 0x5560d71f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 -- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 msgr2=0x5560d71f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d71f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 -- 172.21.15.45:0/542288886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560d6f6fd80 con 0x5560d71f1860 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa592ffd700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d71f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5927fc700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f1860 0x5560d71f6ee0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x5560d71fa8a0 tx=0x7fa5880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6066ca347b28ff server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa572ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa572ffd700 1 -- 172.21.15.45:0/542288886 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588023040 con 0x5560d71f1860 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa572ffd700 1 -- 172.21.15.45:0/542288886 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa588029d80 con 0x5560d71f1860 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa58c010040 con 0x5560d71f1860 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa572ffd700 1 -- 172.21.15.45:0/542288886 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5880295a0 con 0x5560d71f1860 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.447+0000 7fa5a5adbb80 10 client.15501 fetch_fsmap learned FSMap version 22 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5560d71a7870 con 0x5560d71f1860 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa572ffd700 1 -- 172.21.15.45:0/542288886 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58803e080 con 0x5560d71f1860 2023-09-23T09:29:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa572ffd700 1 -- 172.21.15.45:0/542288886 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa58803a4a0 con 0x5560d71f1860 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa5a5adbb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 22 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa5a5adbb80 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa5a5adbb80 20 client.15501 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7265140 0x5560d7267560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa5a5adbb80 4 client.15501 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560d713dad0 con 0x5560d7265140 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa592ffd700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7265140 0x5560d7267560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa572ffd700 10 client.15501 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.448+0000 7fa592ffd700 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7265140 0x5560d7267560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9bdf82686fb867 server_cookie=33c015424067e51b in_seq=0 out_seq=0 2023-09-23T09:29:41.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:41 smithi098 ceph-mon[102386]: pgmap v10: 97 pgs: 97 active+clean; 30 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.6 MiB/s wr, 84 op/s 2023-09-23T09:29:41.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:41 smithi045 ceph-mon[98627]: pgmap v10: 97 pgs: 97 active+clean; 30 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.6 MiB/s wr, 84 op/s 2023-09-23T09:29:41.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:41 smithi123 ceph-mon[103577]: pgmap v10: 97 pgs: 97 active+clean; 30 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.6 MiB/s wr, 84 op/s 2023-09-23T09:29:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa572ffd700 1 -- 172.21.15.45:0/542288886 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560d713dad0 con 0x5560d7265140 2023-09-23T09:29:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa572ffd700 10 client.15501 handle_command_reply: tid=0 2023-09-23T09:29:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 client.15501 shutdown 2023-09-23T09:29:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f1860 msgr2=0x5560d71f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f1860 0x5560d71f6ee0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x5560d71fa8a0 tx=0x7fa5880047c0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 shutdown_connections 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5560d7265140 0x5560d7267560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d71fc710 0x5560d71f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa572ffd700 0 client.15501 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d71f0f40 0x5560d71f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 --2- 172.21.15.45:0/542288886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5560d71f1860 0x5560d71f6ee0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 >> 172.21.15.45:0/542288886 conn(0x5560d7166200 msgr2=0x5560d71f2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 shutdown_connections 2023-09-23T09:29:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.626+0000 7fa5a5adbb80 1 -- 172.21.15.45:0/542288886 wait complete. 2023-09-23T09:29:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5a5adbb80 20 client.15501 trim_cache size 0 max 16384 2023-09-23T09:29:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:29:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:29:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:29:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:29:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa57c0a1820 msgr2=0x7fa57c0a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa57c0a1820 0x7fa57c0a3ce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa588005950 tx=0x7fa588009ee0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594068d90 msgr2=0x7fa594196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594068d90 0x7fa594196e80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098e0 tx=0x7fa58400f720 comp rx=0 tx=0).stop 2023-09-23T09:29:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 shutdown_connections 2023-09-23T09:29:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa594106ea0 0x7fa59419ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa57c0a1820 0x7fa57c0a3ce0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594068d90 0x7fa594196e80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 --2- 172.21.15.45:0/3192816553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa594069750 0x7fa5941973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 >> 172.21.15.45:0/3192816553 conn(0x7fa594077bf0 msgr2=0x7fa594104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 shutdown_connections 2023-09-23T09:29:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:41.627+0000 7fa5998f1700 1 -- 172.21.15.45:0/3192816553 wait complete. 2023-09-23T09:29:41.656 INFO:teuthology.orchestra.run.smithi045.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.69 s, remaining 3.24 s) 2023-09-23T09:29:41.765 INFO:teuthology.orchestra.run.smithi045.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.80 s, remaining 3.01 s) 2023-09-23T09:29:41.923 INFO:teuthology.orchestra.run.smithi045.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.96 s, remaining 2.87 s) 2023-09-23T09:29:42.099 INFO:teuthology.orchestra.run.smithi045.stderr:1400000 of 3200000 tuples (43%) done (elapsed 2.14 s, remaining 2.75 s) 2023-09-23T09:29:42.165 INFO:teuthology.orchestra.run.smithi045.stderr:1500000 of 3200000 tuples (46%) done (elapsed 2.20 s, remaining 2.50 s) 2023-09-23T09:29:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.38 s, remaining 2.38 s) 2023-09-23T09:29:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.50 s, remaining 2.21 s) 2023-09-23T09:29:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.66 s, remaining 2.07 s) 2023-09-23T09:29:42.799 INFO:teuthology.orchestra.run.smithi045.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.84 s, remaining 1.94 s) 2023-09-23T09:29:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.94 s, remaining 1.76 s) 2023-09-23T09:29:43.054 INFO:teuthology.orchestra.run.smithi045.stderr:2100000 of 3200000 tuples (65%) done (elapsed 3.09 s, remaining 1.62 s) 2023-09-23T09:29:43.226 INFO:teuthology.orchestra.run.smithi045.stderr:2200000 of 3200000 tuples (68%) done (elapsed 3.27 s, remaining 1.48 s) 2023-09-23T09:29:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2300000 of 3200000 tuples (71%) done (elapsed 3.44 s, remaining 1.35 s) 2023-09-23T09:29:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.52 s, remaining 1.17 s) 2023-09-23T09:29:43.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:43 smithi098 ceph-mon[102386]: pgmap v11: 97 pgs: 97 active+clean; 105 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 15 MiB/s wr, 246 op/s 2023-09-23T09:29:43.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:43 smithi045 ceph-mon[98627]: pgmap v11: 97 pgs: 97 active+clean; 105 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 15 MiB/s wr, 246 op/s 2023-09-23T09:29:43.612 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:43 smithi123 ceph-mon[103577]: pgmap v11: 97 pgs: 97 active+clean; 105 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 15 MiB/s wr, 246 op/s 2023-09-23T09:29:43.660 INFO:teuthology.orchestra.run.smithi045.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.70 s, remaining 1.04 s) 2023-09-23T09:29:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.81 s, remaining 0.88 s) 2023-09-23T09:29:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.96 s, remaining 0.73 s) 2023-09-23T09:29:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2800000 of 3200000 tuples (87%) done (elapsed 4.14 s, remaining 0.59 s) 2023-09-23T09:29:44.171 INFO:teuthology.orchestra.run.smithi045.stderr:2900000 of 3200000 tuples (90%) done (elapsed 4.21 s, remaining 0.44 s) 2023-09-23T09:29:44.400 INFO:teuthology.orchestra.run.smithi045.stderr:3000000 of 3200000 tuples (93%) done (elapsed 4.44 s, remaining 0.30 s) 2023-09-23T09:29:44.580 INFO:teuthology.orchestra.run.smithi045.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.62 s, remaining 0.15 s) 2023-09-23T09:29:44.652 INFO:teuthology.orchestra.run.smithi045.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.69 s, remaining 0.00 s) 2023-09-23T09:29:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:vacuum... 2023-09-23T09:29:45.329 INFO:teuthology.orchestra.run.smithi045.stderr:set primary keys... 2023-09-23T09:29:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:45 smithi045 ceph-mon[98627]: pgmap v12: 97 pgs: 97 active+clean; 155 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 22 MiB/s wr, 340 op/s 2023-09-23T09:29:45.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:45 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:45.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:45 smithi123 ceph-mon[103577]: pgmap v12: 97 pgs: 97 active+clean; 155 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 22 MiB/s wr, 340 op/s 2023-09-23T09:29:45.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:45 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:45.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:45 smithi098 ceph-mon[102386]: pgmap v12: 97 pgs: 97 active+clean; 155 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 22 MiB/s wr, 340 op/s 2023-09-23T09:29:45.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:45 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:done. 2023-09-23T09:29:46.825 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-09-23T09:29:46.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:29:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:starting vacuum...end. 2023-09-23T09:29:47.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.049+0000 7f05649ed700 1 -- 172.21.15.45:0/2187820032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560108370 msgr2=0x7f056010fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.049+0000 7f05649ed700 1 --2- 172.21.15.45:0/2187820032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560108370 0x7f056010fef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0558009f90 tx=0x7f055800b330 comp rx=0 tx=0).stop 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.050+0000 7f05649ed700 1 -- 172.21.15.45:0/2187820032 shutdown_connections 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.050+0000 7f05649ed700 1 --2- 172.21.15.45:0/2187820032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560108370 0x7f056010fef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.050+0000 7f05649ed700 1 --2- 172.21.15.45:0/2187820032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560103250 0x7f0560103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.050+0000 7f05649ed700 1 --2- 172.21.15.45:0/2187820032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560103c00 0x7f0560107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.050+0000 7f05649ed700 1 -- 172.21.15.45:0/2187820032 >> 172.21.15.45:0/2187820032 conn(0x7f05600feab0 msgr2=0x7f0560100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.050+0000 7f05649ed700 1 -- 172.21.15.45:0/2187820032 shutdown_connections 2023-09-23T09:29:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.050+0000 7f05649ed700 1 -- 172.21.15.45:0/2187820032 wait complete. 2023-09-23T09:29:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 Processor -- start 2023-09-23T09:29:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 -- start start 2023-09-23T09:29:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560103250 0x7f0560075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560108370 0x7f0560075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560075db0 0x7f0560071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560071680 con 0x7f0560108370 2023-09-23T09:29:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05600717c0 con 0x7f0560075db0 2023-09-23T09:29:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f05649ed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560071930 con 0x7f0560103250 2023-09-23T09:29:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560108370 0x7f0560075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.051+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560108370 0x7f0560075870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32778/0 (socket says 172.21.15.45:32778) 2023-09-23T09:29:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055dd9b700 1 -- 172.21.15.45:0/1348737173 learned_addr learned my addr 172.21.15.45:0/1348737173 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055e59c700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560103250 0x7f0560075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055dd9b700 1 -- 172.21.15.45:0/1348737173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560103250 msgr2=0x7f0560075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560103250 0x7f0560075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055dd9b700 1 -- 172.21.15.45:0/1348737173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560075db0 msgr2=0x7f0560071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560075db0 0x7f0560071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055dd9b700 1 -- 172.21.15.45:0/1348737173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0558009c40 con 0x7f0560108370 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560108370 0x7f0560075870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9f0 tx=0x7f055400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.052+0000 7f054f7fe700 1 -- 172.21.15.45:0/1348737173 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554009690 con 0x7f0560108370 2023-09-23T09:29:47.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.053+0000 7f054f7fe700 1 -- 172.21.15.45:0/1348737173 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f055400ce80 con 0x7f0560108370 2023-09-23T09:29:47.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.053+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0560071c10 con 0x7f0560108370 2023-09-23T09:29:47.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.053+0000 7f054f7fe700 1 -- 172.21.15.45:0/1348737173 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554018420 con 0x7f0560108370 2023-09-23T09:29:47.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.053+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0560072170 con 0x7f0560108370 2023-09-23T09:29:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.058+0000 7f0570bd7b80 1 Processor -- start 2023-09-23T09:29:47.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.058+0000 7f054f7fe700 1 -- 172.21.15.45:0/1348737173 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7f055400c9c0 con 0x7f0560108370 2023-09-23T09:29:47.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.059+0000 7f054f7fe700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05480a18e0 0x7f05480a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.059+0000 7f054f7fe700 1 -- 172.21.15.45:0/1348737173 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0554014070 con 0x7f0560108370 2023-09-23T09:29:47.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f0570bd7b80 1 -- start start 2023-09-23T09:29:47.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185400 0x55b0fa1857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa190690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa192fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa0ee460 con 0x55b0fa185d20 2023-09-23T09:29:47.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa0dfc20 con 0x55b0fa185400 2023-09-23T09:29:47.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa0dd540 con 0x55b0fa190bd0 2023-09-23T09:29:47.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.060+0000 7f055ed9d700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05480a18e0 0x7f05480a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa190690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185400 0x55b0fa1857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa190690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32786/0 (socket says 172.21.15.45:32786) 2023-09-23T09:29:47.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055dd9b700 1 -- 172.21.15.45:0/2232584876 learned_addr learned my addr 172.21.15.45:0/2232584876 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055ed9d700 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa192fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055e59c700 1 -- 172.21.15.45:0/2232584876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 msgr2=0x55b0fa192fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055e59c700 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa192fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055e59c700 1 -- 172.21.15.45:0/2232584876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 msgr2=0x55b0fa190690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055e59c700 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa190690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055e59c700 1 -- 172.21.15.45:0/2232584876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0fa13b8d0 con 0x55b0fa185400 2023-09-23T09:29:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055dd9b700 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa190690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.061+0000 7f055ed9d700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05480a18e0 0x7f05480a3da0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0558006070 tx=0x7f0558008d10 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.062+0000 7f055e59c700 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185400 0x55b0fa1857e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0550009cb0 tx=0x7f0550011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4805c04943104093 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.062+0000 7f054cff9700 1 -- 172.21.15.45:0/2232584876 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550011d20 con 0x55b0fa185400 2023-09-23T09:29:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.062+0000 7f054cff9700 1 -- 172.21.15.45:0/2232584876 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0550011e80 con 0x55b0fa185400 2023-09-23T09:29:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.062+0000 7f054cff9700 1 -- 172.21.15.45:0/2232584876 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500193e0 con 0x55b0fa185400 2023-09-23T09:29:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.063+0000 7f0570bd7b80 1 -- 172.21.15.45:0/2232584876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185400 msgr2=0x55b0fa1857e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.063+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185400 0x55b0fa1857e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0550009cb0 tx=0x7f0550011440 comp rx=0 tx=0).stop 2023-09-23T09:29:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 -- 172.21.15.45:0/2232584876 shutdown_connections 2023-09-23T09:29:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa192fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185400 0x55b0fa1857e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/2232584876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa190690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 -- 172.21.15.45:0/2232584876 >> 172.21.15.45:0/2232584876 conn(0x55b0fa0fa260 msgr2=0x55b0fa181790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 -- 172.21.15.45:0/2232584876 shutdown_connections 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 -- 172.21.15.45:0/2232584876 wait complete. 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 -- start start 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.064+0000 7f0570bd7b80 1 Processor -- start 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa190680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185d20 0x55b0fa18d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 0x55b0fa18e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa0d8e30 con 0x55b0fa185400 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa075710 con 0x55b0fa190bd0 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa0753b0 con 0x55b0fa185d20 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f0570bd7b80 10 client.0 _ll_register_callbacks cb 0x55b0f9ed2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa190680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185d20 0x55b0fa18d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa190680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32792/0 (socket says 172.21.15.45:32792) 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185d20 0x55b0fa18d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34900/0 (socket says 172.21.15.45:34900) 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055e59c700 1 -- 172.21.15.45:0/1053285838 learned_addr learned my addr 172.21.15.45:0/1053285838 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055ed9d700 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 0x55b0fa18e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055dd9b700 1 -- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 msgr2=0x55b0fa18e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 0x55b0fa18e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055dd9b700 1 -- 172.21.15.45:0/1053285838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 msgr2=0x55b0fa190680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa190680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.065+0000 7f055dd9b700 1 -- 172.21.15.45:0/1053285838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0560072170 con 0x55b0fa185d20 2023-09-23T09:29:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185d20 0x55b0fa18d900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f055409a4f0 tx=0x7f055409aaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c49612539edaa9a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f0545ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f0545ffb700 1 -- 172.21.15.45:0/1053285838 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055409a560 con 0x55b0fa185d20 2023-09-23T09:29:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f0545ffb700 1 -- 172.21.15.45:0/1053285838 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f055409e6b0 con 0x55b0fa185d20 2023-09-23T09:29:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0fa0d1e10 con 0x55b0fa185d20 2023-09-23T09:29:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f0545ffb700 1 -- 172.21.15.45:0/1053285838 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05540a28f0 con 0x55b0fa185d20 2023-09-23T09:29:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f0570bd7b80 10 client.25201 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.067+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f05500098e0 con 0x55b0fa185d20 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0545ffb700 1 -- 172.21.15.45:0/1053285838 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0fa0d1e10 con 0x55b0fa185d20 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0545ffb700 1 -- 172.21.15.45:0/1053285838 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f05540cb0a0 con 0x55b0fa185d20 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0570bd7b80 10 client.25201 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0570bd7b80 10 client.25201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0570bd7b80 20 client.25201 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1a0760 0x55b0fa1a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0570bd7b80 4 client.25201 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0fa0d1b30 con 0x55b0fa1a0760 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f055e59c700 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1a0760 0x55b0fa1a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f0545ffb700 10 client.25201 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.068+0000 7f055e59c700 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1a0760 0x55b0fa1a2b80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc53604b9a7a5b1 server_cookie=167632a9fa17fbd5 in_seq=0 out_seq=0 2023-09-23T09:29:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.069+0000 7f0545ffb700 1 -- 172.21.15.45:0/1053285838 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0fa0d1b30 con 0x55b0fa1a0760 2023-09-23T09:29:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.069+0000 7f0545ffb700 10 client.25201 handle_command_reply: tid=0 2023-09-23T09:29:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.069+0000 7f0570bd7b80 1 client.25201 shutdown 2023-09-23T09:29:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.069+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185d20 msgr2=0x55b0fa18d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.069+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185d20 0x55b0fa18d900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f055409a4f0 tx=0x7f055409aaa0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 shutdown_connections 2023-09-23T09:29:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1a0760 0x55b0fa1a2b80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 0x55b0fa18e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa190680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1053285838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185d20 0x55b0fa18d900 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 >> 172.21.15.45:0/1053285838 conn(0x55b0fa0fa260 msgr2=0x55b0fa181320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0545ffb700 0 client.25201 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 shutdown_connections 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.070+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1053285838 wait complete. 2023-09-23T09:29:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.071+0000 7f0570bd7b80 20 client.25201 trim_cache size 0 max 16384 2023-09-23T09:29:47.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 Processor -- start 2023-09-23T09:29:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 -- start start 2023-09-23T09:29:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa188330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185d20 0x55b0fa188c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa1891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f9f59920 con 0x55b0fa185400 2023-09-23T09:29:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa186130 con 0x55b0fa185d20 2023-09-23T09:29:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.093+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa186c40 con 0x55b0fa190bd0 2023-09-23T09:29:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa1891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa1891b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34908/0 (socket says 172.21.15.45:34908) 2023-09-23T09:29:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 -- 172.21.15.45:0/493599662 learned_addr learned my addr 172.21.15.45:0/493599662 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 -- 172.21.15.45:0/493599662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185d20 msgr2=0x55b0fa188c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 --2- 172.21.15.45:0/493599662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185d20 0x55b0fa188c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 -- 172.21.15.45:0/493599662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 msgr2=0x55b0fa188330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 --2- 172.21.15.45:0/493599662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa188330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 -- 172.21.15.45:0/493599662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0fa17d480 con 0x55b0fa190bd0 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f055ed9d700 1 --2- 172.21.15.45:0/493599662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa1891b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0558004e00 tx=0x7f0558021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e7c4bb50ad9567 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f054d7fa700 1 -- 172.21.15.45:0/493599662 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055800e070 con 0x55b0fa190bd0 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f054d7fa700 1 -- 172.21.15.45:0/493599662 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05580078a0 con 0x55b0fa190bd0 2023-09-23T09:29:47.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.094+0000 7f054d7fa700 1 -- 172.21.15.45:0/493599662 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055801fe80 con 0x55b0fa190bd0 2023-09-23T09:29:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 -- 172.21.15.45:0/493599662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 msgr2=0x55b0fa1891b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/493599662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa1891b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0558004e00 tx=0x7f0558021000 comp rx=0 tx=0).stop 2023-09-23T09:29:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 -- 172.21.15.45:0/493599662 shutdown_connections 2023-09-23T09:29:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/493599662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa190bd0 0x55b0fa1891b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/493599662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185400 0x55b0fa188330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/493599662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa185d20 0x55b0fa188c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 -- 172.21.15.45:0/493599662 >> 172.21.15.45:0/493599662 conn(0x55b0fa0fa260 msgr2=0x55b0fa19be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 -- 172.21.15.45:0/493599662 shutdown_connections 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 -- 172.21.15.45:0/493599662 wait complete. 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 -- start start 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 Processor -- start 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.096+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185400 0x55b0fa1958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa195e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f0570bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 0x55b0fa1f3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa1f3b00 con 0x55b0fa185d20 2023-09-23T09:29:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa1f3c70 con 0x55b0fa190bd0 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f0570bd7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fa1f3e30 con 0x55b0fa185400 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f0570bd7b80 10 client.0 _ll_register_callbacks cb 0x7f055409a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185400 0x55b0fa1958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32806/0 (socket says 172.21.15.45:32806) 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055dd9b700 1 -- 172.21.15.45:0/1804791644 learned_addr learned my addr 172.21.15.45:0/1804791644 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055e59c700 1 -- 172.21.15.45:0/1804791644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 msgr2=0x55b0fa1f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055e59c700 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 0x55b0fa1f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055e59c700 1 -- 172.21.15.45:0/1804791644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 msgr2=0x55b0fa195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055e59c700 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055e59c700 1 -- 172.21.15.45:0/1804791644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0560110760 con 0x55b0fa185400 2023-09-23T09:29:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.097+0000 7f055e59c700 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185400 0x55b0fa1958f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0550011790 tx=0x7f05500037c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2b74b8bd4a7f383 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.098+0000 7f0545ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.098+0000 7f0545ffb700 1 -- 172.21.15.45:0/1804791644 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550011ba0 con 0x55b0fa185400 2023-09-23T09:29:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.098+0000 7f0545ffb700 1 -- 172.21.15.45:0/1804791644 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0550011d00 con 0x55b0fa185400 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0fa179250 con 0x55b0fa185400 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0545ffb700 1 -- 172.21.15.45:0/1804791644 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0550003a90 con 0x55b0fa185400 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 10 client.25207 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b0f9f03de0 con 0x55b0fa185400 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0545ffb700 1 -- 172.21.15.45:0/1804791644 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0fa179250 con 0x55b0fa185400 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0545ffb700 1 -- 172.21.15.45:0/1804791644 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f055001a540 con 0x55b0fa185400 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 10 client.25207 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 10 client.25207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 20 client.25207 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1f54f0 0x55b0fa1f7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 4 client.25207 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.099+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b0f9f24f70 con 0x55b0fa1f54f0 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1f54f0 0x55b0fa1f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0545ffb700 10 client.25207 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f055dd9b700 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1f54f0 0x55b0fa1f7910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d134b86d73a0bf3c server_cookie=ed15e1d0bae4bd18 in_seq=0 out_seq=0 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0545ffb700 1 -- 172.21.15.45:0/1804791644 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b0f9f24f70 con 0x55b0fa1f54f0 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0545ffb700 10 client.25207 handle_command_reply: tid=0 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 client.25207 shutdown 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185400 msgr2=0x55b0fa1958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185400 0x55b0fa1958f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0550011790 tx=0x7f05500037c0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 shutdown_connections 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b0fa1f54f0 0x55b0fa1f7910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0fa190bd0 0x55b0fa1f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0545ffb700 0 client.25207 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0fa185400 0x55b0fa1958f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 --2- 172.21.15.45:0/1804791644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0fa185d20 0x55b0fa195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 >> 172.21.15.45:0/1804791644 conn(0x55b0fa0fa260 msgr2=0x55b0fa188c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 shutdown_connections 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.100+0000 7f0570bd7b80 1 -- 172.21.15.45:0/1804791644 wait complete. 2023-09-23T09:29:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f0570bd7b80 20 client.25207 trim_cache size 0 max 16384 2023-09-23T09:29:47.107 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:47.107 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2023-09-23T09:29:47.107 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2023-09-23T09:29:47.107 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05480a18e0 msgr2=0x7f05480a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05480a18e0 0x7f05480a3da0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0558006070 tx=0x7f0558008d10 comp rx=0 tx=0).stop 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560108370 msgr2=0x7f0560075870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560108370 0x7f0560075870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9f0 tx=0x7f055400ed00 comp rx=0 tx=0).stop 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 shutdown_connections 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0560075db0 0x7f0560071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560103250 0x7f0560075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05480a18e0 0x7f05480a3da0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 --2- 172.21.15.45:0/1348737173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0560108370 0x7f0560075870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 >> 172.21.15.45:0/1348737173 conn(0x7f05600feab0 msgr2=0x7f05600ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 shutdown_connections 2023-09-23T09:29:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.101+0000 7f05649ed700 1 -- 172.21.15.45:0/1348737173 wait complete. 2023-09-23T09:29:47.118 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T09:29:47.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 -- 172.21.15.45:0/1270290274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d40ffeb0 msgr2=0x7f07d4100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/1270290274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d40ffeb0 0x7f07d4100330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009990 tx=0x7f07bc009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 -- 172.21.15.45:0/1270290274 shutdown_connections 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/1270290274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d4100870 0x7f07d410cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/1270290274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d4104d00 0x7f07d41050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/1270290274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d40ffeb0 0x7f07d4100330 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 -- 172.21.15.45:0/1270290274 >> 172.21.15.45:0/1270290274 conn(0x7f07d4077bc0 msgr2=0x7f07d4077fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 -- 172.21.15.45:0/1270290274 shutdown_connections 2023-09-23T09:29:47.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.339+0000 7f07d8fa4700 1 -- 172.21.15.45:0/1270290274 wait complete. 2023-09-23T09:29:47.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 Processor -- start 2023-09-23T09:29:47.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 -- start start 2023-09-23T09:29:47.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d40ffeb0 0x7f07d4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d4100870 0x7f07d4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d4104d00 0x7f07d41a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d41a32c0 con 0x7f07d4100870 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d41a3400 con 0x7f07d4104d00 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d8fa4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d41a3540 con 0x7f07d40ffeb0 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07cbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d4100870 0x7f07d4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07cbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d4100870 0x7f07d4198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32818/0 (socket says 172.21.15.45:32818) 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d40ffeb0 0x7f07d4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07cbfff700 1 -- 172.21.15.45:0/2696759284 learned_addr learned my addr 172.21.15.45:0/2696759284 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.340+0000 7f07d2d9d700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d4104d00 0x7f07d41a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d2d9d700 1 -- 172.21.15.45:0/2696759284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d40ffeb0 msgr2=0x7f07d4197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d2d9d700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d40ffeb0 0x7f07d4197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d2d9d700 1 -- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d4100870 msgr2=0x7f07d4198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d2d9d700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d4100870 0x7f07d4198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d2d9d700 1 -- 172.21.15.45:0/2696759284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc0095f0 con 0x7f07d4104d00 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07cbfff700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d4100870 0x7f07d4198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d259c700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d40ffeb0 0x7f07d4197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d2d9d700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d4104d00 0x7f07d41a2d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07cc007f10 tx=0x7f07cc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07cb7fe700 1 -- 172.21.15.45:0/2696759284 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc004db0 con 0x7f07d4104d00 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d41a3820 con 0x7f07d4104d00 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d41a3d30 con 0x7f07d4104d00 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07cb7fe700 1 -- 172.21.15.45:0/2696759284 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07cc00f040 con 0x7f07d4104d00 2023-09-23T09:29:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.341+0000 7f07cb7fe700 1 -- 172.21.15.45:0/2696759284 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc005830 con 0x7f07d4104d00 2023-09-23T09:29:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.344+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b4005320 con 0x7f07d4104d00 2023-09-23T09:29:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.344+0000 7f07cb7fe700 1 -- 172.21.15.45:0/2696759284 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7f07cc00db20 con 0x7f07d4104d00 2023-09-23T09:29:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.345+0000 7f07cb7fe700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07c00a1860 0x7f07c00a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.345+0000 7f07cb7fe700 1 -- 172.21.15.45:0/2696759284 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f07cc0ccc10 con 0x7f07d4104d00 2023-09-23T09:29:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.346+0000 7f07d259c700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07c00a1860 0x7f07c00a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.346+0000 7f07d259c700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07c00a1860 0x7f07c00a3d20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009fd0 tx=0x7f07c4009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.350+0000 7f07cb7fe700 1 -- 172.21.15.45:0/2696759284 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f07cc04eec0 con 0x7f07d4104d00 2023-09-23T09:29:47.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:47 smithi045 ceph-mon[98627]: pgmap v13: 97 pgs: 97 active+clean; 155 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 22 MiB/s wr, 302 op/s 2023-09-23T09:29:47.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:47 smithi123 ceph-mon[103577]: pgmap v13: 97 pgs: 97 active+clean; 155 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 22 MiB/s wr, 302 op/s 2023-09-23T09:29:47.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.724+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07b40055d0 con 0x7f07d4104d00 2023-09-23T09:29:47.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.725+0000 7f07cb7fe700 1 -- 172.21.15.45:0/2696759284 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11307 (secure 0 0 0) 0x7f07cc095d40 con 0x7f07d4104d00 2023-09-23T09:29:47.727 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:29:47.727 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:29:43.935555+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07c00a1860 msgr2=0x7f07c00a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07c00a1860 0x7f07c00a3d20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009fd0 tx=0x7f07c4009260 comp rx=0 tx=0).stop 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d4104d00 msgr2=0x7f07d41a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d4104d00 0x7f07d41a2d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07cc007f10 tx=0x7f07cc00d3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 shutdown_connections 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d4104d00 0x7f07d41a2d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d40ffeb0 0x7f07d4197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07c00a1860 0x7f07c00a3d20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 --2- 172.21.15.45:0/2696759284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07d4100870 0x7f07d4198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 >> 172.21.15.45:0/2696759284 conn(0x7f07d4077bc0 msgr2=0x7f07d40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 shutdown_connections 2023-09-23T09:29:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.728+0000 7f07d8fa4700 1 -- 172.21.15.45:0/2696759284 wait complete. 2023-09-23T09:29:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 23 2023-09-23T09:29:47.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T09:29:47.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:47 smithi098 ceph-mon[102386]: pgmap v13: 97 pgs: 97 active+clean; 155 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 22 MiB/s wr, 302 op/s 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.959+0000 7f89cb05d700 1 -- 172.21.15.45:0/1488608926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89c4106150 msgr2=0x7f89c410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.959+0000 7f89cb05d700 1 --2- 172.21.15.45:0/1488608926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89c4106150 0x7f89c410dcd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89b40098f0 tx=0x7f89b4009c00 comp rx=0 tx=0).stop 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.960+0000 7f89cb05d700 1 -- 172.21.15.45:0/1488608926 shutdown_connections 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.960+0000 7f89cb05d700 1 --2- 172.21.15.45:0/1488608926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89c4106150 0x7f89c410dcd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.960+0000 7f89cb05d700 1 --2- 172.21.15.45:0/1488608926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4101030 0x7f89c4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.960+0000 7f89cb05d700 1 --2- 172.21.15.45:0/1488608926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c41019e0 0x7f89c4105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.960+0000 7f89cb05d700 1 -- 172.21.15.45:0/1488608926 >> 172.21.15.45:0/1488608926 conn(0x7f89c40fc8d0 msgr2=0x7f89c40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.960+0000 7f89cb05d700 1 -- 172.21.15.45:0/1488608926 shutdown_connections 2023-09-23T09:29:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.960+0000 7f89cb05d700 1 -- 172.21.15.45:0/1488608926 wait complete. 2023-09-23T09:29:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.961+0000 7f89cb05d700 1 Processor -- start 2023-09-23T09:29:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.962+0000 7f89cb05d700 1 -- start start 2023-09-23T09:29:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.962+0000 7f89cb05d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4101030 0x7f89c4198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.962+0000 7f89cb05d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c4106150 0x7f89c41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.962+0000 7f89cb05d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89c419d950 0x7f89c419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.962+0000 7f89cb05d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c419e3d0 con 0x7f89c4106150 2023-09-23T09:29:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.962+0000 7f89cb05d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c419e540 con 0x7f89c419d950 2023-09-23T09:29:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.962+0000 7f89cb05d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c419e700 con 0x7f89c4101030 2023-09-23T09:29:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c4106150 0x7f89c41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c4106150 0x7f89c41994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32834/0 (socket says 172.21.15.45:32834) 2023-09-23T09:29:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 -- 172.21.15.45:0/3428627633 learned_addr learned my addr 172.21.15.45:0/3428627633 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 -- 172.21.15.45:0/3428627633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4101030 msgr2=0x7f89c4198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4101030 0x7f89c4198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 -- 172.21.15.45:0/3428627633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89c419d950 msgr2=0x7f89c419ddd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89c419d950 0x7f89c419ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 -- 172.21.15.45:0/3428627633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0095f0 con 0x7f89c4106150 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.963+0000 7f89c3fff700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c4106150 0x7f89c41994f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00ec70 tx=0x7f89ac00c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.964+0000 7f89c1ffb700 1 -- 172.21.15.45:0/3428627633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0099d0 con 0x7f89c4106150 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.964+0000 7f89c1ffb700 1 -- 172.21.15.45:0/3428627633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ac0044e0 con 0x7f89c4106150 2023-09-23T09:29:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.964+0000 7f89c1ffb700 1 -- 172.21.15.45:0/3428627633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac010430 con 0x7f89c4106150 2023-09-23T09:29:47.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.965+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b4009520 con 0x7f89c4106150 2023-09-23T09:29:47.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.965+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c41a5f60 con 0x7f89c4106150 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 Processor -- start 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 -- start start 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d0771717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171d20 0x55d07717c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d07717cbd0 0x55d07717efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0770da460 con 0x55d077171d20 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0770cbc20 con 0x55d077171400 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89d7247b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0770c9540 con 0x55d07717cbd0 2023-09-23T09:29:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89c8df9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171d20 0x55d07717c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89c8df9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171d20 0x55d07717c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32838/0 (socket says 172.21.15.45:32838) 2023-09-23T09:29:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89c8df9700 1 -- 172.21.15.45:0/3955035447 learned_addr learned my addr 172.21.15.45:0/3955035447 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.970+0000 7f89c95fa700 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d0771717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89c8df9700 1 -- 172.21.15.45:0/3955035447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d07717cbd0 msgr2=0x55d07717efc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:29:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89c8df9700 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d07717cbd0 0x55d07717efc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89c8df9700 1 -- 172.21.15.45:0/3955035447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 msgr2=0x55d0771717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89c8df9700 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d0771717e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89c8df9700 1 -- 172.21.15.45:0/3955035447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0771278d0 con 0x55d077171d20 2023-09-23T09:29:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89c1ffb700 1 -- 172.21.15.45:0/3428627633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7f89ac003680 con 0x7f89c4106150 2023-09-23T09:29:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89c8df9700 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171d20 0x55d07717c690 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00d1e0 tx=0x7f89bc00d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774beb9dd585b849 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89baffd700 1 -- 172.21.15.45:0/3955035447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc017070 con 0x55d077171d20 2023-09-23T09:29:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89baffd700 1 -- 172.21.15.45:0/3955035447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89bc008bb0 con 0x55d077171d20 2023-09-23T09:29:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.971+0000 7f89baffd700 1 -- 172.21.15.45:0/3955035447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc013420 con 0x55d077171d20 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.972+0000 7f89c1ffb700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f89b00a18e0 0x7f89b00a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.972+0000 7f89c1ffb700 1 -- 172.21.15.45:0/3428627633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f89ac014070 con 0x7f89c4106150 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.972+0000 7f89d7247b80 1 -- 172.21.15.45:0/3955035447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171d20 msgr2=0x55d07717c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.972+0000 7f89d7247b80 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171d20 0x55d07717c690 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00d1e0 tx=0x7f89bc00d4f0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 -- 172.21.15.45:0/3955035447 shutdown_connections 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d07717cbd0 0x55d07717efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d0771717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 --2- 172.21.15.45:0/3955035447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171d20 0x55d07717c690 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 -- 172.21.15.45:0/3955035447 >> 172.21.15.45:0/3955035447 conn(0x55d0770e6260 msgr2=0x55d07716d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 -- 172.21.15.45:0/3955035447 shutdown_connections 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89c95fa700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f89b00a18e0 0x7f89b00a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 -- 172.21.15.45:0/3955035447 wait complete. 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89d7247b80 1 -- start start 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.973+0000 7f89c95fa700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f89b00a18e0 0x7f89b00a3da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89b400c010 tx=0x7f89b4019040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 1 Processor -- start 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171400 0x55d07717c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171d20 0x55d077179820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077179f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0770c4e30 con 0x55d07717cbd0 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d077061710 con 0x55d077171d20 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0770613b0 con 0x55d077171400 2023-09-23T09:29:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89d7247b80 10 client.0 _ll_register_callbacks cb 0x55d076ebe800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89c95fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077179f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89c95fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077179f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32846/0 (socket says 172.21.15.45:32846) 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89c95fa700 1 -- 172.21.15.45:0/286747286 learned_addr learned my addr 172.21.15.45:0/286747286 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89c3fff700 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171d20 0x55d077179820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.974+0000 7f89c8df9700 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171400 0x55d07717c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.975+0000 7f89c95fa700 1 -- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171400 msgr2=0x55d07717c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.975+0000 7f89c95fa700 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171400 0x55d07717c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.975+0000 7f89c95fa700 1 -- 172.21.15.45:0/286747286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171d20 msgr2=0x55d077179820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.975+0000 7f89c95fa700 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171d20 0x55d077179820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.975+0000 7f89c95fa700 1 -- 172.21.15.45:0/286747286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc008d30 con 0x55d07717cbd0 2023-09-23T09:29:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.976+0000 7f89c95fa700 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077179f60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f89b400b980 tx=0x7f89b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23efcefc1a3d7101 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.976+0000 7f89a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.976+0000 7f89a3fff700 1 -- 172.21.15.45:0/286747286 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4029070 con 0x55d07717cbd0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.976+0000 7f89a3fff700 1 -- 172.21.15.45:0/286747286 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f89b4005250 con 0x55d07717cbd0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.977+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0770bde10 con 0x55d07717cbd0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.977+0000 7f89a3fff700 1 -- 172.21.15.45:0/286747286 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b401c040 con 0x55d07717cbd0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.977+0000 7f89d7247b80 10 client.15534 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.977+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d076eefde0 con 0x55d07717cbd0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89a3fff700 1 -- 172.21.15.45:0/286747286 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0770bde10 con 0x55d07717cbd0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89a3fff700 1 -- 172.21.15.45:0/286747286 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f89b4035070 con 0x55d07717cbd0 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89d7247b80 10 client.15534 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89d7247b80 10 client.15534 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89d7247b80 20 client.15534 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89d7247b80 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d07718c570 0x55d07718e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:47.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89d7247b80 4 client.15534 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:47.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0770bdb30 con 0x55d07718c570 2023-09-23T09:29:47.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.978+0000 7f89c8df9700 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d07718c570 0x55d07718e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.980+0000 7f89c8df9700 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d07718c570 0x55d07718e990 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47d8c163af9f218e server_cookie=73e1dfc994d08bdb in_seq=0 out_seq=0 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.980+0000 7f89a3fff700 10 client.15534 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.980+0000 7f89a3fff700 1 -- 172.21.15.45:0/286747286 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d0770bdb30 con 0x55d07718c570 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.980+0000 7f89a3fff700 10 client.15534 handle_command_reply: tid=0 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.980+0000 7f89d7247b80 1 client.15534 shutdown 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 msgr2=0x55d077179f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077179f60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f89b400b980 tx=0x7f89b4009260 comp rx=0 tx=0).stop 2023-09-23T09:29:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 shutdown_connections 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077179f60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d07718c570 0x55d07718e990 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171400 0x55d07717c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 --2- 172.21.15.45:0/286747286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171d20 0x55d077179820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89a3fff700 0 client.15534 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 >> 172.21.15.45:0/286747286 conn(0x55d0770e6260 msgr2=0x55d07716d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 shutdown_connections 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 1 -- 172.21.15.45:0/286747286 wait complete. 2023-09-23T09:29:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:47.981+0000 7f89d7247b80 20 client.15534 trim_cache size 0 max 16384 2023-09-23T09:29:48.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.004+0000 7f89d7247b80 1 Processor -- start 2023-09-23T09:29:48.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.004+0000 7f89d7247b80 1 -- start start 2023-09-23T09:29:48.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d077183750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d07717c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077174b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89d7247b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d076f45920 con 0x55d07717cbd0 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89d7247b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d077183cf0 con 0x55d077171400 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89d7247b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d077175730 con 0x55d077171d20 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c95fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077174b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c8df9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d077183750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c95fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077174b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32854/0 (socket says 172.21.15.45:32854) 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c95fa700 1 -- 172.21.15.45:0/2297774891 learned_addr learned my addr 172.21.15.45:0/2297774891 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c3fff700 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d07717c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c8df9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d077183750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54202/0 (socket says 172.21.15.45:54202) 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c8df9700 1 -- 172.21.15.45:0/2297774891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 msgr2=0x55d07717c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c8df9700 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d07717c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c8df9700 1 -- 172.21.15.45:0/2297774891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 msgr2=0x55d077174b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c8df9700 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077174b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c8df9700 1 -- 172.21.15.45:0/2297774891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d076ed8c60 con 0x55d077171400 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c3fff700 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d07717c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.005+0000 7f89c95fa700 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077174b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.006+0000 7f89c8df9700 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d077183750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89bc004b10 tx=0x7f89bc005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6823c98d6a7d5e78 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.006+0000 7f89bb7fe700 1 -- 172.21.15.45:0/2297774891 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc024070 con 0x55d077171400 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.006+0000 7f89bb7fe700 1 -- 172.21.15.45:0/2297774891 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89bc003a00 con 0x55d077171400 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.006+0000 7f89bb7fe700 1 -- 172.21.15.45:0/2297774891 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc017040 con 0x55d077171400 2023-09-23T09:29:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.007+0000 7f89d7247b80 1 -- 172.21.15.45:0/2297774891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 msgr2=0x55d077183750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.007+0000 7f89d7247b80 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d077183750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89bc004b10 tx=0x7f89bc005680 comp rx=0 tx=0).stop 2023-09-23T09:29:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.008+0000 7f89d7247b80 1 -- 172.21.15.45:0/2297774891 shutdown_connections 2023-09-23T09:29:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.008+0000 7f89d7247b80 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d07717cbd0 0x55d077174b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.008+0000 7f89d7247b80 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d077171400 0x55d077183750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.008+0000 7f89d7247b80 1 --2- 172.21.15.45:0/2297774891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d07717c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.008+0000 7f89d7247b80 1 -- 172.21.15.45:0/2297774891 >> 172.21.15.45:0/2297774891 conn(0x55d0770e6260 msgr2=0x55d077187ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.008+0000 7f89d7247b80 1 -- 172.21.15.45:0/2297774891 shutdown_connections 2023-09-23T09:29:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.009+0000 7f89d7247b80 1 -- 172.21.15.45:0/2297774891 wait complete. 2023-09-23T09:29:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.009+0000 7f89d7247b80 1 -- start start 2023-09-23T09:29:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.009+0000 7f89d7247b80 1 Processor -- start 2023-09-23T09:29:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171400 0x55d077181920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d077181e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89d7247b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d07717cbd0 0x55d077202220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89d7247b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d077202760 con 0x55d077171400 2023-09-23T09:29:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89d7247b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0772028a0 con 0x55d07717cbd0 2023-09-23T09:29:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89d7247b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d077202a60 con 0x55d077171d20 2023-09-23T09:29:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89d7247b80 10 client.0 _ll_register_callbacks cb 0x55d076ebe800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d077181e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d077181e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34964/0 (socket says 172.21.15.45:34964) 2023-09-23T09:29:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89c3fff700 1 -- 172.21.15.45:0/718336965 learned_addr learned my addr 172.21.15.45:0/718336965 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.010+0000 7f89c8df9700 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171400 0x55d077181920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89c3fff700 1 -- 172.21.15.45:0/718336965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d07717cbd0 msgr2=0x55d077202220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:29:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89c3fff700 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d07717cbd0 0x55d077202220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89c3fff700 1 -- 172.21.15.45:0/718336965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171400 msgr2=0x55d077181920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89c3fff700 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171400 0x55d077181920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89c3fff700 1 -- 172.21.15.45:0/718336965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c41a5f60 con 0x55d077171d20 2023-09-23T09:29:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89c3fff700 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d077181e60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89ac09a460 tx=0x7f89ac00cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8586400e4bd2bc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89a3fff700 1 -- 172.21.15.45:0/718336965 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac09e8b0 con 0x55d077171d20 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89a3fff700 1 -- 172.21.15.45:0/718336965 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f89ac0a2af0 con 0x55d077171d20 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.011+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d077165250 con 0x55d077171d20 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89a3fff700 1 -- 172.21.15.45:0/718336965 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac0ba0b0 con 0x55d077171d20 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 10 client.25222 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d076eefde0 con 0x55d077171d20 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89a3fff700 1 -- 172.21.15.45:0/718336965 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d077165250 con 0x55d077171d20 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89a3fff700 1 -- 172.21.15.45:0/718336965 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f89ac0a2660 con 0x55d077171d20 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 10 client.25222 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 10 client.25222 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-23T09:29:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 20 client.25222 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d0772041d0 0x55d0772065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 4 client.25222 mds_command: new command op to 24496 tid=0[{"prefix": "damage ls"}] 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d076f10f70 con 0x55d0772041d0 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.012+0000 7f89c8df9700 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d0772041d0 0x55d0772065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.013+0000 7f89a3fff700 10 client.25222 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.013+0000 7f89c8df9700 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d0772041d0 0x55d0772065f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f662e1ffd23f22de server_cookie=cf377528c6ff3de5 in_seq=0 out_seq=0 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.013+0000 7f89a3fff700 1 -- 172.21.15.45:0/718336965 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d076f10f70 con 0x55d0772041d0 2023-09-23T09:29:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.013+0000 7f89a3fff700 10 client.25222 handle_command_reply: tid=0 2023-09-23T09:29:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.013+0000 7f89d7247b80 1 client.25222 shutdown 2023-09-23T09:29:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.014+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 msgr2=0x55d077181e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.014+0000 7f89d7247b80 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d077181e60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89ac09a460 tx=0x7f89ac00cdc0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.018+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 shutdown_connections 2023-09-23T09:29:48.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89d7247b80 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d07717cbd0 0x55d077202220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89d7247b80 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d077171400 0x55d077181920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89d7247b80 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x55d0772041d0 0x55d0772065f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89d7247b80 1 --2- 172.21.15.45:0/718336965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d077171d20 0x55d077181e60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 >> 172.21.15.45:0/718336965 conn(0x55d0770e6260 msgr2=0x55d077174b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89a3fff700 0 client.25222 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:29:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 shutdown_connections 2023-09-23T09:29:48.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.019+0000 7f89d7247b80 1 -- 172.21.15.45:0/718336965 wait complete. 2023-09-23T09:29:48.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.021+0000 7f89d7247b80 20 client.25222 trim_cache size 0 max 16384 2023-09-23T09:29:48.023 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-09-23T09:29:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f89b00a18e0 msgr2=0x7f89b00a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f89b00a18e0 0x7f89b00a3da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89b400c010 tx=0x7f89b4019040 comp rx=0 tx=0).stop 2023-09-23T09:29:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c4106150 msgr2=0x7f89c41994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c4106150 0x7f89c41994f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00ec70 tx=0x7f89ac00c5b0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 shutdown_connections 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89c419d950 0x7f89c419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4101030 0x7f89c4198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f89b00a18e0 0x7f89b00a3da0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 --2- 172.21.15.45:0/3428627633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c4106150 0x7f89c41994f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.022+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 >> 172.21.15.45:0/3428627633 conn(0x7f89c40fc8d0 msgr2=0x7f89c40fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.023+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 shutdown_connections 2023-09-23T09:29:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.024+0000 7f89cb05d700 1 -- 172.21.15.45:0/3428627633 wait complete. 2023-09-23T09:29:48.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T09:29:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.223+0000 7fa36b02a700 1 -- 172.21.15.45:0/2403024024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3640ff0b0 msgr2=0x7fa3640ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.223+0000 7fa36b02a700 1 --2- 172.21.15.45:0/2403024024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3640ff0b0 0x7fa3640ff490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00b0a0 tx=0x7fa35c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.223+0000 7fa36b02a700 1 -- 172.21.15.45:0/2403024024 shutdown_connections 2023-09-23T09:29:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.223+0000 7fa36b02a700 1 --2- 172.21.15.45:0/2403024024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa364109550 0x7fa36410fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.223+0000 7fa36b02a700 1 --2- 172.21.15.45:0/2403024024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3640ff0b0 0x7fa3640ff490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.223+0000 7fa36b02a700 1 --2- 172.21.15.45:0/2403024024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3640ff9d0 0x7fa364108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.223+0000 7fa36b02a700 1 -- 172.21.15.45:0/2403024024 >> 172.21.15.45:0/2403024024 conn(0x7fa364076e10 msgr2=0x7fa3640fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.227+0000 7fa36b02a700 1 -- 172.21.15.45:0/2403024024 shutdown_connections 2023-09-23T09:29:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.228+0000 7fa36b02a700 1 -- 172.21.15.45:0/2403024024 wait complete. 2023-09-23T09:29:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 Processor -- start 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 -- start start 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3640ff0b0 0x7fa36410f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3640ff9d0 0x7fa36410fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa364109550 0x7fa36410cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36410d340 con 0x7fa3640ff0b0 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36410d480 con 0x7fa3640ff9d0 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.229+0000 7fa36b02a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36410d640 con 0x7fa364109550 2023-09-23T09:29:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.230+0000 7fa368dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3640ff0b0 0x7fa36410f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.230+0000 7fa368dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3640ff0b0 0x7fa36410f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32872/0 (socket says 172.21.15.45:32872) 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.230+0000 7fa368dc6700 1 -- 172.21.15.45:0/3404334356 learned_addr learned my addr 172.21.15.45:0/3404334356 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.230+0000 7fa363fff700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3640ff9d0 0x7fa36410fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.231+0000 7fa3695c7700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa364109550 0x7fa36410cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa3695c7700 1 -- 172.21.15.45:0/3404334356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3640ff9d0 msgr2=0x7fa36410fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa3695c7700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3640ff9d0 0x7fa36410fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa3695c7700 1 -- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3640ff0b0 msgr2=0x7fa36410f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa3695c7700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3640ff0b0 0x7fa36410f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa3695c7700 1 -- 172.21.15.45:0/3404334356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c009d00 con 0x7fa364109550 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa3695c7700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa364109550 0x7fa36410cb70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b3f0 tx=0x7fa35400b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa361ffb700 1 -- 172.21.15.45:0/3404334356 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354033640 con 0x7fa364109550 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa361ffb700 1 -- 172.21.15.45:0/3404334356 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa354033c60 con 0x7fa364109550 2023-09-23T09:29:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.232+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36406a840 con 0x7fa364109550 2023-09-23T09:29:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.233+0000 7fa361ffb700 1 -- 172.21.15.45:0/3404334356 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354032c80 con 0x7fa364109550 2023-09-23T09:29:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.233+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36406acf0 con 0x7fa364109550 2023-09-23T09:29:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.236+0000 7fa361ffb700 1 -- 172.21.15.45:0/3404334356 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7fa3540337a0 con 0x7fa364109550 2023-09-23T09:29:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.237+0000 7fa361ffb700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa34c0a1820 0x7fa34c0a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.237+0000 7fa361ffb700 1 -- 172.21.15.45:0/3404334356 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa3540eeb00 con 0x7fa364109550 2023-09-23T09:29:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.238+0000 7fa377214b80 1 Processor -- start 2023-09-23T09:29:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.238+0000 7fa368dc6700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa34c0a1820 0x7fa34c0a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa377214b80 1 -- start start 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa377214b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556400 0x556eeb5567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa377214b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556d20 0x556eeb561690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa377214b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb563fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa377214b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4bf460 con 0x556eeb556d20 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa377214b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4b0c20 con 0x556eeb561bd0 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa377214b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4ae540 con 0x556eeb556400 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa368dc6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556400 0x556eeb5567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa368dc6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556400 0x556eeb5567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:34984/0 (socket says 172.21.15.45:34984) 2023-09-23T09:29:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa368dc6700 1 -- 172.21.15.45:0/1101888156 learned_addr learned my addr 172.21.15.45:0/1101888156 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa3695c7700 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb563fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa363fff700 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556d20 0x556eeb561690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa3695c7700 1 -- 172.21.15.45:0/1101888156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556400 msgr2=0x556eeb5567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.239+0000 7fa3695c7700 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556400 0x556eeb5567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa3695c7700 1 -- 172.21.15.45:0/1101888156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556d20 msgr2=0x556eeb561690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa3695c7700 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556d20 0x556eeb561690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa3695c7700 1 -- 172.21.15.45:0/1101888156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eeb50c8d0 con 0x556eeb561bd0 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa3695c7700 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb563fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ec290 tx=0x7fa354004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5eb52cde168d5e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa368dc6700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa34c0a1820 0x7fa34c0a3ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00bfd0 tx=0x7fa35c004a50 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa34affd700 1 -- 172.21.15.45:0/1101888156 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354033dd0 con 0x556eeb561bd0 2023-09-23T09:29:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa34affd700 1 -- 172.21.15.45:0/1101888156 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3540bba00 con 0x556eeb561bd0 2023-09-23T09:29:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.240+0000 7fa34affd700 1 -- 172.21.15.45:0/1101888156 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354032de0 con 0x556eeb561bd0 2023-09-23T09:29:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.241+0000 7fa377214b80 1 -- 172.21.15.45:0/1101888156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 msgr2=0x556eeb563fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.241+0000 7fa377214b80 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb563fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ec290 tx=0x7fa354004750 comp rx=0 tx=0).stop 2023-09-23T09:29:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 -- 172.21.15.45:0/1101888156 shutdown_connections 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb563fc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556400 0x556eeb5567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 --2- 172.21.15.45:0/1101888156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556d20 0x556eeb561690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 -- 172.21.15.45:0/1101888156 >> 172.21.15.45:0/1101888156 conn(0x556eeb4cb260 msgr2=0x556eeb552790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 -- 172.21.15.45:0/1101888156 shutdown_connections 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 -- 172.21.15.45:0/1101888156 wait complete. 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 -- start start 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.242+0000 7fa377214b80 1 Processor -- start 2023-09-23T09:29:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa377214b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb55ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa377214b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556d20 0x556eeb55ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa377214b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb561bd0 0x556eeb55f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa377214b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4a9e30 con 0x556eeb556400 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa377214b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb446710 con 0x556eeb556d20 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa377214b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4463b0 con 0x556eeb561bd0 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa377214b80 10 client.0 _ll_register_callbacks cb 0x556eeb2a3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa363fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556d20 0x556eeb55ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa363fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556d20 0x556eeb55ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54234/0 (socket says 172.21.15.45:54234) 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.243+0000 7fa363fff700 1 -- 172.21.15.45:0/3872323621 learned_addr learned my addr 172.21.15.45:0/3872323621 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa363fff700 1 -- 172.21.15.45:0/3872323621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb561bd0 msgr2=0x556eeb55f4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:29:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa363fff700 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb561bd0 0x556eeb55f4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa363fff700 1 -- 172.21.15.45:0/3872323621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 msgr2=0x556eeb55ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa363fff700 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb55ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa363fff700 1 -- 172.21.15.45:0/3872323621 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36406acf0 con 0x556eeb556d20 2023-09-23T09:29:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa363fff700 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556d20 0x556eeb55ef60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa3580126d0 tx=0x7fa358012a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4867c9e80a5365ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa343fff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:29:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa343fff700 1 -- 172.21.15.45:0/3872323621 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358017070 con 0x556eeb556d20 2023-09-23T09:29:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556eeb54a250 con 0x556eeb556d20 2023-09-23T09:29:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.244+0000 7fa343fff700 1 -- 172.21.15.45:0/3872323621 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa35800f540 con 0x556eeb556d20 2023-09-23T09:29:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa343fff700 1 -- 172.21.15.45:0/3872323621 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35800f700 con 0x556eeb556d20 2023-09-23T09:29:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 10 client.25223 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556eeb2d4de0 con 0x556eeb556d20 2023-09-23T09:29:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa343fff700 1 -- 172.21.15.45:0/3872323621 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa358028080 con 0x556eeb556d20 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa343fff700 1 -- 172.21.15.45:0/3872323621 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa358010870 con 0x556eeb556d20 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 10 client.25223 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 10 client.25223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 20 client.25223 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5718b0 0x556eeb573cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 4 client.25223 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.245+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556eeb4a2e10 con 0x556eeb5718b0 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.246+0000 7fa368dc6700 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5718b0 0x556eeb573cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.247+0000 7fa343fff700 10 client.25223 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.247+0000 7fa368dc6700 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5718b0 0x556eeb573cd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df1da27328b5853d server_cookie=2c019f598d065f0 in_seq=0 out_seq=0 2023-09-23T09:29:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.247+0000 7fa343fff700 1 -- 172.21.15.45:0/3872323621 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556eeb4a2e10 con 0x556eeb5718b0 2023-09-23T09:29:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.247+0000 7fa343fff700 10 client.25223 handle_command_reply: tid=0 2023-09-23T09:29:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 client.25223 shutdown 2023-09-23T09:29:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556d20 msgr2=0x556eeb55ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556d20 0x556eeb55ef60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa3580126d0 tx=0x7fa358012a90 comp rx=0 tx=0).stop 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 shutdown_connections 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5718b0 0x556eeb573cd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb561bd0 0x556eeb55f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb55ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 --2- 172.21.15.45:0/3872323621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556d20 0x556eeb55ef60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 >> 172.21.15.45:0/3872323621 conn(0x556eeb4cb260 msgr2=0x556eeb552380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.248+0000 7fa343fff700 0 client.25223 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.249+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 shutdown_connections 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.249+0000 7fa377214b80 1 -- 172.21.15.45:0/3872323621 wait complete. 2023-09-23T09:29:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.250+0000 7fa377214b80 20 client.25223 trim_cache size 0 max 16384 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.267+0000 7fa377214b80 1 Processor -- start 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.267+0000 7fa377214b80 1 -- start start 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.267+0000 7fa377214b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb568710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.267+0000 7fa377214b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb559d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.267+0000 7fa377214b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb55a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.267+0000 7fa377214b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb32a920 con 0x556eeb556400 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.267+0000 7fa377214b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb54c860 con 0x556eeb561bd0 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa377214b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb568c50 con 0x556eeb556d20 2023-09-23T09:29:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa368dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb568710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa363fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb559d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa368dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb568710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32892/0 (socket says 172.21.15.45:32892) 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa363fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb559d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35000/0 (socket says 172.21.15.45:35000) 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa368dc6700 1 -- 172.21.15.45:0/1847959752 learned_addr learned my addr 172.21.15.45:0/1847959752 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa363fff700 1 -- 172.21.15.45:0/1847959752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 msgr2=0x556eeb55a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa363fff700 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb55a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa363fff700 1 -- 172.21.15.45:0/1847959752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 msgr2=0x556eeb568710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa363fff700 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb568710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa363fff700 1 -- 172.21.15.45:0/1847959752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eeb2bdc60 con 0x556eeb556d20 2023-09-23T09:29:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.268+0000 7fa368dc6700 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb568710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:48.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.269+0000 7fa363fff700 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb559d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa358010d20 tx=0x7fa358005ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d65d9793dba1f928 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.269+0000 7fa34b7fe700 1 -- 172.21.15.45:0/1847959752 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358017070 con 0x556eeb556d20 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.269+0000 7fa34b7fe700 1 -- 172.21.15.45:0/1847959752 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556eeb54a250 con 0x556eeb556d20 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.269+0000 7fa34b7fe700 1 -- 172.21.15.45:0/1847959752 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358005650 con 0x556eeb556d20 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 -- 172.21.15.45:0/1847959752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 msgr2=0x556eeb559d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb559d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa358010d20 tx=0x7fa358005ba0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 -- 172.21.15.45:0/1847959752 shutdown_connections 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb561bd0 0x556eeb55a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb556400 0x556eeb568710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 --2- 172.21.15.45:0/1847959752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb559d70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 -- 172.21.15.45:0/1847959752 >> 172.21.15.45:0/1847959752 conn(0x556eeb4cb260 msgr2=0x556eeb5738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 -- 172.21.15.45:0/1847959752 shutdown_connections 2023-09-23T09:29:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 -- 172.21.15.45:0/1847959752 wait complete. 2023-09-23T09:29:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.270+0000 7fa377214b80 1 -- start start 2023-09-23T09:29:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 1 Processor -- start 2023-09-23T09:29:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556400 0x556eeb5668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb566e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb561bd0 0x556eeb5c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb567540 con 0x556eeb561bd0 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb5c4c20 con 0x556eeb556400 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb5c4d60 con 0x556eeb556d20 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa377214b80 10 client.0 _ll_register_callbacks cb 0x7fa358009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa363fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb566e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa363fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb566e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35004/0 (socket says 172.21.15.45:35004) 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.271+0000 7fa363fff700 1 -- 172.21.15.45:0/2652011725 learned_addr learned my addr 172.21.15.45:0/2652011725 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa368dc6700 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556400 0x556eeb5668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa363fff700 1 -- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556400 msgr2=0x556eeb5668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa363fff700 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556400 0x556eeb5668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa363fff700 1 -- 172.21.15.45:0/2652011725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb561bd0 msgr2=0x556eeb5c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa363fff700 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb561bd0 0x556eeb5c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa363fff700 1 -- 172.21.15.45:0/2652011725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa364110710 con 0x556eeb556d20 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa363fff700 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb566e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa358005c50 tx=0x7fa358005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4bcad762b503c47 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa343fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa343fff700 1 -- 172.21.15.45:0/2652011725 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358017040 con 0x556eeb556d20 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa343fff700 1 -- 172.21.15.45:0/2652011725 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa358010840 con 0x556eeb556d20 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3540bba00 con 0x556eeb556d20 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.272+0000 7fa343fff700 1 -- 172.21.15.45:0/2652011725 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa358010a00 con 0x556eeb556d20 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.273+0000 7fa377214b80 10 client.25237 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.273+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556eeb2d4de0 con 0x556eeb556d20 2023-09-23T09:29:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa343fff700 1 -- 172.21.15.45:0/2652011725 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3540bba00 con 0x556eeb556d20 2023-09-23T09:29:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa343fff700 1 -- 172.21.15.45:0/2652011725 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa35802d070 con 0x556eeb556d20 2023-09-23T09:29:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa377214b80 10 client.25237 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa377214b80 10 client.25237 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-23T09:29:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa377214b80 20 client.25237 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa377214b80 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5c6140 0x556eeb5c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa377214b80 4 client.25237 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.274+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556eeb4a2b30 con 0x556eeb5c6140 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.275+0000 7fa368dc6700 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5c6140 0x556eeb5c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.275+0000 7fa343fff700 10 client.25237 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.275+0000 7fa368dc6700 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5c6140 0x556eeb5c8560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbb132bdee32b2a3 server_cookie=f7354f52bd0436b4 in_seq=0 out_seq=0 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa343fff700 1 -- 172.21.15.45:0/2652011725 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556eeb4a2b30 con 0x556eeb5c6140 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa343fff700 10 client.25237 handle_command_reply: tid=0 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 client.25237 shutdown 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 msgr2=0x556eeb566e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb566e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa358005c50 tx=0x7fa358005840 comp rx=0 tx=0).stop 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 shutdown_connections 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x556eeb5c6140 0x556eeb5c8560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556eeb561bd0 0x556eeb5c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556eeb556400 0x556eeb5668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa343fff700 0 client.25237 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 --2- 172.21.15.45:0/2652011725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556eeb556d20 0x556eeb566e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 >> 172.21.15.45:0/2652011725 conn(0x556eeb4cb260 msgr2=0x556eeb559d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 shutdown_connections 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.276+0000 7fa377214b80 1 -- 172.21.15.45:0/2652011725 wait complete. 2023-09-23T09:29:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa377214b80 20 client.25237 trim_cache size 0 max 16384 2023-09-23T09:29:48.281 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-09-23T09:29:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa34c0a1820 msgr2=0x7fa34c0a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa34c0a1820 0x7fa34c0a3ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00bfd0 tx=0x7fa35c004a50 comp rx=0 tx=0).stop 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa364109550 msgr2=0x7fa36410cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa364109550 0x7fa36410cb70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b3f0 tx=0x7fa35400b700 comp rx=0 tx=0).stop 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 shutdown_connections 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa364109550 0x7fa36410cb70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa34c0a1820 0x7fa34c0a3ce0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3640ff0b0 0x7fa36410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 --2- 172.21.15.45:0/3404334356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3640ff9d0 0x7fa36410fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 >> 172.21.15.45:0/3404334356 conn(0x7fa364076e10 msgr2=0x7fa364107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 shutdown_connections 2023-09-23T09:29:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.277+0000 7fa36b02a700 1 -- 172.21.15.45:0/3404334356 wait complete. 2023-09-23T09:29:48.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T09:29:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 -- 172.21.15.45:0/2566258453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340101980 msgr2=0x7ff3401059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 --2- 172.21.15.45:0/2566258453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340101980 0x7ff3401059c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff330009990 tx=0x7ff330009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 -- 172.21.15.45:0/2566258453 shutdown_connections 2023-09-23T09:29:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 --2- 172.21.15.45:0/2566258453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3401060f0 0x7ff34010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 --2- 172.21.15.45:0/2566258453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff340100fd0 0x7ff3401013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 --2- 172.21.15.45:0/2566258453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340101980 0x7ff3401059c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 -- 172.21.15.45:0/2566258453 >> 172.21.15.45:0/2566258453 conn(0x7ff3400fc890 msgr2=0x7ff3400fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 -- 172.21.15.45:0/2566258453 shutdown_connections 2023-09-23T09:29:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.509+0000 7ff344a18700 1 -- 172.21.15.45:0/2566258453 wait complete. 2023-09-23T09:29:48.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.510+0000 7ff344a18700 1 Processor -- start 2023-09-23T09:29:48.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff344a18700 1 -- start start 2023-09-23T09:29:48.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff344a18700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff340100fd0 0x7ff34010cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff344a18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3401060f0 0x7ff34010d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff344a18700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3401afb80 0x7ff3401b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff344a18700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401b2540 con 0x7ff3401060f0 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff344a18700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401b26b0 con 0x7ff3401afb80 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff344a18700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401b2870 con 0x7ff340100fd0 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff33dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3401060f0 0x7ff34010d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3401afb80 0x7ff3401b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff33dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3401060f0 0x7ff34010d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32910/0 (socket says 172.21.15.45:32910) 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.511+0000 7ff33dd9b700 1 -- 172.21.15.45:0/3562115558 learned_addr learned my addr 172.21.15.45:0/3562115558 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.512+0000 7ff33e59c700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff340100fd0 0x7ff34010cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.512+0000 7ff33dd9b700 1 -- 172.21.15.45:0/3562115558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff340100fd0 msgr2=0x7ff34010cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.512+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff340100fd0 0x7ff34010cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.512+0000 7ff33dd9b700 1 -- 172.21.15.45:0/3562115558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3401afb80 msgr2=0x7ff3401b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.512+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3401afb80 0x7ff3401b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.512+0000 7ff33dd9b700 1 -- 172.21.15.45:0/3562115558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3300095f0 con 0x7ff3401060f0 2023-09-23T09:29:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.512+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3401060f0 0x7ff34010d140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff330004760 tx=0x7ff330004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.514+0000 7ff32f7fe700 1 -- 172.21.15.45:0/3562115558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33003f070 con 0x7ff3401060f0 2023-09-23T09:29:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.514+0000 7ff32f7fe700 1 -- 172.21.15.45:0/3562115558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff330044470 con 0x7ff3401060f0 2023-09-23T09:29:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.514+0000 7ff32f7fe700 1 -- 172.21.15.45:0/3562115558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330039660 con 0x7ff3401060f0 2023-09-23T09:29:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.514+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34019eb70 con 0x7ff3401060f0 2023-09-23T09:29:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.514+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34019efd0 con 0x7ff3401060f0 2023-09-23T09:29:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.517+0000 7ff32f7fe700 1 -- 172.21.15.45:0/3562115558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7ff33000fad0 con 0x7ff3401060f0 2023-09-23T09:29:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.518+0000 7ff32f7fe700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff3280a1920 0x7ff3280a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.518+0000 7ff32f7fe700 1 -- 172.21.15.45:0/3562115558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff3300f03a0 con 0x7ff3401060f0 2023-09-23T09:29:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.519+0000 7ff33e59c700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff3280a1920 0x7ff3280a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.519+0000 7ff33e59c700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff3280a1920 0x7ff3280a3de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff334009bf0 tx=0x7ff334009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 Processor -- start 2023-09-23T09:29:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 -- start start 2023-09-23T09:29:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327400 0x5603143277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327d20 0x560314332690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314334fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560314290460 con 0x560314332bd0 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560314281c20 con 0x560314327400 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff350c02b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031427f540 con 0x560314327d20 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314334fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.520+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314334fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32922/0 (socket says 172.21.15.45:32922) 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33ed9d700 1 -- 172.21.15.45:0/2773610700 learned_addr learned my addr 172.21.15.45:0/2773610700 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327d20 0x560314332690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327400 0x5603143277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33ed9d700 1 -- 172.21.15.45:0/2773610700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327d20 msgr2=0x560314332690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33ed9d700 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327d20 0x560314332690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33ed9d700 1 -- 172.21.15.45:0/2773610700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327400 msgr2=0x5603143277e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33ed9d700 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327400 0x5603143277e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33ed9d700 1 -- 172.21.15.45:0/2773610700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603142dd8d0 con 0x560314332bd0 2023-09-23T09:29:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327400 0x5603143277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff33ed9d700 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314334fc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff338011600 tx=0x7ff338009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8c4ae168df6bb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.521+0000 7ff32cff9700 1 -- 172.21.15.45:0/2773610700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33800e040 con 0x560314332bd0 2023-09-23T09:29:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff32cff9700 1 -- 172.21.15.45:0/2773610700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33800f040 con 0x560314332bd0 2023-09-23T09:29:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff32cff9700 1 -- 172.21.15.45:0/2773610700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338004760 con 0x560314332bd0 2023-09-23T09:29:48.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 -- 172.21.15.45:0/2773610700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 msgr2=0x560314334fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314334fc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff338011600 tx=0x7ff338009920 comp rx=0 tx=0).stop 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 -- 172.21.15.45:0/2773610700 shutdown_connections 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314334fc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327400 0x5603143277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2773610700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327d20 0x560314332690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 -- 172.21.15.45:0/2773610700 >> 172.21.15.45:0/2773610700 conn(0x56031429c260 msgr2=0x560314323790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.522+0000 7ff350c02b80 1 -- 172.21.15.45:0/2773610700 shutdown_connections 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 -- 172.21.15.45:0/2773610700 wait complete. 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 -- start start 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 Processor -- start 2023-09-23T09:29:48.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327400 0x56031432f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314330340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031427ae30 con 0x560314332bd0 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560314217710 con 0x560314327d20 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603142173b0 con 0x560314327400 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff350c02b80 10 client.0 _ll_register_callbacks cb 0x560314074800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314330340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff33e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327400 0x56031432f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314330340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32928/0 (socket says 172.21.15.45:32928) 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff33ed9d700 1 -- 172.21.15.45:0/3376379175 learned_addr learned my addr 172.21.15.45:0/3376379175 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.523+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33e59c700 1 -- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 msgr2=0x56031432fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33e59c700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33e59c700 1 -- 172.21.15.45:0/3376379175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 msgr2=0x560314330340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33e59c700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314330340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33e59c700 1 -- 172.21.15.45:0/3376379175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff334011a60 con 0x560314327400 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33ed9d700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314330340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff33e59c700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327400 0x56031432f8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097d0 tx=0x7ff3340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d82a1d810948f882 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff325ffb700 1 -- 172.21.15.45:0/3376379175 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33401e810 con 0x560314327400 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56031431b250 con 0x560314327400 2023-09-23T09:29:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff325ffb700 1 -- 172.21.15.45:0/3376379175 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff334018070 con 0x560314327400 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff325ffb700 1 -- 172.21.15.45:0/3376379175 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33401e970 con 0x560314327400 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff350c02b80 10 client.25249 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.524+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff3380112b0 con 0x560314327400 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff325ffb700 1 -- 172.21.15.45:0/3376379175 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff334034080 con 0x560314327400 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff325ffb700 1 -- 172.21.15.45:0/3376379175 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff334028ca0 con 0x560314327400 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff350c02b80 10 client.25249 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff350c02b80 10 client.25249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff350c02b80 20 client.25249 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff350c02b80 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560314353750 0x560314355b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff350c02b80 4 client.25249 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560314273e10 con 0x560314353750 2023-09-23T09:29:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560314353750 0x560314355b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff325ffb700 10 client.25249 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.525+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560314353750 0x560314355b70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6c45962ae88e2c server_cookie=ce66d53148d55904 in_seq=0 out_seq=0 2023-09-23T09:29:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff325ffb700 1 -- 172.21.15.45:0/3376379175 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560314273e10 con 0x560314353750 2023-09-23T09:29:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff325ffb700 10 client.25249 handle_command_reply: tid=0 2023-09-23T09:29:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 client.25249 shutdown 2023-09-23T09:29:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327400 msgr2=0x56031432f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327400 0x56031432f8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097d0 tx=0x7ff3340047c0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 shutdown_connections 2023-09-23T09:29:48.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560314353750 0x560314355b70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314332bd0 0x560314330340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314327400 0x56031432f8c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 --2- 172.21.15.45:0/3376379175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 >> 172.21.15.45:0/3376379175 conn(0x56031429c260 msgr2=0x560314323380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff325ffb700 0 client.25249 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:48.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 shutdown_connections 2023-09-23T09:29:48.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.527+0000 7ff350c02b80 1 -- 172.21.15.45:0/3376379175 wait complete. 2023-09-23T09:29:48.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.528+0000 7ff350c02b80 20 client.25249 trim_cache size 0 max 16384 2023-09-23T09:29:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.543+0000 7ff350c02b80 1 Processor -- start 2023-09-23T09:29:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff350c02b80 1 -- start start 2023-09-23T09:29:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031434a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff350c02b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603140fb920 con 0x560314327400 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff350c02b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031431fd90 con 0x560314327d20 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff350c02b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560314350d50 con 0x560314332bd0 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031434a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031434a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32932/0 (socket says 172.21.15.45:32932) 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 -- 172.21.15.45:0/2815805493 learned_addr learned my addr 172.21.15.45:0/2815805493 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33ed9d700 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 -- 172.21.15.45:0/2815805493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 msgr2=0x5603143303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 -- 172.21.15.45:0/2815805493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 msgr2=0x56031432fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33e59c700 1 -- 172.21.15.45:0/2815805493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56031431f480 con 0x560314327400 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.544+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031434a730 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff334009a90 tx=0x7ff334004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3470190285343b4f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff32d7fa700 1 -- 172.21.15.45:0/2815805493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011bf0 con 0x560314327400 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff32d7fa700 1 -- 172.21.15.45:0/2815805493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56031431b250 con 0x560314327400 2023-09-23T09:29:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff32d7fa700 1 -- 172.21.15.45:0/2815805493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33401e970 con 0x560314327400 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff350c02b80 1 -- 172.21.15.45:0/2815805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 msgr2=0x56031434a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031434a730 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff334009a90 tx=0x7ff334004750 comp rx=0 tx=0).stop 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff350c02b80 1 -- 172.21.15.45:0/2815805493 shutdown_connections 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031434a730 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2815805493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031432fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.545+0000 7ff350c02b80 1 -- 172.21.15.45:0/2815805493 >> 172.21.15.45:0/2815805493 conn(0x56031429c260 msgr2=0x56031432ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 -- 172.21.15.45:0/2815805493 shutdown_connections 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 -- 172.21.15.45:0/2815805493 wait complete. 2023-09-23T09:29:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 -- start start 2023-09-23T09:29:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 Processor -- start 2023-09-23T09:29:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031439b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031439ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031439c360 con 0x560314327400 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603143a6ad0 con 0x560314327d20 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603143a6c10 con 0x560314332bd0 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff350c02b80 10 client.0 _ll_register_callbacks cb 0x560314074800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff33e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031439b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff33dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031439ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff33e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031439b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32938/0 (socket says 172.21.15.45:32938) 2023-09-23T09:29:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff33e59c700 1 -- 172.21.15.45:0/2717852595 learned_addr learned my addr 172.21.15.45:0/2717852595 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.546+0000 7ff33ed9d700 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff33e59c700 1 -- 172.21.15.45:0/2717852595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 msgr2=0x5603143a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff33e59c700 1 -- 172.21.15.45:0/2717852595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 msgr2=0x56031439ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031439ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff33e59c700 1 -- 172.21.15.45:0/2717852595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34019efd0 con 0x560314327400 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff33e59c700 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031439b510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff334011820 tx=0x7ff334005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7070da134038bfb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff325ffb700 1 -- 172.21.15.45:0/2717852595 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011df0 con 0x560314327400 2023-09-23T09:29:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff33800f040 con 0x560314327400 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff325ffb700 1 -- 172.21.15.45:0/2717852595 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff334018040 con 0x560314327400 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff325ffb700 1 -- 172.21.15.45:0/2717852595 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff334005040 con 0x560314327400 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff350c02b80 10 client.15570 fetch_fsmap learned FSMap version 23 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.547+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff3380112b0 con 0x560314327400 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff325ffb700 1 -- 172.21.15.45:0/2717852595 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33800f040 con 0x560314327400 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff325ffb700 1 -- 172.21.15.45:0/2717852595 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff33401f770 con 0x560314327400 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff350c02b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 23 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff350c02b80 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff350c02b80 20 client.15570 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5603143a7ea0 0x5603143aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff350c02b80 4 client.15570 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560314273b30 con 0x5603143a7ea0 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5603143a7ea0 0x5603143aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff325ffb700 10 client.15570 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.548+0000 7ff33dd9b700 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5603143a7ea0 0x5603143aa2c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd9e975af7759ee server_cookie=825626d542c889ce in_seq=0 out_seq=0 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff325ffb700 1 -- 172.21.15.45:0/2717852595 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560314273b30 con 0x5603143a7ea0 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff325ffb700 10 client.15570 handle_command_reply: tid=0 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 client.15570 shutdown 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 msgr2=0x56031439b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031439b510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff334011820 tx=0x7ff334005d50 comp rx=0 tx=0).stop 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 shutdown_connections 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5603143a7ea0 0x5603143aa2c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560314332bd0 0x5603143a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560314327400 0x56031439b510 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff325ffb700 0 client.15570 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 --2- 172.21.15.45:0/2717852595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560314327d20 0x56031439ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 >> 172.21.15.45:0/2717852595 conn(0x56031429c260 msgr2=0x56031432fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 shutdown_connections 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 1 -- 172.21.15.45:0/2717852595 wait complete. 2023-09-23T09:29:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.549+0000 7ff350c02b80 20 client.15570 trim_cache size 0 max 16384 2023-09-23T09:29:48.559 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-09-23T09:29:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff3280a1920 msgr2=0x7ff3280a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff3280a1920 0x7ff3280a3de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff334009bf0 tx=0x7ff334009260 comp rx=0 tx=0).stop 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3401060f0 msgr2=0x7ff34010d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3401060f0 0x7ff34010d140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff330004760 tx=0x7ff330004790 comp rx=0 tx=0).stop 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 shutdown_connections 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3401afb80 0x7ff3401b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff340100fd0 0x7ff34010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff3280a1920 0x7ff3280a3de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 --2- 172.21.15.45:0/3562115558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3401060f0 0x7ff34010d140 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 >> 172.21.15.45:0/3562115558 conn(0x7ff3400fc890 msgr2=0x7ff3400fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 shutdown_connections 2023-09-23T09:29:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:48.550+0000 7ff344a18700 1 -- 172.21.15.45:0/3562115558 wait complete. 2023-09-23T09:29:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:48 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2696759284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:48.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:48 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2696759284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:48.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:48 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2696759284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:29:49.568 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T09:29:49.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T09:29:49.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:49 smithi045 ceph-mon[98627]: pgmap v14: 97 pgs: 97 active+clean; 448 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 69 MiB/s wr, 334 op/s 2023-09-23T09:29:49.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:49 smithi123 ceph-mon[103577]: pgmap v14: 97 pgs: 97 active+clean; 448 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 69 MiB/s wr, 334 op/s 2023-09-23T09:29:49.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 -- 172.21.15.45:0/3288830516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 msgr2=0x7f3e6010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 --2- 172.21.15.45:0/3288830516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e6010ad40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400dd20 tx=0x7f3e5400f330 comp rx=0 tx=0).stop 2023-09-23T09:29:49.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 -- 172.21.15.45:0/3288830516 shutdown_connections 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 --2- 172.21.15.45:0/3288830516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e6010b360 0x7f3e6010d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 --2- 172.21.15.45:0/3288830516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e60101f40 0x7f3e60102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 --2- 172.21.15.45:0/3288830516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e6010ad40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 -- 172.21.15.45:0/3288830516 >> 172.21.15.45:0/3288830516 conn(0x7f3e600fb740 msgr2=0x7f3e600fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 -- 172.21.15.45:0/3288830516 shutdown_connections 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 -- 172.21.15.45:0/3288830516 wait complete. 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.759+0000 7f3e65d09700 1 Processor -- start 2023-09-23T09:29:49.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e65d09700 1 -- start start 2023-09-23T09:29:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e65d09700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e60101f40 0x7f3e601b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e65d09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e601b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e65d09700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e6010b360 0x7f3e601bb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e65d09700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e601bbdc0 con 0x7f3e60102860 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e65d09700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e601bbf30 con 0x7f3e60101f40 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e65d09700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e601bc0f0 con 0x7f3e6010b360 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e601b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e601b7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32954/0 (socket says 172.21.15.45:32954) 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e5effd700 1 -- 172.21.15.45:0/2144233712 learned_addr learned my addr 172.21.15.45:0/2144233712 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e60101f40 0x7f3e601b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.760+0000 7f3e5ffff700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e6010b360 0x7f3e601bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/2144233712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e6010b360 msgr2=0x7f3e601bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e6010b360 0x7f3e601bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 msgr2=0x7f3e601b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e601b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/2144233712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5400da00 con 0x7f3e60101f40 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5effd700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e601b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e60101f40 0x7f3e601b6de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000bf10 tx=0x7f3e500094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5cff9700 1 -- 172.21.15.45:0/2144233712 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500110d0 con 0x7f3e60101f40 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e601bc3d0 con 0x7f3e60101f40 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5cff9700 1 -- 172.21.15.45:0/2144233712 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e500116f0 con 0x7f3e60101f40 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e5cff9700 1 -- 172.21.15.45:0/2144233712 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000fd90 con 0x7f3e60101f40 2023-09-23T09:29:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.761+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e601c3c10 con 0x7f3e60101f40 2023-09-23T09:29:49.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.765+0000 7f3e5cff9700 1 -- 172.21.15.45:0/2144233712 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7f3e50011230 con 0x7f3e60101f40 2023-09-23T09:29:49.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.765+0000 7f3e5cff9700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3e480a1780 0x7f3e480a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.766+0000 7f3e5cff9700 1 -- 172.21.15.45:0/2144233712 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3e500cd3d0 con 0x7f3e60101f40 2023-09-23T09:29:49.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.766+0000 7f3e5effd700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3e480a1780 0x7f3e480a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.766+0000 7f3e5effd700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3e480a1780 0x7f3e480a3c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400dfd0 tx=0x7f3e5401a040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.767+0000 7f3e71ef3b80 1 Processor -- start 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e71ef3b80 1 -- start start 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f829233860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f829233da0 0x55f82923e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f82923ec50 0x55f829241040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82919c4e0 con 0x55f829233480 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82918dca0 con 0x55f829233da0 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82918b5c0 con 0x55f82923ec50 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f829233860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f829233860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32960/0 (socket says 172.21.15.45:32960) 2023-09-23T09:29:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/878805548 learned_addr learned my addr 172.21.15.45:0/878805548 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/878805548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f82923ec50 msgr2=0x55f829241040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/878805548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f82923ec50 0x55f829241040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/878805548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f829233da0 msgr2=0x55f82923e710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:29:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/878805548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f829233da0 0x55f82923e710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.768+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/878805548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8291e9950 con 0x55f829233480 2023-09-23T09:29:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.769+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/878805548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f829233860 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000bee0 tx=0x7f3e50096620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb76997ce0d6229d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.769+0000 7f3e45ffb700 1 -- 172.21.15.45:0/878805548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5009a8c0 con 0x55f829233480 2023-09-23T09:29:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.769+0000 7f3e45ffb700 1 -- 172.21.15.45:0/878805548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5009a1c0 con 0x55f829233480 2023-09-23T09:29:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.769+0000 7f3e45ffb700 1 -- 172.21.15.45:0/878805548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000f460 con 0x55f829233480 2023-09-23T09:29:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/878805548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 msgr2=0x55f829233860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/878805548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f829233860 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000bee0 tx=0x7f3e50096620 comp rx=0 tx=0).stop 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/878805548 shutdown_connections 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/878805548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f82923ec50 0x55f829241040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/878805548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f829233860 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/878805548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f829233da0 0x55f82923e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/878805548 >> 172.21.15.45:0/878805548 conn(0x55f8291a82e0 msgr2=0x55f82922f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/878805548 shutdown_connections 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/878805548 wait complete. 2023-09-23T09:29:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.770+0000 7f3e71ef3b80 1 -- start start 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 1 Processor -- start 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f82923b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233da0 0x55f82923bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f82923c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829186eb0 con 0x55f829233480 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829123790 con 0x55f82923ec50 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829123430 con 0x55f829233da0 2023-09-23T09:29:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e71ef3b80 10 client.0 _ll_register_callbacks cb 0x55f828f80880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f82923b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f82923b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32966/0 (socket says 172.21.15.45:32966) 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.771+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/3050400772 learned_addr learned my addr 172.21.15.45:0/3050400772 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/3050400772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233da0 msgr2=0x55f82923bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233da0 0x55f82923bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/3050400772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 msgr2=0x55f82923c430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f82923c430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e5f7fe700 1 -- 172.21.15.45:0/3050400772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e601c3c10 con 0x55f829233480 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f82923b9b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cb110 tx=0x7f3e50099d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1171c255c1548268 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e3effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e3effd700 1 -- 172.21.15.45:0/3050400772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5009a8c0 con 0x55f829233480 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e3effd700 1 -- 172.21.15.45:0/3050400772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3e5009e310 con 0x55f829233480 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8292272d0 con 0x55f829233480 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e3effd700 1 -- 172.21.15.45:0/3050400772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e500a2550 con 0x55f829233480 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e71ef3b80 10 client.15582 fetch_fsmap learned FSMap version 24 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f828fb1e60 con 0x55f829233480 2023-09-23T09:29:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e3effd700 1 -- 172.21.15.45:0/3050400772 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8292272d0 con 0x55f829233480 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.772+0000 7f3e3effd700 1 -- 172.21.15.45:0/3050400772 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e500b2ae0 con 0x55f829233480 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.773+0000 7f3e71ef3b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 24 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.773+0000 7f3e71ef3b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.773+0000 7f3e71ef3b80 20 client.15582 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.773+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f82924e420 0x55f829250840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.773+0000 7f3e71ef3b80 4 client.15582 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.773+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f82917fe90 con 0x55f82924e420 2023-09-23T09:29:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.774+0000 7f3e5effd700 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f82924e420 0x55f829250840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.775+0000 7f3e5effd700 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f82924e420 0x55f829250840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9406fd785de6601 server_cookie=345f87d2861eb35e in_seq=0 out_seq=0 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.775+0000 7f3e3effd700 10 client.15582 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e3effd700 1 -- 172.21.15.45:0/3050400772 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f82917fe90 con 0x55f82924e420 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e3effd700 10 client.15582 handle_command_reply: tid=0 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 client.15582 shutdown 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 msgr2=0x55f82923b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f82923b9b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cb110 tx=0x7f3e50099d80 comp rx=0 tx=0).stop 2023-09-23T09:29:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 shutdown_connections 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f82924e420 0x55f829250840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f82923c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e3effd700 0 client.15582 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233480 0x55f82923b9b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/3050400772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233da0 0x55f82923bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 >> 172.21.15.45:0/3050400772 conn(0x55f8291a82e0 msgr2=0x55f82922f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 shutdown_connections 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.776+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/3050400772 wait complete. 2023-09-23T09:29:49.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.777+0000 7f3e71ef3b80 20 client.15582 trim_cache size 0 max 16384 2023-09-23T09:29:49.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:49 smithi098 ceph-mon[102386]: pgmap v14: 97 pgs: 97 active+clean; 448 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 69 MiB/s wr, 334 op/s 2023-09-23T09:29:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 Processor -- start 2023-09-23T09:29:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 -- start start 2023-09-23T09:29:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f829245740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f829235a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829235fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8290079a0 con 0x55f829233da0 2023-09-23T09:29:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829233ae0 con 0x55f82923ec50 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829234c70 con 0x55f829233480 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f829235a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f829235a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32976/0 (socket says 172.21.15.45:32976) 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e5effd700 1 -- 172.21.15.45:0/1503424447 learned_addr learned my addr 172.21.15.45:0/1503424447 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e5ffff700 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829235fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.795+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f829245740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e5effd700 1 -- 172.21.15.45:0/1503424447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 msgr2=0x55f829245740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e5effd700 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f829245740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e5effd700 1 -- 172.21.15.45:0/1503424447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 msgr2=0x55f829235fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e5effd700 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829235fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e5effd700 1 -- 172.21.15.45:0/1503424447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f829255470 con 0x55f829233da0 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e5effd700 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f829235a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3e54006010 tx=0x7f3e5400f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf9fa290148bcfb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e467fc700 1 -- 172.21.15.45:0/1503424447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54031ce0 con 0x55f829233da0 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e467fc700 1 -- 172.21.15.45:0/1503424447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5401e040 con 0x55f829233da0 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.796+0000 7f3e467fc700 1 -- 172.21.15.45:0/1503424447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54018b00 con 0x55f829233da0 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1503424447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 msgr2=0x55f829235a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f829235a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3e54006010 tx=0x7f3e5400f330 comp rx=0 tx=0).stop 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1503424447 shutdown_connections 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829235fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f829245740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1503424447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f829235a80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1503424447 >> 172.21.15.45:0/1503424447 conn(0x55f8291a82e0 msgr2=0x55f829235370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1503424447 shutdown_connections 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1503424447 wait complete. 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.797+0000 7f3e71ef3b80 1 -- start start 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 1 Processor -- start 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f82929c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f82929c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829296590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829296b70 con 0x55f829233da0 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829296ce0 con 0x55f82923ec50 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f829296ea0 con 0x55f829233480 2023-09-23T09:29:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e71ef3b80 10 client.0 _ll_register_callbacks cb 0x7f3e50096500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f82929c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f82929c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32982/0 (socket says 172.21.15.45:32982) 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 -- 172.21.15.45:0/1195584818 learned_addr learned my addr 172.21.15.45:0/1195584818 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f82929c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5ffff700 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829296590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 -- 172.21.15.45:0/1195584818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 msgr2=0x55f82929c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f82929c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 -- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 msgr2=0x55f829296590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829296590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.798+0000 7f3e5effd700 1 -- 172.21.15.45:0/1195584818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6010e4e0 con 0x55f829233da0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e5effd700 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f82929c790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3e540314f0 tx=0x7f3e5400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204d6a20f3aff353 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e3effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e3effd700 1 -- 172.21.15.45:0/1195584818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54031d60 con 0x55f829233da0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e3effd700 1 -- 172.21.15.45:0/1195584818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3e540039f0 con 0x55f829233da0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e5009a1c0 con 0x55f829233da0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e3effd700 1 -- 172.21.15.45:0/1195584818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e54003bd0 con 0x55f829233da0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 10 client.15588 fetch_fsmap learned FSMap version 24 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f82922b500 con 0x55f829233da0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e3effd700 1 -- 172.21.15.45:0/1195584818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5009a1c0 con 0x55f829233da0 2023-09-23T09:29:49.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e3effd700 1 -- 172.21.15.45:0/1195584818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e54005350 con 0x55f829233da0 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 24 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 20 client.15588 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f8292a3cf0 0x55f8292a6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 4 client.15588 mds_command: new command op to 24530 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.799+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f82917fbb0 con 0x55f8292a3cf0 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.800+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f8292a3cf0 0x55f8292a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.800+0000 7f3e3effd700 10 client.15588 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:49.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.800+0000 7f3e5f7fe700 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f8292a3cf0 0x55f8292a6110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88364e38a9bbdd18 server_cookie=abfdc1b2117ad110 in_seq=0 out_seq=0 2023-09-23T09:29:49.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.801+0000 7f3e3effd700 1 -- 172.21.15.45:0/1195584818 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f82917fbb0 con 0x55f8292a3cf0 2023-09-23T09:29:49.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.801+0000 7f3e3effd700 10 client.15588 handle_command_reply: tid=0 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.801+0000 7f3e71ef3b80 1 client.15588 shutdown 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 msgr2=0x55f82929c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f82929c790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3e540314f0 tx=0x7f3e5400f680 comp rx=0 tx=0).stop 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 shutdown_connections 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f8292a3cf0 0x55f8292a6110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f82923ec50 0x55f829296590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e3effd700 0 client.15588 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f829233480 0x55f82929c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 --2- 172.21.15.45:0/1195584818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f829233da0 0x55f82929c790 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 >> 172.21.15.45:0/1195584818 conn(0x55f8291a82e0 msgr2=0x55f829235be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:49.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 shutdown_connections 2023-09-23T09:29:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 1 -- 172.21.15.45:0/1195584818 wait complete. 2023-09-23T09:29:49.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e71ef3b80 20 client.15588 trim_cache size 0 max 16384 2023-09-23T09:29:49.811 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:49.811 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2023-09-23T09:29:49.811 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "7549712f-6fd3-4691-b7d0-877c4aff605f", 2023-09-23T09:29:49.811 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2023-09-23T09:29:49.811 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3e480a1780 msgr2=0x7f3e480a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3e480a1780 0x7f3e480a3c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400dfd0 tx=0x7f3e5401a040 comp rx=0 tx=0).stop 2023-09-23T09:29:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e60101f40 msgr2=0x7f3e601b6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e60101f40 0x7f3e601b6de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000bf10 tx=0x7f3e500094c0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 shutdown_connections 2023-09-23T09:29:49.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e6010b360 0x7f3e601bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e60101f40 0x7f3e601b6de0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3e480a1780 0x7f3e480a3c40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 --2- 172.21.15.45:0/2144233712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e60102860 0x7f3e601b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.802+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 >> 172.21.15.45:0/2144233712 conn(0x7f3e600fb740 msgr2=0x7f3e6010d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.803+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 shutdown_connections 2023-09-23T09:29:49.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:49.803+0000 7f3e65d09700 1 -- 172.21.15.45:0/2144233712 wait complete. 2023-09-23T09:29:49.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:29:50.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.058+0000 7f55f9440700 1 -- 172.21.15.45:0/2409169659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 msgr2=0x7f55f4115160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.058+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409169659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 0x7f55f4115160 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b1a0 tx=0x7f55ec00b4b0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 -- 172.21.15.45:0/2409169659 shutdown_connections 2023-09-23T09:29:50.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409169659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 0x7f55f4115160 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409169659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f41097c0 0x7f55f4109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409169659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f410a170 0x7f55f4112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 -- 172.21.15.45:0/2409169659 >> 172.21.15.45:0/2409169659 conn(0x7f55f406d340 msgr2=0x7f55f406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:50.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 -- 172.21.15.45:0/2409169659 shutdown_connections 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 -- 172.21.15.45:0/2409169659 wait complete. 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.059+0000 7f55f9440700 1 Processor -- start 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f9440700 1 -- start start 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f9440700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f41097c0 0x7f55f41ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f9440700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f410a170 0x7f55f41aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f9440700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 0x7f55f41b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f9440700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f41b3550 con 0x7f55f4112d70 2023-09-23T09:29:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f9440700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f41b36c0 con 0x7f55f410a170 2023-09-23T09:29:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f9440700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f41b3880 con 0x7f55f41097c0 2023-09-23T09:29:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55ea7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f410a170 0x7f55f41aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55ea7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f410a170 0x7f55f41aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54344/0 (socket says 172.21.15.45:54344) 2023-09-23T09:29:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55ea7fc700 1 -- 172.21.15.45:0/2409382336 learned_addr learned my addr 172.21.15.45:0/2409382336 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f37fe700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 0x7f55f41b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.060+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f41097c0 0x7f55f41ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55ea7fc700 1 -- 172.21.15.45:0/2409382336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f41097c0 msgr2=0x7f55f41ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f41097c0 0x7f55f41ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55ea7fc700 1 -- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 msgr2=0x7f55f41b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 0x7f55f41b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55ea7fc700 1 -- 172.21.15.45:0/2409382336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec009d00 con 0x7f55f410a170 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f41097c0 0x7f55f41ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f410a170 0x7f55f41aeab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00d950 tx=0x7f55dc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.061+0000 7f55f0ff9700 1 -- 172.21.15.45:0/2409382336 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00f890 con 0x7f55f410a170 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.062+0000 7f55f0ff9700 1 -- 172.21.15.45:0/2409382336 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55dc010460 con 0x7f55f410a170 2023-09-23T09:29:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.062+0000 7f55f0ff9700 1 -- 172.21.15.45:0/2409382336 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00e610 con 0x7f55f410a170 2023-09-23T09:29:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.062+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f41b3b60 con 0x7f55f410a170 2023-09-23T09:29:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.062+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f41bb3a0 con 0x7f55f410a170 2023-09-23T09:29:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.065+0000 7f55f0ff9700 1 -- 172.21.15.45:0/2409382336 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7f55dc0105d0 con 0x7f55f410a170 2023-09-23T09:29:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.067+0000 7f560562ab80 1 Processor -- start 2023-09-23T09:29:50.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.067+0000 7f560562ab80 1 -- start start 2023-09-23T09:29:50.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f560562ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f560562ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168caed20 0x558168cb9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f560562ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cb9bd0 0x558168cbbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f560562ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168c17460 con 0x558168cae400 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f560562ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168c08c20 con 0x558168caed20 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f560562ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168c06540 con 0x558168cb9bd0 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55f2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168caed20 0x558168cb9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55ea7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cb9bd0 0x558168cbbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33000/0 (socket says 172.21.15.45:33000) 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55f37fe700 1 -- 172.21.15.45:0/1380448121 learned_addr learned my addr 172.21.15.45:0/1380448121 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55ea7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cb9bd0 0x558168cbbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35110/0 (socket says 172.21.15.45:35110) 2023-09-23T09:29:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55f0ff9700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55e00a1850 0x7f55e00a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55ea7fc700 1 -- 172.21.15.45:0/1380448121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168caed20 msgr2=0x558168cb9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/1380448121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168caed20 0x558168cb9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55ea7fc700 1 -- 172.21.15.45:0/1380448121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 msgr2=0x558168cae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.068+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/1380448121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.069+0000 7f55ea7fc700 1 -- 172.21.15.45:0/1380448121 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558168c648d0 con 0x558168cb9bd0 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.069+0000 7f55f37fe700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55e00a1850 0x7f55e00a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.069+0000 7f55f0ff9700 1 -- 172.21.15.45:0/2409382336 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f55dc0cd0e0 con 0x7f55f410a170 2023-09-23T09:29:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.069+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/1380448121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cb9bd0 0x558168cbbfc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0ca760 tx=0x7f55dc0651c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4da977ea2d1f1a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.069+0000 7f55e97fa700 1 -- 172.21.15.45:0/1380448121 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc099a10 con 0x558168cb9bd0 2023-09-23T09:29:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.069+0000 7f55e97fa700 1 -- 172.21.15.45:0/1380448121 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55dc099830 con 0x558168cb9bd0 2023-09-23T09:29:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.069+0000 7f55e97fa700 1 -- 172.21.15.45:0/1380448121 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc0656d0 con 0x558168cb9bd0 2023-09-23T09:29:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.070+0000 7f560562ab80 1 -- 172.21.15.45:0/1380448121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cb9bd0 msgr2=0x558168cbbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.070+0000 7f560562ab80 1 --2- 172.21.15.45:0/1380448121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cb9bd0 0x558168cbbfc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0ca760 tx=0x7f55dc0651c0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.070+0000 7f55f37fe700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55e00a1850 0x7f55e00a3d10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00bc00 tx=0x7f55ec00b4b0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.071+0000 7f560562ab80 1 -- 172.21.15.45:0/1380448121 shutdown_connections 2023-09-23T09:29:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.071+0000 7f560562ab80 1 --2- 172.21.15.45:0/1380448121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cb9bd0 0x558168cbbfc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.071+0000 7f560562ab80 1 --2- 172.21.15.45:0/1380448121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.071+0000 7f560562ab80 1 --2- 172.21.15.45:0/1380448121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168caed20 0x558168cb9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.071+0000 7f560562ab80 1 -- 172.21.15.45:0/1380448121 >> 172.21.15.45:0/1380448121 conn(0x558168c23260 msgr2=0x558168caa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.071+0000 7f560562ab80 1 -- 172.21.15.45:0/1380448121 shutdown_connections 2023-09-23T09:29:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.071+0000 7f560562ab80 1 -- 172.21.15.45:0/1380448121 wait complete. 2023-09-23T09:29:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 -- start start 2023-09-23T09:29:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 Processor -- start 2023-09-23T09:29:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cae400 0x558168cb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168caed20 0x558168cb6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cb6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168c01e30 con 0x558168caed20 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168b9e710 con 0x558168cb9bd0 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168b9e3b0 con 0x558168cae400 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f560562ab80 10 client.0 _ll_register_callbacks cb 0x5581689fb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f55ea7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168caed20 0x558168cb6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f55ea7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168caed20 0x558168cb6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33010/0 (socket says 172.21.15.45:33010) 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f55ea7fc700 1 -- 172.21.15.45:0/1472486572 learned_addr learned my addr 172.21.15.45:0/1472486572 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.072+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cae400 0x558168cb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55f37fe700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cb6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55f2ffd700 1 -- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 msgr2=0x558168cb6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cb6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55f2ffd700 1 -- 172.21.15.45:0/1472486572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168caed20 msgr2=0x558168cb6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168caed20 0x558168cb6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55f2ffd700 1 -- 172.21.15.45:0/1472486572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f41bb3a0 con 0x558168cae400 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168caed20 0x558168cb6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cae400 0x558168cb9590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55e4012d00 tx=0x7f55e400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b5d5bb20ee9eb5b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55da7fc700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55da7fc700 1 -- 172.21.15.45:0/1472486572 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4017070 con 0x558168cae400 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f55da7fc700 1 -- 172.21.15.45:0/1472486572 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f55e400fc30 con 0x558168cae400 2023-09-23T09:29:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.073+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558168ca2250 con 0x558168cae400 2023-09-23T09:29:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.074+0000 7f55da7fc700 1 -- 172.21.15.45:0/1472486572 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e400fdf0 con 0x558168cae400 2023-09-23T09:29:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.074+0000 7f560562ab80 10 client.25279 fetch_fsmap learned FSMap version 24 2023-09-23T09:29:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.074+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558168a2cde0 con 0x558168cae400 2023-09-23T09:29:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.075+0000 7f55da7fc700 1 -- 172.21.15.45:0/1472486572 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558168ca2250 con 0x558168cae400 2023-09-23T09:29:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.076+0000 7f55da7fc700 1 -- 172.21.15.45:0/1472486572 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f55e4003b50 con 0x558168cae400 2023-09-23T09:29:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.076+0000 7f560562ab80 10 client.25279 fetch_fsmap finished waiting for FSMap version 24 2023-09-23T09:29:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.076+0000 7f560562ab80 10 client.25279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.076+0000 7f560562ab80 20 client.25279 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.076+0000 7f560562ab80 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168cc9690 0x558168ccbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.076+0000 7f560562ab80 4 client.25279 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.076+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558168bfae10 con 0x558168cc9690 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.077+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168cc9690 0x558168ccbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.077+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168cc9690 0x558168ccbad0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7274d840e0ac3cf server_cookie=34e999aad52bb34 in_seq=0 out_seq=0 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.077+0000 7f55da7fc700 10 client.25279 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.078+0000 7f55da7fc700 1 -- 172.21.15.45:0/1472486572 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558168bfae10 con 0x558168cc9690 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.078+0000 7f55da7fc700 10 client.25279 handle_command_reply: tid=0 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.078+0000 7f560562ab80 1 client.25279 shutdown 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.078+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cae400 msgr2=0x558168cb9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.078+0000 7f560562ab80 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cae400 0x558168cb9590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55e4012d00 tx=0x7f55e400f6a0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 shutdown_connections 2023-09-23T09:29:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168cc9690 0x558168ccbad0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cb6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168cae400 0x558168cb9590 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 --2- 172.21.15.45:0/1472486572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168caed20 0x558168cb6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 >> 172.21.15.45:0/1472486572 conn(0x558168c23260 msgr2=0x558168caa380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f55da7fc700 0 client.25279 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 shutdown_connections 2023-09-23T09:29:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 1 -- 172.21.15.45:0/1472486572 wait complete. 2023-09-23T09:29:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.079+0000 7f560562ab80 20 client.25279 trim_cache size 0 max 16384 2023-09-23T09:29:50.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.102+0000 7f560562ab80 1 Processor -- start 2023-09-23T09:29:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.102+0000 7f560562ab80 1 -- start start 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f560562ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cae400 0x558168cb9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f560562ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cb1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f560562ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cb9bd0 0x558168cb20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f560562ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168a82920 con 0x558168cb9bd0 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f560562ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168ca9580 con 0x558168cae400 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f560562ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168cb2700 con 0x558168caed20 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cb9bd0 0x558168cb20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cb9bd0 0x558168cb20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33016/0 (socket says 172.21.15.45:33016) 2023-09-23T09:29:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.103+0000 7f55f37fe700 1 -- 172.21.15.45:0/2475424485 learned_addr learned my addr 172.21.15.45:0/2475424485 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55f37fe700 1 -- 172.21.15.45:0/2475424485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 msgr2=0x558168cb1b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55f37fe700 1 --2- 172.21.15.45:0/2475424485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cb1b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55f37fe700 1 -- 172.21.15.45:0/2475424485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cae400 msgr2=0x558168cb9130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55f37fe700 1 --2- 172.21.15.45:0/2475424485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cae400 0x558168cb9130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55f37fe700 1 -- 172.21.15.45:0/2475424485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558168a15c60 con 0x558168cb9bd0 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55f37fe700 1 --2- 172.21.15.45:0/2475424485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cb9bd0 0x558168cb20d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0076c0 tx=0x7f55ec00b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730f16187aacaad9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55e9ffb700 1 -- 172.21.15.45:0/2475424485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec010070 con 0x558168cb9bd0 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55e9ffb700 1 -- 172.21.15.45:0/2475424485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55ec011040 con 0x558168cb9bd0 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.104+0000 7f55e9ffb700 1 -- 172.21.15.45:0/2475424485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00bea0 con 0x558168cb9bd0 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.105+0000 7f560562ab80 1 -- 172.21.15.45:0/2475424485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cb9bd0 msgr2=0x558168cb20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.105+0000 7f560562ab80 1 --2- 172.21.15.45:0/2475424485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cb9bd0 0x558168cb20d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0076c0 tx=0x7f55ec00b100 comp rx=0 tx=0).stop 2023-09-23T09:29:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.105+0000 7f560562ab80 1 -- 172.21.15.45:0/2475424485 shutdown_connections 2023-09-23T09:29:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.105+0000 7f560562ab80 1 --2- 172.21.15.45:0/2475424485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cb9bd0 0x558168cb20d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.105+0000 7f560562ab80 1 --2- 172.21.15.45:0/2475424485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cae400 0x558168cb9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.105+0000 7f560562ab80 1 --2- 172.21.15.45:0/2475424485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cb1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.105+0000 7f560562ab80 1 -- 172.21.15.45:0/2475424485 >> 172.21.15.45:0/2475424485 conn(0x558168c23260 msgr2=0x558168ccb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.106+0000 7f560562ab80 1 -- 172.21.15.45:0/2475424485 shutdown_connections 2023-09-23T09:29:50.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.106+0000 7f560562ab80 1 -- 172.21.15.45:0/2475424485 wait complete. 2023-09-23T09:29:50.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 -- start start 2023-09-23T09:29:50.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 Processor -- start 2023-09-23T09:29:50.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cbe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cbebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cbf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168d1c520 con 0x558168cae400 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168d1c690 con 0x558168cb9bd0 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.107+0000 7f560562ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558168d1c850 con 0x558168caed20 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f560562ab80 10 client.0 _ll_register_callbacks cb 0x7f55e40127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cbf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cbf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54376/0 (socket says 172.21.15.45:54376) 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55f37fe700 1 -- 172.21.15.45:0/528797786 learned_addr learned my addr 172.21.15.45:0/528797786 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cbebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cbe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55ea7fc700 1 -- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 msgr2=0x558168cbf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cbf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55ea7fc700 1 -- 172.21.15.45:0/528797786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 msgr2=0x558168cbe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cbe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55ea7fc700 1 -- 172.21.15.45:0/528797786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f41086b0 con 0x558168caed20 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cbe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55f37fe700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cbf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55ea7fc700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cbebe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55dc099bd0 tx=0x7f55dc099b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f80eb9df3d8546a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.108+0000 7f55da7fc700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.109+0000 7f55da7fc700 1 -- 172.21.15.45:0/528797786 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc09a5d0 con 0x558168caed20 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.109+0000 7f55da7fc700 1 -- 172.21.15.45:0/528797786 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f55dc06d2d0 con 0x558168caed20 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.109+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558168cbc3f0 con 0x558168caed20 2023-09-23T09:29:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.109+0000 7f55da7fc700 1 -- 172.21.15.45:0/528797786 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55dc099ed0 con 0x558168caed20 2023-09-23T09:29:50.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 10 client.25282 fetch_fsmap learned FSMap version 24 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558168a53250 con 0x558168caed20 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f55da7fc700 1 -- 172.21.15.45:0/528797786 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55dc0b1070 con 0x558168caed20 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f55da7fc700 1 -- 172.21.15.45:0/528797786 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f55dc06b0c0 con 0x558168caed20 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 10 client.25282 fetch_fsmap finished waiting for FSMap version 24 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 20 client.25282 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168d1dbb0 0x558168d1ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 4 client.25282 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f55dc099830 con 0x558168d1dbb0 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.110+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168d1dbb0 0x558168d1ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.111+0000 7f55f2ffd700 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168d1dbb0 0x558168d1ffd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225dd598f36d6e21 server_cookie=d210a6db3e35b45 in_seq=0 out_seq=0 2023-09-23T09:29:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.111+0000 7f55da7fc700 10 client.25282 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.111+0000 7f55da7fc700 1 -- 172.21.15.45:0/528797786 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x7f55dc099830 con 0x558168d1dbb0 2023-09-23T09:29:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.111+0000 7f55da7fc700 10 client.25282 handle_command_reply: tid=0 2023-09-23T09:29:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.111+0000 7f560562ab80 1 client.25282 shutdown 2023-09-23T09:29:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.112+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 msgr2=0x558168cbebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.112+0000 7f560562ab80 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cbebe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55dc099bd0 tx=0x7f55dc099b30 comp rx=0 tx=0).stop 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 shutdown_connections 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558168d1dbb0 0x558168d1ffd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558168cb9bd0 0x558168cbf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558168cae400 0x558168cbe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 --2- 172.21.15.45:0/528797786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558168caed20 0x558168cbebe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 >> 172.21.15.45:0/528797786 conn(0x558168c23260 msgr2=0x558168cb1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f55da7fc700 0 client.25282 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 shutdown_connections 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 1 -- 172.21.15.45:0/528797786 wait complete. 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.113+0000 7f560562ab80 20 client.25282 trim_cache size 0 max 16384 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout: "7549712f-6fd3-4691-b7d0-877c4aff605f": { 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "7549712f-6fd3-4691-b7d0-877c4aff605f", 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:29:50.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:29:50.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:29:50.119 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:29:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55e00a1850 msgr2=0x7f55e00a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55e00a1850 0x7f55e00a3d10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00bc00 tx=0x7f55ec00b4b0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f410a170 msgr2=0x7f55f41aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f410a170 0x7f55f41aeab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00d950 tx=0x7f55dc00dc60 comp rx=0 tx=0).stop 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 shutdown_connections 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55e00a1850 0x7f55e00a3d10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f4112d70 0x7f55f41b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f41097c0 0x7f55f41ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 --2- 172.21.15.45:0/2409382336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f410a170 0x7f55f41aeab0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00d950 tx=0x7f55dc00dc60 comp rx=0 tx=0).stop 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 >> 172.21.15.45:0/2409382336 conn(0x7f55f406d340 msgr2=0x7f55f4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 shutdown_connections 2023-09-23T09:29:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:29:50.115+0000 7f55f9440700 1 -- 172.21.15.45:0/2409382336 wait complete. 2023-09-23T09:29:50.137 INFO:tasks.cephfs.filesystem:scrub status for tag:7549712f-6fd3-4691-b7d0-877c4aff605f - {'path': '/', 'tag': '7549712f-6fd3-4691-b7d0-877c4aff605f', 'options': 'recursive,force'} 2023-09-23T09:29:50.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:50.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub queued for path: / 2023-09-23T09:29:50.323 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:29:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:29:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:29:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:29:50.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:50 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:50.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub queued for path: / 2023-09-23T09:29:50.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:29:50.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:29:50.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:29:50.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:29:50.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:50 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:50.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub queued for path: / 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:50.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:50 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:29:51.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:51 smithi045 ceph-mon[98627]: pgmap v15: 97 pgs: 97 active+clean; 448 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 67 MiB/s wr, 290 op/s 2023-09-23T09:29:51.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:51 smithi123 ceph-mon[103577]: pgmap v15: 97 pgs: 97 active+clean; 448 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 67 MiB/s wr, 290 op/s 2023-09-23T09:29:51.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:51 smithi098 ceph-mon[102386]: pgmap v15: 97 pgs: 97 active+clean; 448 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 67 MiB/s wr, 290 op/s 2023-09-23T09:29:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 5.0 s, 3274.4 tps, lat 30.039 ms stddev 35.948 2023-09-23T09:29:53.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:53 smithi123 ceph-mon[103577]: pgmap v16: 97 pgs: 97 active+clean; 707 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 84 MiB/s wr, 469 op/s 2023-09-23T09:29:53.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:53 smithi045 ceph-mon[98627]: pgmap v16: 97 pgs: 97 active+clean; 707 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 84 MiB/s wr, 469 op/s 2023-09-23T09:29:53.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:53 smithi098 ceph-mon[102386]: pgmap v16: 97 pgs: 97 active+clean; 707 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 84 MiB/s wr, 469 op/s 2023-09-23T09:29:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.771+0000 7fd2ca072700 1 -- 172.21.15.45:0/9130023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c410f570 msgr2=0x7fd2c4107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.771+0000 7fd2ca072700 1 --2- 172.21.15.45:0/9130023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c410f570 0x7fd2c4107e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009990 tx=0x7fd2b4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.772+0000 7fd2ca072700 1 -- 172.21.15.45:0/9130023 shutdown_connections 2023-09-23T09:29:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.772+0000 7fd2ca072700 1 --2- 172.21.15.45:0/9130023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c411a600 0x7fd2c411c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.772+0000 7fd2ca072700 1 --2- 172.21.15.45:0/9130023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c410f570 0x7fd2c4107e20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.772+0000 7fd2ca072700 1 --2- 172.21.15.45:0/9130023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c410f9e0 0x7fd2c4108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.772+0000 7fd2ca072700 1 -- 172.21.15.45:0/9130023 >> 172.21.15.45:0/9130023 conn(0x7fd2c406cee0 msgr2=0x7fd2c406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.772+0000 7fd2ca072700 1 -- 172.21.15.45:0/9130023 shutdown_connections 2023-09-23T09:29:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.772+0000 7fd2ca072700 1 -- 172.21.15.45:0/9130023 wait complete. 2023-09-23T09:29:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.773+0000 7fd2ca072700 1 Processor -- start 2023-09-23T09:29:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.773+0000 7fd2ca072700 1 -- start start 2023-09-23T09:29:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.774+0000 7fd2ca072700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c410f570 0x7fd2c4119a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.774+0000 7fd2ca072700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c410f9e0 0x7fd2c4119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.774+0000 7fd2ca072700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c411a600 0x7fd2c41138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.774+0000 7fd2ca072700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41140d0 con 0x7fd2c411a600 2023-09-23T09:29:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.774+0000 7fd2ca072700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4114240 con 0x7fd2c410f570 2023-09-23T09:29:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.774+0000 7fd2ca072700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4114400 con 0x7fd2c410f9e0 2023-09-23T09:29:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c410f570 0x7fd2c4119a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c410f570 0x7fd2c4119a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54386/0 (socket says 172.21.15.45:54386) 2023-09-23T09:29:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2007072885 learned_addr learned my addr 172.21.15.45:0/2007072885 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2007072885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c410f9e0 msgr2=0x7fd2c4119fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:29:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c410f9e0 0x7fd2c4119fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c411a600 msgr2=0x7fd2c41138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c411a600 0x7fd2c41138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2007072885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40095f0 con 0x7fd2c410f570 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.775+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c410f570 0x7fd2c4119a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005950 tx=0x7fd2b4003bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.776+0000 7fd2c0ff9700 1 -- 172.21.15.45:0/2007072885 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b401d070 con 0x7fd2c410f570 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.776+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c4114620 con 0x7fd2c410f570 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.776+0000 7fd2c0ff9700 1 -- 172.21.15.45:0/2007072885 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b400f460 con 0x7fd2c410f570 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.776+0000 7fd2c0ff9700 1 -- 172.21.15.45:0/2007072885 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017650 con 0x7fd2c410f570 2023-09-23T09:29:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.776+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c41bb4b0 con 0x7fd2c410f570 2023-09-23T09:29:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.778+0000 7fd2c0ff9700 1 -- 172.21.15.45:0/2007072885 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) v1 ==== 136368+0+0 (secure 0 0 0) 0x7fd2b400f5d0 con 0x7fd2c410f570 2023-09-23T09:29:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.779+0000 7fd2c0ff9700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd2ac0a1860 0x7fd2ac0a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.779+0000 7fd2c0ff9700 1 -- 172.21.15.45:0/2007072885 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd2b40ce680 con 0x7fd2c410f570 2023-09-23T09:29:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.780+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd2ac0a1860 0x7fd2ac0a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.781+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd2ac0a1860 0x7fd2ac0a3d20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4115290 tx=0x7fd2b800b3f0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 Processor -- start 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 -- start start 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53df40 0x5606cd53e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53e860 0x5606cd5491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd4a7400 con 0x5606cd549710 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd498bc0 con 0x5606cd53e860 2023-09-23T09:29:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.782+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd4964e0 con 0x5606cd53df40 2023-09-23T09:29:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53df40 0x5606cd53e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53df40 0x5606cd53e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35148/0 (socket says 172.21.15.45:35148) 2023-09-23T09:29:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2002530712 learned_addr learned my addr 172.21.15.45:0/2002530712 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53e860 0x5606cd5491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c3fff700 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2002530712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53e860 msgr2=0x5606cd5491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53e860 0x5606cd5491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2002530712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 msgr2=0x5606cd54bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2002530712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606cd4f4870 con 0x5606cd53df40 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53df40 0x5606cd53e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2b409b430 tx=0x7fd2b4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d6ea4e4007a649 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.783+0000 7fd2a9ffb700 1 -- 172.21.15.45:0/2002530712 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4020e50 con 0x5606cd53df40 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.784+0000 7fd2a9ffb700 1 -- 172.21.15.45:0/2002530712 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b409bba0 con 0x5606cd53df40 2023-09-23T09:29:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.784+0000 7fd2a9ffb700 1 -- 172.21.15.45:0/2002530712 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40cbf50 con 0x5606cd53df40 2023-09-23T09:29:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.785+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2002530712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53df40 msgr2=0x5606cd53e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.785+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53df40 0x5606cd53e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2b409b430 tx=0x7fd2b4009310 comp rx=0 tx=0).stop 2023-09-23T09:29:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.785+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2002530712 shutdown_connections 2023-09-23T09:29:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.785+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.785+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53df40 0x5606cd53e320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.785+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2002530712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53e860 0x5606cd5491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.785+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2002530712 >> 172.21.15.45:0/2002530712 conn(0x5606cd4b3200 msgr2=0x5606cd53a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.786+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2002530712 shutdown_connections 2023-09-23T09:29:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.786+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2002530712 wait complete. 2023-09-23T09:29:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 -- start start 2023-09-23T09:29:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 Processor -- start 2023-09-23T09:29:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd549af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd54a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd491dd0 con 0x5606cd549710 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd42e6b0 con 0x5606cd53df40 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.787+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd42e350 con 0x5606cd53e860 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2d625cb80 10 client.0 _ll_register_callbacks cb 0x5606cd28b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd54a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd549af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd54a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35158/0 (socket says 172.21.15.45:35158) 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c2ffd700 1 -- 172.21.15.45:0/1147938997 learned_addr learned my addr 172.21.15.45:0/1147938997 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33050/0 (socket says 172.21.15.45:33050) 2023-09-23T09:29:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c3fff700 1 -- 172.21.15.45:0/1147938997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 msgr2=0x5606cd54a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c3fff700 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd54a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c3fff700 1 -- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 msgr2=0x5606cd549af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c3fff700 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd549af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.788+0000 7fd2c3fff700 1 -- 172.21.15.45:0/1147938997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c41bb4b0 con 0x5606cd549710 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.789+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd549af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.789+0000 7fd2c3fff700 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54a570 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd2c406a190 tx=0x7fd2bc007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ade2714f2376848 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.789+0000 7fd2a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.789+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/1147938997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc019070 con 0x5606cd549710 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.789+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/1147938997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd2bc003a00 con 0x5606cd549710 2023-09-23T09:29:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.790+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606cd532090 con 0x5606cd549710 2023-09-23T09:29:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.790+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/1147938997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2bc003bc0 con 0x5606cd549710 2023-09-23T09:29:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.790+0000 7fd2d625cb80 10 client.15615 fetch_fsmap learned FSMap version 24 2023-09-23T09:29:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.790+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5606cd2bcd80 con 0x5606cd549710 2023-09-23T09:29:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/1147938997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606cd532090 con 0x5606cd549710 2023-09-23T09:29:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/1147938997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2bc01c020 con 0x5606cd549710 2023-09-23T09:29:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2d625cb80 10 client.15615 fetch_fsmap finished waiting for FSMap version 24 2023-09-23T09:29:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2d625cb80 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2d625cb80 20 client.15615 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd558f80 0x5606cd55b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2d625cb80 4 client.15615 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:29:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.791+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606cd48adb0 con 0x5606cd558f80 2023-09-23T09:29:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.792+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd558f80 0x5606cd55b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.792+0000 7fd2a2ffd700 10 client.15615 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.792+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd558f80 0x5606cd55b3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eeb4e4d991856fb server_cookie=d2915e014123596a in_seq=0 out_seq=0 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/1147938997 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5606cd48adb0 con 0x5606cd558f80 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2a2ffd700 10 client.15615 handle_command_reply: tid=0 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 client.15615 shutdown 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 msgr2=0x5606cd54a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54a570 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd2c406a190 tx=0x7fd2bc007af0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 shutdown_connections 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd558f80 0x5606cd55b3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd549710 0x5606cd54a570 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2a2ffd700 0 client.15615 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd549af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/1147938997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd54a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 >> 172.21.15.45:0/1147938997 conn(0x5606cd4b3200 msgr2=0x5606cd539fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 shutdown_connections 2023-09-23T09:29:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.793+0000 7fd2d625cb80 1 -- 172.21.15.45:0/1147938997 wait complete. 2023-09-23T09:29:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.794+0000 7fd2d625cb80 20 client.15615 trim_cache size 0 max 16384 2023-09-23T09:29:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.813+0000 7fd2d625cb80 1 Processor -- start 2023-09-23T09:29:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.813+0000 7fd2d625cb80 1 -- start start 2023-09-23T09:29:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53df40 0x5606cd54db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd547f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd549710 0x5606cd5459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd3128c0 con 0x5606cd53df40 2023-09-23T09:29:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd534600 con 0x5606cd549710 2023-09-23T09:29:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd548de0 con 0x5606cd53e860 2023-09-23T09:29:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53df40 0x5606cd54db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53df40 0x5606cd54db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33056/0 (socket says 172.21.15.45:33056) 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2c37fe700 1 -- 172.21.15.45:0/4054407011 learned_addr learned my addr 172.21.15.45:0/4054407011 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.814+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd547f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.815+0000 7fd2c2ffd700 1 -- 172.21.15.45:0/4054407011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd549710 msgr2=0x5606cd5459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.815+0000 7fd2c3fff700 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd549710 0x5606cd5459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.815+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd549710 0x5606cd5459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.815+0000 7fd2c2ffd700 1 -- 172.21.15.45:0/4054407011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53df40 msgr2=0x5606cd54db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.815+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53df40 0x5606cd54db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.815+0000 7fd2c2ffd700 1 -- 172.21.15.45:0/4054407011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606cd2a5c00 con 0x5606cd53e860 2023-09-23T09:29:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.815+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd547f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800bfd0 tx=0x7fd2b8010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107f53736c828c76 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.816+0000 7fd2aa7fc700 1 -- 172.21.15.45:0/4054407011 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8010bb0 con 0x5606cd53e860 2023-09-23T09:29:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.816+0000 7fd2aa7fc700 1 -- 172.21.15.45:0/4054407011 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b8010d10 con 0x5606cd53e860 2023-09-23T09:29:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.816+0000 7fd2aa7fc700 1 -- 172.21.15.45:0/4054407011 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b801e660 con 0x5606cd53e860 2023-09-23T09:29:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.818+0000 7fd2d625cb80 1 -- 172.21.15.45:0/4054407011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 msgr2=0x5606cd547f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.818+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd547f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800bfd0 tx=0x7fd2b8010000 comp rx=0 tx=0).stop 2023-09-23T09:29:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.818+0000 7fd2d625cb80 1 -- 172.21.15.45:0/4054407011 shutdown_connections 2023-09-23T09:29:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.818+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd549710 0x5606cd5459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.818+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53df40 0x5606cd54db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.818+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/4054407011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd53e860 0x5606cd547f20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.818+0000 7fd2d625cb80 1 -- 172.21.15.45:0/4054407011 >> 172.21.15.45:0/4054407011 conn(0x5606cd4b3200 msgr2=0x5606cd543a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.819+0000 7fd2d625cb80 1 -- 172.21.15.45:0/4054407011 shutdown_connections 2023-09-23T09:29:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.819+0000 7fd2d625cb80 1 -- 172.21.15.45:0/4054407011 wait complete. 2023-09-23T09:29:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.820+0000 7fd2d625cb80 1 -- start start 2023-09-23T09:29:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.820+0000 7fd2d625cb80 1 Processor -- start 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd5a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53e860 0x5606cd5a1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2d625cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd549710 0x5606cd5abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd5ac300 con 0x5606cd53e860 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd5ac440 con 0x5606cd53df40 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2d625cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cd5ac580 con 0x5606cd549710 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2d625cb80 10 client.0 _ll_register_callbacks cb 0x7fd2bc009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd5a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd5a0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54412/0 (socket says 172.21.15.45:54412) 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2299363546 learned_addr learned my addr 172.21.15.45:0/2299363546 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:29:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2c3fff700 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd549710 0x5606cd5abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2299363546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd549710 msgr2=0x5606cd5abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:29:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.821+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd549710 0x5606cd5abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.822+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2299363546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53e860 msgr2=0x5606cd5a1470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:29:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.822+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53e860 0x5606cd5a1470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.822+0000 7fd2c37fe700 1 -- 172.21.15.45:0/2299363546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b800d970 con 0x5606cd53df40 2023-09-23T09:29:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.823+0000 7fd2c37fe700 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd5a0f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40cbbe0 tx=0x7fd2b409bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83300fbf83f86e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:29:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.823+0000 7fd2a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:29:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/2299363546 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b409f370 con 0x5606cd53df40 2023-09-23T09:29:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/2299363546 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd2b40b2070 con 0x5606cd53df40 2023-09-23T09:29:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b409bba0 con 0x5606cd53df40 2023-09-23T09:29:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/2299363546 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b409f5f0 con 0x5606cd53df40 2023-09-23T09:29:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2d625cb80 10 client.25286 fetch_fsmap learned FSMap version 24 2023-09-23T09:29:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5606cd2e31f0 con 0x5606cd53df40 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/2299363546 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b409bba0 con 0x5606cd53df40 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/2299363546 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2b409a990 con 0x5606cd53df40 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2d625cb80 10 client.25286 fetch_fsmap finished waiting for FSMap version 24 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2d625cb80 10 client.25286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.824+0000 7fd2d625cb80 20 client.25286 populate_metadata read hostname 'smithi045' 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.825+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd5ad950 0x5606cd5afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.825+0000 7fd2d625cb80 4 client.25286 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:29:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.825+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5606cd48aad0 con 0x5606cd5ad950 2023-09-23T09:29:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.825+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd5ad950 0x5606cd5afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:29:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.826+0000 7fd2a2ffd700 10 client.25286 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.826+0000 7fd2c2ffd700 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd5ad950 0x5606cd5afd70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc62d8360938f009 server_cookie=2e25c0342844ddaa in_seq=0 out_seq=0 2023-09-23T09:29:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2a2ffd700 1 -- 172.21.15.45:0/2299363546 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5606cd48aad0 con 0x5606cd5ad950 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2a2ffd700 10 client.25286 handle_command_reply: tid=0 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 client.25286 shutdown 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 msgr2=0x5606cd5a0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd5a0f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40cbbe0 tx=0x7fd2b409bd10 comp rx=0 tx=0).stop 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 shutdown_connections 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5606cd5ad950 0x5606cd5afd70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606cd549710 0x5606cd5abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606cd53df40 0x5606cd5a0f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2a2ffd700 0 client.25286 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 --2- 172.21.15.45:0/2299363546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606cd53e860 0x5606cd5a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.853+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 >> 172.21.15.45:0/2299363546 conn(0x5606cd4b3200 msgr2=0x5606cd539590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 shutdown_connections 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2d625cb80 1 -- 172.21.15.45:0/2299363546 wait complete. 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2d625cb80 20 client.25286 trim_cache size 0 max 16384 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:29:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:29:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd2ac0a1860 msgr2=0x7fd2ac0a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd2ac0a1860 0x7fd2ac0a3d20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4115290 tx=0x7fd2b800b3f0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c410f570 msgr2=0x7fd2c4119a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:29:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c410f570 0x7fd2c4119a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005950 tx=0x7fd2b4003bf0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 shutdown_connections 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd2ac0a1860 0x7fd2ac0a3d20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c411a600 0x7fd2c41138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c410f570 0x7fd2c4119a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 --2- 172.21.15.45:0/2007072885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c410f9e0 0x7fd2c4119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 >> 172.21.15.45:0/2007072885 conn(0x7fd2c406cee0 msgr2=0x7fd2c411a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 shutdown_connections 2023-09-23T09:29:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:29:54.854+0000 7fd2ca072700 1 -- 172.21.15.45:0/2007072885 wait complete. 2023-09-23T09:29:55.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:55 smithi123 ceph-mon[103577]: pgmap v17: 97 pgs: 97 active+clean; 1.0 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 89 MiB/s wr, 510 op/s 2023-09-23T09:29:55.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:55 smithi098 ceph-mon[102386]: pgmap v17: 97 pgs: 97 active+clean; 1.0 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 89 MiB/s wr, 510 op/s 2023-09-23T09:29:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:55 smithi045 ceph-mon[98627]: pgmap v17: 97 pgs: 97 active+clean; 1.0 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 89 MiB/s wr, 510 op/s 2023-09-23T09:29:56.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:56 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:56.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:56 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:56.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:56 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 10.0 s, 3813.0 tps, lat 26.299 ms stddev 44.723 2023-09-23T09:29:57.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:57 smithi098 ceph-mon[102386]: pgmap v18: 97 pgs: 97 active+clean; 1.0 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 82 MiB/s wr, 413 op/s 2023-09-23T09:29:57.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:57 smithi045 ceph-mon[98627]: pgmap v18: 97 pgs: 97 active+clean; 1.0 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 82 MiB/s wr, 413 op/s 2023-09-23T09:29:57.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:57 smithi123 ceph-mon[103577]: pgmap v18: 97 pgs: 97 active+clean; 1.0 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 82 MiB/s wr, 413 op/s 2023-09-23T09:29:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:58 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:58.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:58 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:58.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:58 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:29:59.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:29:59 smithi098 ceph-mon[102386]: pgmap v19: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 137 MiB/s wr, 1.11k op/s 2023-09-23T09:29:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:29:59 smithi045 ceph-mon[98627]: pgmap v19: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 137 MiB/s wr, 1.11k op/s 2023-09-23T09:29:59.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:29:59 smithi123 ceph-mon[103577]: pgmap v19: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 137 MiB/s wr, 1.11k op/s 2023-09-23T09:30:00.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:30:00 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:30:00.258+0000 7f87da6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:30:00.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:30:00 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:30:00.258+0000 7f87da6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:30:00.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:30:00 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:30:00.258+0000 7f87da6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:30:00.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:30:00 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:30:00.258+0000 7f87da6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:30:00.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:30:00 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:30:00.258+0000 7f87da6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T09:30:00.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:00 smithi045 ceph-mon[98627]: overall HEALTH_OK 2023-09-23T09:30:00.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:00 smithi098 ceph-mon[102386]: overall HEALTH_OK 2023-09-23T09:30:00.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:00 smithi123 ceph-mon[103577]: overall HEALTH_OK 2023-09-23T09:30:01.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:01 smithi098 ceph-mon[102386]: pgmap v20: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 90 MiB/s wr, 1.08k op/s 2023-09-23T09:30:01.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:01 smithi045 ceph-mon[98627]: pgmap v20: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 90 MiB/s wr, 1.08k op/s 2023-09-23T09:30:01.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:01 smithi123 ceph-mon[103577]: pgmap v20: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 90 MiB/s wr, 1.08k op/s 2023-09-23T09:30:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 15.0 s, 3925.9 tps, lat 25.410 ms stddev 54.349 2023-09-23T09:30:03.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:03 smithi098 ceph-mon[102386]: pgmap v21: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 103 MiB/s wr, 1.70k op/s 2023-09-23T09:30:03.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:03 smithi098 ceph-mon[102386]: mgrmap e24: x(active, since 36s), standbys: z, y 2023-09-23T09:30:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:03 smithi045 ceph-mon[98627]: pgmap v21: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 103 MiB/s wr, 1.70k op/s 2023-09-23T09:30:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:03 smithi045 ceph-mon[98627]: mgrmap e24: x(active, since 36s), standbys: z, y 2023-09-23T09:30:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:03 smithi123 ceph-mon[103577]: pgmap v21: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 103 MiB/s wr, 1.70k op/s 2023-09-23T09:30:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:03 smithi123 ceph-mon[103577]: mgrmap e24: x(active, since 36s), standbys: z, y 2023-09-23T09:30:04.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:04 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:30:04.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:04 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:04.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:04 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:30:04.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:04 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:04.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:04 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:30:04.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:04 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:05 smithi098 ceph-mon[102386]: pgmap v22: 97 pgs: 97 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 104 MiB/s wr, 2.47k op/s 2023-09-23T09:30:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:05 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:05 smithi123 ceph-mon[103577]: pgmap v22: 97 pgs: 97 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 104 MiB/s wr, 2.47k op/s 2023-09-23T09:30:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:05 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:05.839 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:05 smithi045 ceph-mon[98627]: pgmap v22: 97 pgs: 97 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 104 MiB/s wr, 2.47k op/s 2023-09-23T09:30:05.839 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:05 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 20.0 s, 4342.4 tps, lat 23.033 ms stddev 27.959 2023-09-23T09:30:07.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:07 smithi098 ceph-mon[102386]: pgmap v23: 97 pgs: 97 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 86 MiB/s wr, 2.26k op/s 2023-09-23T09:30:07.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:07 smithi045 ceph-mon[98627]: pgmap v23: 97 pgs: 97 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 86 MiB/s wr, 2.26k op/s 2023-09-23T09:30:07.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:07 smithi123 ceph-mon[103577]: pgmap v23: 97 pgs: 97 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 86 MiB/s wr, 2.26k op/s 2023-09-23T09:30:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.042+0000 7faea1f9e700 1 -- 172.21.15.45:0/2641577340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c1008c0 msgr2=0x7fae9c104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.042+0000 7faea1f9e700 1 --2- 172.21.15.45:0/2641577340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c1008c0 0x7fae9c104900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009990 tx=0x7fae8c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.042+0000 7faea1f9e700 1 -- 172.21.15.45:0/2641577340 shutdown_connections 2023-09-23T09:30:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.042+0000 7faea1f9e700 1 --2- 172.21.15.45:0/2641577340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c105030 0x7fae9c10cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.042+0000 7faea1f9e700 1 --2- 172.21.15.45:0/2641577340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c0fff10 0x7fae9c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.042+0000 7faea1f9e700 1 --2- 172.21.15.45:0/2641577340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c1008c0 0x7fae9c104900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.042+0000 7faea1f9e700 1 -- 172.21.15.45:0/2641577340 >> 172.21.15.45:0/2641577340 conn(0x7fae9c0fb750 msgr2=0x7fae9c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.043+0000 7faea1f9e700 1 -- 172.21.15.45:0/2641577340 shutdown_connections 2023-09-23T09:30:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.043+0000 7faea1f9e700 1 -- 172.21.15.45:0/2641577340 wait complete. 2023-09-23T09:30:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.044+0000 7faea1f9e700 1 Processor -- start 2023-09-23T09:30:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.044+0000 7faea1f9e700 1 -- start start 2023-09-23T09:30:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7faea1f9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c0fff10 0x7fae9c10d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7faea1f9e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c105030 0x7fae9c10d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7faea1f9e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c10ddb0 0x7fae9c1a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7faea1f9e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1a9d80 con 0x7fae9c0fff10 2023-09-23T09:30:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7faea1f9e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1a9ec0 con 0x7fae9c10ddb0 2023-09-23T09:30:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7faea1f9e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1aa030 con 0x7fae9c105030 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c0fff10 0x7fae9c10d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c105030 0x7fae9c10d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c10ddb0 0x7fae9c1a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c0fff10 0x7fae9c10d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33092/0 (socket says 172.21.15.45:33092) 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9b7fe700 1 -- 172.21.15.45:0/3302283561 learned_addr learned my addr 172.21.15.45:0/3302283561 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9affd700 1 -- 172.21.15.45:0/3302283561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c10ddb0 msgr2=0x7fae9c1a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9affd700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c10ddb0 0x7fae9c1a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9affd700 1 -- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c0fff10 msgr2=0x7fae9c10d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9affd700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c0fff10 0x7fae9c10d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.045+0000 7fae9affd700 1 -- 172.21.15.45:0/3302283561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94009c40 con 0x7fae9c105030 2023-09-23T09:30:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.046+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c10ddb0 0x7fae9c1a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.048+0000 7fae9affd700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c105030 0x7fae9c10d870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0098e0 tx=0x7fae8c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.048+0000 7fae98ff9700 1 -- 172.21.15.45:0/3302283561 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01d070 con 0x7fae9c105030 2023-09-23T09:30:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.048+0000 7fae98ff9700 1 -- 172.21.15.45:0/3302283561 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae8c021410 con 0x7fae9c105030 2023-09-23T09:30:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.048+0000 7fae98ff9700 1 -- 172.21.15.45:0/3302283561 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c017650 con 0x7fae9c105030 2023-09-23T09:30:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.048+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c0095f0 con 0x7fae9c105030 2023-09-23T09:30:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.048+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c1aa540 con 0x7fae9c105030 2023-09-23T09:30:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.052+0000 7fae98ff9700 1 -- 172.21.15.45:0/3302283561 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7fae8c021580 con 0x7fae9c105030 2023-09-23T09:30:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.053+0000 7fae98ff9700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae840a1900 0x7fae840a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.053+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae840a1900 0x7fae840a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.053+0000 7fae98ff9700 1 -- 172.21.15.45:0/3302283561 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae8c0ce390 con 0x7fae9c105030 2023-09-23T09:30:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.053+0000 7faeae188b80 1 Processor -- start 2023-09-23T09:30:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7faeae188b80 1 -- start start 2023-09-23T09:30:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7faeae188b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b430542320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7faeae188b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 0x55b43054d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7faeae188b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b43054fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7faeae188b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4304ab400 con 0x55b430541f40 2023-09-23T09:30:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7faeae188b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43049cbc0 con 0x55b43054d710 2023-09-23T09:30:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7faeae188b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43049a4e0 con 0x55b430542860 2023-09-23T09:30:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.054+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae840a1900 0x7fae840a3dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae90009d20 tx=0x7fae90009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b430542320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b430542320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33100/0 (socket says 172.21.15.45:33100) 2023-09-23T09:30:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9b7fe700 1 -- 172.21.15.45:0/2638539152 learned_addr learned my addr 172.21.15.45:0/2638539152 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9bfff700 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b43054fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9bfff700 1 -- 172.21.15.45:0/2638539152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 msgr2=0x55b43054d1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:30:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9bfff700 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 0x55b43054d1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9bfff700 1 -- 172.21.15.45:0/2638539152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 msgr2=0x55b430542320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9bfff700 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b430542320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9bfff700 1 -- 172.21.15.45:0/2638539152 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4304f8870 con 0x55b43054d710 2023-09-23T09:30:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.055+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b430542320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.056+0000 7fae9bfff700 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b43054fb00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae940094d0 tx=0x7fae94009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc10741c482ffa6e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.056+0000 7fae81ffb700 1 -- 172.21.15.45:0/2638539152 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94004bb0 con 0x55b43054d710 2023-09-23T09:30:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7fae81ffb700 1 -- 172.21.15.45:0/2638539152 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae9400f040 con 0x55b43054d710 2023-09-23T09:30:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7fae81ffb700 1 -- 172.21.15.45:0/2638539152 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940176c0 con 0x55b43054d710 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7faeae188b80 1 -- 172.21.15.45:0/2638539152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 msgr2=0x55b43054fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7faeae188b80 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b43054fb00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae940094d0 tx=0x7fae94009920 comp rx=0 tx=0).stop 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7faeae188b80 1 -- 172.21.15.45:0/2638539152 shutdown_connections 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7faeae188b80 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b43054fb00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7faeae188b80 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b430542320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7faeae188b80 1 --2- 172.21.15.45:0/2638539152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 0x55b43054d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.057+0000 7faeae188b80 1 -- 172.21.15.45:0/2638539152 >> 172.21.15.45:0/2638539152 conn(0x55b4304b7200 msgr2=0x55b43053e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.058+0000 7faeae188b80 1 -- 172.21.15.45:0/2638539152 shutdown_connections 2023-09-23T09:30:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.058+0000 7faeae188b80 1 -- 172.21.15.45:0/2638539152 wait complete. 2023-09-23T09:30:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.058+0000 7faeae188b80 1 -- start start 2023-09-23T09:30:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.058+0000 7faeae188b80 1 Processor -- start 2023-09-23T09:30:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7faeae188b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b43054a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7faeae188b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430542860 0x55b43054aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7faeae188b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43054d710 0x55b43054b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7faeae188b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b430495dd0 con 0x55b430541f40 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7faeae188b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4304326b0 con 0x55b430542860 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7faeae188b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b430432350 con 0x55b43054d710 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7faeae188b80 10 client.0 _ll_register_callbacks cb 0x55b43028f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b43054a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b43054a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33104/0 (socket says 172.21.15.45:33104) 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7fae9b7fe700 1 -- 172.21.15.45:0/3253687971 learned_addr learned my addr 172.21.15.45:0/3253687971 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7fae9affd700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430542860 0x55b43054aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.059+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43054d710 0x55b43054b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae9bfff700 1 -- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430542860 msgr2=0x55b43054aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430542860 0x55b43054aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae9bfff700 1 -- 172.21.15.45:0/3253687971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 msgr2=0x55b43054a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b43054a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae9bfff700 1 -- 172.21.15.45:0/3253687971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae90017070 con 0x55b43054d710 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b43054a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43054d710 0x55b43054b020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae94003c30 tx=0x7fae94003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17bcc79276b201d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae7affd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae7affd700 1 -- 172.21.15.45:0/3253687971 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94003e00 con 0x55b43054d710 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae7affd700 1 -- 172.21.15.45:0/3253687971 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae9400e700 con 0x55b43054d710 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b430536090 con 0x55b43054d710 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7fae7affd700 1 -- 172.21.15.45:0/3253687971 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae9400e8c0 con 0x55b43054d710 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7faeae188b80 10 client.25309 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.060+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fae940091b0 con 0x55b43054d710 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7fae7affd700 1 -- 172.21.15.45:0/3253687971 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b430536090 con 0x55b43054d710 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7fae7affd700 1 -- 172.21.15.45:0/3253687971 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fae94025020 con 0x55b43054d710 2023-09-23T09:30:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7faeae188b80 10 client.25309 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7faeae188b80 10 client.25309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7faeae188b80 20 client.25309 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7faeae188b80 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b43056e2f0 0x55b430570710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7faeae188b80 4 client.25309 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b43048edb0 con 0x55b43056e2f0 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b43056e2f0 0x55b430570710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7fae7affd700 10 client.25309 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.061+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b43056e2f0 0x55b430570710 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c5aca9c682f92 server_cookie=12f6b010b767ed05 in_seq=0 out_seq=0 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.062+0000 7fae7affd700 1 -- 172.21.15.45:0/3253687971 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b43048edb0 con 0x55b43056e2f0 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.062+0000 7fae7affd700 10 client.25309 handle_command_reply: tid=0 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.062+0000 7faeae188b80 1 client.25309 shutdown 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43054d710 msgr2=0x55b43054b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43054d710 0x55b43054b020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae94003c30 tx=0x7fae94003c60 comp rx=0 tx=0).stop 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 shutdown_connections 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b43056e2f0 0x55b430570710 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43054d710 0x55b43054b020 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430541f40 0x55b43054a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 --2- 172.21.15.45:0/3253687971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430542860 0x55b43054aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 >> 172.21.15.45:0/3253687971 conn(0x55b4304b7200 msgr2=0x55b43053dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7fae7affd700 0 client.25309 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.064+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 shutdown_connections 2023-09-23T09:30:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.065+0000 7faeae188b80 1 -- 172.21.15.45:0/3253687971 wait complete. 2023-09-23T09:30:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.065+0000 7faeae188b80 20 client.25309 trim_cache size 0 max 16384 2023-09-23T09:30:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 Processor -- start 2023-09-23T09:30:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 -- start start 2023-09-23T09:30:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430541f40 0x55b430565260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430542860 0x55b430545910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b430545e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4303168c0 con 0x55b430542860 2023-09-23T09:30:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4305657a0 con 0x55b43054d710 2023-09-23T09:30:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7faeae188b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b430546450 con 0x55b430541f40 2023-09-23T09:30:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7fae9affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430542860 0x55b430545910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7fae9affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430542860 0x55b430545910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33112/0 (socket says 172.21.15.45:33112) 2023-09-23T09:30:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7fae9affd700 1 -- 172.21.15.45:0/1245126517 learned_addr learned my addr 172.21.15.45:0/1245126517 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7fae9bfff700 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b430545e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.091+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430541f40 0x55b430565260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae9affd700 1 -- 172.21.15.45:0/1245126517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430541f40 msgr2=0x55b430565260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae9affd700 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430541f40 0x55b430565260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae9affd700 1 -- 172.21.15.45:0/1245126517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 msgr2=0x55b430545e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae9affd700 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b430545e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae9affd700 1 -- 172.21.15.45:0/1245126517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4302c0d80 con 0x55b430542860 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae9bfff700 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b430545e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae9affd700 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430542860 0x55b430545910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b2e0 tx=0x7fae8c09aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda668b859b474bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae827fc700 1 -- 172.21.15.45:0/1245126517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c09aa30 con 0x55b430542860 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae827fc700 1 -- 172.21.15.45:0/1245126517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae8c09b7d0 con 0x55b430542860 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.092+0000 7fae827fc700 1 -- 172.21.15.45:0/1245126517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c099590 con 0x55b430542860 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 -- 172.21.15.45:0/1245126517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430542860 msgr2=0x55b430545910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430542860 0x55b430545910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b2e0 tx=0x7fae8c09aea0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 -- 172.21.15.45:0/1245126517 shutdown_connections 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b43054d710 0x55b430545e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430541f40 0x55b430565260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 --2- 172.21.15.45:0/1245126517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b430542860 0x55b430545910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 -- 172.21.15.45:0/1245126517 >> 172.21.15.45:0/1245126517 conn(0x55b4304b7200 msgr2=0x55b4305699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 -- 172.21.15.45:0/1245126517 shutdown_connections 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 -- 172.21.15.45:0/1245126517 wait complete. 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 -- start start 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.093+0000 7faeae188b80 1 Processor -- start 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430541f40 0x55b430564ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 0x55b4305477a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b43054d710 0x55b430547ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b430548220 con 0x55b43054d710 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b430548390 con 0x55b430541f40 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b430548550 con 0x55b430542860 2023-09-23T09:30:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 10 client.0 _ll_register_callbacks cb 0x55b43028f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430541f40 0x55b430564ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b43054d710 0x55b430547ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b43054d710 0x55b430547ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33122/0 (socket says 172.21.15.45:33122) 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430541f40 0x55b430564ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54468/0 (socket says 172.21.15.45:54468) 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 -- 172.21.15.45:0/3211156565 learned_addr learned my addr 172.21.15.45:0/3211156565 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9affd700 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 0x55b4305477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 -- 172.21.15.45:0/3211156565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 msgr2=0x55b4305477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 0x55b4305477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 -- 172.21.15.45:0/3211156565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430541f40 msgr2=0x55b430564ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430541f40 0x55b430564ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 -- 172.21.15.45:0/3211156565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae940091b0 con 0x55b43054d710 2023-09-23T09:30:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae9bfff700 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b43054d710 0x55b430547ce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fae940060b0 tx=0x7fae9400e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b527b031afe00f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae7affd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae7affd700 1 -- 172.21.15.45:0/3211156565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94009920 con 0x55b43054d710 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7fae7affd700 1 -- 172.21.15.45:0/3211156565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae9400ebd0 con 0x55b43054d710 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.094+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae9400f040 con 0x55b43054d710 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7fae7affd700 1 -- 172.21.15.45:0/3211156565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae940174b0 con 0x55b43054d710 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 10 client.15642 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fae9c1aa540 con 0x55b43054d710 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7fae7affd700 1 -- 172.21.15.45:0/3211156565 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae9403e080 con 0x55b43054d710 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7fae7affd700 1 -- 172.21.15.45:0/3211156565 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fae940078c0 con 0x55b43054d710 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 20 client.15642 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b4305c21e0 0x55b4305c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 4 client.15642 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:30:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b43048ead0 con 0x55b4305c21e0 2023-09-23T09:30:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.095+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b4305c21e0 0x55b4305c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.096+0000 7fae7affd700 10 client.15642 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.096+0000 7fae9b7fe700 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b4305c21e0 0x55b4305c4600 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6d00e728fbdafa server_cookie=c3dbb2da0a0b1dc in_seq=0 out_seq=0 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.115+0000 7fae7affd700 1 -- 172.21.15.45:0/3211156565 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b43048ead0 con 0x55b4305c21e0 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.115+0000 7fae7affd700 10 client.15642 handle_command_reply: tid=0 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.115+0000 7faeae188b80 1 client.15642 shutdown 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b43054d710 msgr2=0x55b430547ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b43054d710 0x55b430547ce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fae940060b0 tx=0x7fae9400e930 comp rx=0 tx=0).stop 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 shutdown_connections 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b4305c21e0 0x55b4305c4600 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b43054d710 0x55b430547ce0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7fae7affd700 0 client.15642 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b430541f40 0x55b430564ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 --2- 172.21.15.45:0/3211156565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b430542860 0x55b4305477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 >> 172.21.15.45:0/3211156565 conn(0x55b4304b7200 msgr2=0x55b430545910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 shutdown_connections 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 1 -- 172.21.15.45:0/3211156565 wait complete. 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faeae188b80 20 client.15642 trim_cache size 0 max 16384 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:30:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:30:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:30:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:30:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae840a1900 msgr2=0x7fae840a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae840a1900 0x7fae840a3dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae90009d20 tx=0x7fae90009260 comp rx=0 tx=0).stop 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c105030 msgr2=0x7fae9c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c105030 0x7fae9c10d870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0098e0 tx=0x7fae8c009580 comp rx=0 tx=0).stop 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 shutdown_connections 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c10ddb0 0x7fae9c1a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae840a1900 0x7fae840a3dc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c0fff10 0x7fae9c10d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 --2- 172.21.15.45:0/3302283561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c105030 0x7fae9c10d870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.116+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 >> 172.21.15.45:0/3302283561 conn(0x7fae9c0fb750 msgr2=0x7fae9c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.117+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 shutdown_connections 2023-09-23T09:30:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:08.117+0000 7faea1f9e700 1 -- 172.21.15.45:0/3302283561 wait complete. 2023-09-23T09:30:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:09 smithi098 ceph-mon[102386]: pgmap v24: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 106 MiB/s wr, 2.82k op/s 2023-09-23T09:30:09.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:09 smithi045 ceph-mon[98627]: pgmap v24: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 106 MiB/s wr, 2.82k op/s 2023-09-23T09:30:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:09 smithi123 ceph-mon[103577]: pgmap v24: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 106 MiB/s wr, 2.82k op/s 2023-09-23T09:30:11.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:11 smithi098 ceph-mon[102386]: pgmap v25: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 52 MiB/s wr, 2.12k op/s 2023-09-23T09:30:11.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:11 smithi045 ceph-mon[98627]: pgmap v25: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 52 MiB/s wr, 2.12k op/s 2023-09-23T09:30:11.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:11 smithi123 ceph-mon[103577]: pgmap v25: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 52 MiB/s wr, 2.12k op/s 2023-09-23T09:30:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 25.0 s, 3577.0 tps, lat 27.921 ms stddev 32.934 2023-09-23T09:30:13.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:13 smithi098 ceph-mon[102386]: pgmap v26: 97 pgs: 97 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 65 MiB/s wr, 2.40k op/s 2023-09-23T09:30:13.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:13 smithi045 ceph-mon[98627]: pgmap v26: 97 pgs: 97 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 65 MiB/s wr, 2.40k op/s 2023-09-23T09:30:13.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:13 smithi123 ceph-mon[103577]: pgmap v26: 97 pgs: 97 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 65 MiB/s wr, 2.40k op/s 2023-09-23T09:30:14.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:14 smithi098 ceph-mon[102386]: scrub summary: idle 2023-09-23T09:30:14.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:14 smithi098 ceph-mon[102386]: scrub summary: idle 2023-09-23T09:30:14.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:14 smithi098 ceph-mon[102386]: scrub summary: idle 2023-09-23T09:30:14.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:14 smithi045 ceph-mon[98627]: scrub summary: idle 2023-09-23T09:30:14.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:14 smithi045 ceph-mon[98627]: scrub summary: idle 2023-09-23T09:30:14.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:14 smithi045 ceph-mon[98627]: scrub summary: idle 2023-09-23T09:30:14.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:14 smithi123 ceph-mon[103577]: scrub summary: idle 2023-09-23T09:30:14.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:14 smithi123 ceph-mon[103577]: scrub summary: idle 2023-09-23T09:30:14.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:14 smithi123 ceph-mon[103577]: scrub summary: idle 2023-09-23T09:30:15.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:15 smithi098 ceph-mon[102386]: pgmap v27: 97 pgs: 97 active+clean; 1.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 66 MiB/s wr, 2.10k op/s 2023-09-23T09:30:15.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:15 smithi045 ceph-mon[98627]: pgmap v27: 97 pgs: 97 active+clean; 1.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 66 MiB/s wr, 2.10k op/s 2023-09-23T09:30:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:15 smithi123 ceph-mon[103577]: pgmap v27: 97 pgs: 97 active+clean; 1.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 66 MiB/s wr, 2.10k op/s 2023-09-23T09:30:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 30.0 s, 3560.6 tps, lat 28.132 ms stddev 59.072 2023-09-23T09:30:17.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:17 smithi098 ceph-mon[102386]: pgmap v28: 97 pgs: 97 active+clean; 1.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 49 MiB/s wr, 1.16k op/s 2023-09-23T09:30:17.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:17 smithi045 ceph-mon[98627]: pgmap v28: 97 pgs: 97 active+clean; 1.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 49 MiB/s wr, 1.16k op/s 2023-09-23T09:30:17.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:17 smithi123 ceph-mon[103577]: pgmap v28: 97 pgs: 97 active+clean; 1.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 49 MiB/s wr, 1.16k op/s 2023-09-23T09:30:19.424 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:19 smithi098 ceph-mon[102386]: pgmap v29: 97 pgs: 97 active+clean; 1.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 80 MiB/s wr, 2.86k op/s 2023-09-23T09:30:19.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:19 smithi045 ceph-mon[98627]: pgmap v29: 97 pgs: 97 active+clean; 1.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 80 MiB/s wr, 2.86k op/s 2023-09-23T09:30:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:19 smithi123 ceph-mon[103577]: pgmap v29: 97 pgs: 97 active+clean; 1.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 80 MiB/s wr, 2.86k op/s 2023-09-23T09:30:20.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:30:20.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.378+0000 7fed42f00700 1 -- 172.21.15.45:0/1692885097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 msgr2=0x7fed3c10feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.378+0000 7fed42f00700 1 --2- 172.21.15.45:0/1692885097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 0x7fed3c10feb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b0a0 tx=0x7fed3400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.379+0000 7fed42f00700 1 -- 172.21.15.45:0/1692885097 shutdown_connections 2023-09-23T09:30:20.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.379+0000 7fed42f00700 1 --2- 172.21.15.45:0/1692885097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 0x7fed3c10feb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.379+0000 7fed42f00700 1 --2- 172.21.15.45:0/1692885097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed3c103210 0x7fed3c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.379+0000 7fed42f00700 1 --2- 172.21.15.45:0/1692885097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c103bc0 0x7fed3c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.379+0000 7fed42f00700 1 -- 172.21.15.45:0/1692885097 >> 172.21.15.45:0/1692885097 conn(0x7fed3c0fea90 msgr2=0x7fed3c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:20.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.380+0000 7fed42f00700 1 -- 172.21.15.45:0/1692885097 shutdown_connections 2023-09-23T09:30:20.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.380+0000 7fed42f00700 1 -- 172.21.15.45:0/1692885097 wait complete. 2023-09-23T09:30:20.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 Processor -- start 2023-09-23T09:30:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 -- start start 2023-09-23T09:30:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed3c103210 0x7fed3c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 0x7fed3c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c19db80 0x7fed3c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19e600 con 0x7fed3c108330 2023-09-23T09:30:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19e770 con 0x7fed3c103210 2023-09-23T09:30:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed42f00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19e930 con 0x7fed3c19db80 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 0x7fed3c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed4149d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c19db80 0x7fed3c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 0x7fed3c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33136/0 (socket says 172.21.15.45:33136) 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed4149d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c19db80 0x7fed3c19e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35240/0 (socket says 172.21.15.45:35240) 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed3bfff700 1 -- 172.21.15.45:0/3536142001 learned_addr learned my addr 172.21.15.45:0/3536142001 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.381+0000 7fed40c9c700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed3c103210 0x7fed3c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed4149d700 1 -- 172.21.15.45:0/3536142001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed3c103210 msgr2=0x7fed3c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed4149d700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed3c103210 0x7fed3c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed4149d700 1 -- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 msgr2=0x7fed3c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed4149d700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 0x7fed3c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed4149d700 1 -- 172.21.15.45:0/3536142001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009d00 con 0x7fed3c19db80 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed40c9c700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed3c103210 0x7fed3c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed4149d700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c19db80 0x7fed3c19e000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b0a0 tx=0x7fed34007970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed39ffb700 1 -- 172.21.15.45:0/3536142001 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400e040 con 0x7fed3c19db80 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed39ffb700 1 -- 172.21.15.45:0/3536142001 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed34004c50 con 0x7fed3c19db80 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed39ffb700 1 -- 172.21.15.45:0/3536142001 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3401b6b0 con 0x7fed3c19db80 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.382+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c1a5e30 con 0x7fed3c19db80 2023-09-23T09:30:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.383+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c1a6260 con 0x7fed3c19db80 2023-09-23T09:30:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.386+0000 7fed39ffb700 1 -- 172.21.15.45:0/3536142001 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7fed3401d480 con 0x7fed3c19db80 2023-09-23T09:30:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.387+0000 7fed39ffb700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fed2c0a1980 0x7fed2c0a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.387+0000 7fed39ffb700 1 -- 172.21.15.45:0/3536142001 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fed340ce970 con 0x7fed3c19db80 2023-09-23T09:30:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 Processor -- start 2023-09-23T09:30:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed40c9c700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fed2c0a1980 0x7fed2c0a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 -- start start 2023-09-23T09:30:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6121a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f612307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f61183460 con 0x561f6121ad20 2023-09-23T09:30:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f61174c20 con 0x561f6122e3e0 2023-09-23T09:30:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.388+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f61172540 con 0x561f6121a400 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f612307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed40c9c700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fed2c0a1980 0x7fed2c0a3e40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed30009cb0 tx=0x7fed30009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f612307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54490/0 (socket says 172.21.15.45:54490) 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 -- 172.21.15.45:0/2694112309 learned_addr learned my addr 172.21.15.45:0/2694112309 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed40c9c700 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6121a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed3bfff700 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 -- 172.21.15.45:0/2694112309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 msgr2=0x561f6121a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6121a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 -- 172.21.15.45:0/2694112309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 msgr2=0x561f6122dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 -- 172.21.15.45:0/2694112309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f611d08d0 con 0x561f6122e3e0 2023-09-23T09:30:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.389+0000 7fed4149d700 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f612307d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed3409b720 tx=0x7fed34009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46c243addd024d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.390+0000 7fed26ffd700 1 -- 172.21.15.45:0/2694112309 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3409b790 con 0x561f6122e3e0 2023-09-23T09:30:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.390+0000 7fed26ffd700 1 -- 172.21.15.45:0/2694112309 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed3409f8e0 con 0x561f6122e3e0 2023-09-23T09:30:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.390+0000 7fed26ffd700 1 -- 172.21.15.45:0/2694112309 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3409ac80 con 0x561f6122e3e0 2023-09-23T09:30:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.391+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2694112309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 msgr2=0x561f612307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.391+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f612307d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed3409b720 tx=0x7fed34009780 comp rx=0 tx=0).stop 2023-09-23T09:30:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.391+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2694112309 shutdown_connections 2023-09-23T09:30:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.391+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f612307d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.391+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6121a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.391+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2694112309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.391+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2694112309 >> 172.21.15.45:0/2694112309 conn(0x561f6118f260 msgr2=0x561f61216790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.392+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2694112309 shutdown_connections 2023-09-23T09:30:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.392+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2694112309 wait complete. 2023-09-23T09:30:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.393+0000 7fed4f0eab80 1 -- start start 2023-09-23T09:30:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.393+0000 7fed4f0eab80 1 Processor -- start 2023-09-23T09:30:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f6122b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f6116de30 con 0x561f6121ad20 2023-09-23T09:30:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f6110a710 con 0x561f6122e3e0 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f6110a3b0 con 0x561f6121a400 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed4f0eab80 10 client.0 _ll_register_callbacks cb 0x561f60f67800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed40c9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed40c9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35252/0 (socket says 172.21.15.45:35252) 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.394+0000 7fed40c9c700 1 -- 172.21.15.45:0/1488368113 learned_addr learned my addr 172.21.15.45:0/1488368113 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed3bfff700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed4149d700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f6122b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed40c9c700 1 -- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 msgr2=0x561f6122b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed40c9c700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f6122b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed40c9c700 1 -- 172.21.15.45:0/1488368113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 msgr2=0x561f6122b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed40c9c700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed40c9c700 1 -- 172.21.15.45:0/1488368113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c1a6260 con 0x561f6121a400 2023-09-23T09:30:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed4149d700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f6122b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:30:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed40c9c700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122de80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed300045a0 tx=0x7fed300047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf2c62d839ca6c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.395+0000 7fed1bfff700 1 -- 172.21.15.45:0/1488368113 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30012450 con 0x561f6121a400 2023-09-23T09:30:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.396+0000 7fed1bfff700 1 -- 172.21.15.45:0/1488368113 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fed30018070 con 0x561f6121a400 2023-09-23T09:30:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.396+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f6120e250 con 0x561f6121a400 2023-09-23T09:30:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.396+0000 7fed1bfff700 1 -- 172.21.15.45:0/1488368113 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed300125b0 con 0x561f6121a400 2023-09-23T09:30:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.397+0000 7fed4f0eab80 10 client.25327 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.397+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561f60f98de0 con 0x561f6121a400 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.397+0000 7fed1bfff700 1 -- 172.21.15.45:0/1488368113 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f6120e250 con 0x561f6121a400 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.397+0000 7fed1bfff700 1 -- 172.21.15.45:0/1488368113 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fed3002a820 con 0x561f6121a400 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.397+0000 7fed4f0eab80 10 client.25327 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.397+0000 7fed4f0eab80 10 client.25327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.398+0000 7fed4f0eab80 20 client.25327 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.398+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f6123ddf0 0x561f61240210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.398+0000 7fed4f0eab80 4 client.25327 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.398+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f61166e10 con 0x561f6123ddf0 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.398+0000 7fed3bfff700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f6123ddf0 0x561f61240210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.398+0000 7fed1bfff700 10 client.25327 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.398+0000 7fed3bfff700 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f6123ddf0 0x561f61240210 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820f2fe35de8169 server_cookie=b1386f249b329106 in_seq=0 out_seq=0 2023-09-23T09:30:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.399+0000 7fed1bfff700 1 -- 172.21.15.45:0/1488368113 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f61166e10 con 0x561f6123ddf0 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.399+0000 7fed1bfff700 10 client.25327 handle_command_reply: tid=0 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 client.25327 shutdown 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 msgr2=0x561f6122de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122de80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed300045a0 tx=0x7fed300047c0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 shutdown_connections 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f6123ddf0 0x561f61240210 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6122e3e0 0x561f6122b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122de80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/1488368113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121ad20 0x561f6122b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed1bfff700 0 client.25327 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.400+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 >> 172.21.15.45:0/1488368113 conn(0x561f6118f260 msgr2=0x561f61216320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.401+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 shutdown_connections 2023-09-23T09:30:20.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.401+0000 7fed4f0eab80 1 -- 172.21.15.45:0/1488368113 wait complete. 2023-09-23T09:30:20.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.401+0000 7fed4f0eab80 20 client.25327 trim_cache size 0 max 16384 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 Processor -- start 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 -- start start 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f6122de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6122e3e0 0x561f6122a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f60fee920 con 0x561f6122e3e0 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f61226f80 con 0x561f6121ad20 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f6121b360 con 0x561f6121a400 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4149d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6122e3e0 0x561f6122a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed40c9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4149d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6122e3e0 0x561f6122a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33158/0 (socket says 172.21.15.45:33158) 2023-09-23T09:30:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed40c9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35260/0 (socket says 172.21.15.45:35260) 2023-09-23T09:30:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.430+0000 7fed4149d700 1 -- 172.21.15.45:0/2913345004 learned_addr learned my addr 172.21.15.45:0/2913345004 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed4149d700 1 -- 172.21.15.45:0/2913345004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 msgr2=0x561f6122d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed4149d700 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed4149d700 1 -- 172.21.15.45:0/2913345004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 msgr2=0x561f6122de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed3bfff700 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f6122de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed4149d700 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f6122de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed4149d700 1 -- 172.21.15.45:0/2913345004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f60f81c60 con 0x561f6122e3e0 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed3bfff700 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f6122de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.431+0000 7fed4149d700 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6122e3e0 0x561f6122a9d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed34009fd0 tx=0x7fed3409b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3992328c745f0dc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.432+0000 7fed277fe700 1 -- 172.21.15.45:0/2913345004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3409b790 con 0x561f6122e3e0 2023-09-23T09:30:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.432+0000 7fed277fe700 1 -- 172.21.15.45:0/2913345004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed340a3a90 con 0x561f6122e3e0 2023-09-23T09:30:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.432+0000 7fed277fe700 1 -- 172.21.15.45:0/2913345004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3409aee0 con 0x561f6122e3e0 2023-09-23T09:30:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed40c9c700 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2913345004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6122e3e0 msgr2=0x561f6122a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6122e3e0 0x561f6122a9d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed34009fd0 tx=0x7fed3409b690 comp rx=0 tx=0).stop 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2913345004 shutdown_connections 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6122e3e0 0x561f6122a9d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6121a400 0x561f6122d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/2913345004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f6122de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2913345004 >> 172.21.15.45:0/2913345004 conn(0x561f6118f260 msgr2=0x561f61226440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2913345004 shutdown_connections 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.433+0000 7fed4f0eab80 1 -- 172.21.15.45:0/2913345004 wait complete. 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 -- start start 2023-09-23T09:30:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 Processor -- start 2023-09-23T09:30:20.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121a400 0x561f61232600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f61232b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6122e3e0 0x561f61233080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f61290d00 con 0x561f6121a400 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f61290e70 con 0x561f6121ad20 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.434+0000 7fed4f0eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f61291030 con 0x561f6122e3e0 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4f0eab80 10 client.0 _ll_register_callbacks cb 0x7fed300043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6122e3e0 0x561f61233080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6122e3e0 0x561f61233080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35268/0 (socket says 172.21.15.45:35268) 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 -- 172.21.15.45:0/265156492 learned_addr learned my addr 172.21.15.45:0/265156492 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed40c9c700 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121a400 0x561f61232600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed3bfff700 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f61232b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 -- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 msgr2=0x561f61232b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f61232b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 -- 172.21.15.45:0/265156492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121a400 msgr2=0x561f61232600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121a400 0x561f61232600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.435+0000 7fed4149d700 1 -- 172.21.15.45:0/265156492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f611d08d0 con 0x561f6122e3e0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed4149d700 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6122e3e0 0x561f61233080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed3409b490 tx=0x7fed340cbfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3496d216611af73f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed1bfff700 1 -- 172.21.15.45:0/265156492 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340af120 con 0x561f6122e3e0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed1bfff700 1 -- 172.21.15.45:0/265156492 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fed3409b790 con 0x561f6122e3e0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed3409f8e0 con 0x561f6122e3e0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed1bfff700 1 -- 172.21.15.45:0/265156492 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed34004dc0 con 0x561f6122e3e0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed4f0eab80 10 client.25330 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.436+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561f60fbf250 con 0x561f6122e3e0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed1bfff700 1 -- 172.21.15.45:0/265156492 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed3409f8e0 con 0x561f6122e3e0 2023-09-23T09:30:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed1bfff700 1 -- 172.21.15.45:0/265156492 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fed3401d850 con 0x561f6122e3e0 2023-09-23T09:30:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed4f0eab80 10 client.25330 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed4f0eab80 10 client.25330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed4f0eab80 20 client.25330 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f612922b0 0x561f612946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed4f0eab80 4 client.25330 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:30:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f61166b30 con 0x561f612922b0 2023-09-23T09:30:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.437+0000 7fed40c9c700 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f612922b0 0x561f612946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.438+0000 7fed1bfff700 10 client.25330 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.438+0000 7fed40c9c700 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f612922b0 0x561f612946d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc56f55c70b4109e server_cookie=cf6190ac5f3f7c93 in_seq=0 out_seq=0 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.438+0000 7fed1bfff700 1 -- 172.21.15.45:0/265156492 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561f61166b30 con 0x561f612922b0 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.438+0000 7fed1bfff700 10 client.25330 handle_command_reply: tid=0 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.438+0000 7fed4f0eab80 1 client.25330 shutdown 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6122e3e0 msgr2=0x561f61233080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6122e3e0 0x561f61233080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed3409b490 tx=0x7fed340cbfb0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 shutdown_connections 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f612922b0 0x561f612946d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f6122e3e0 0x561f61233080 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed1bfff700 0 client.25330 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f6121a400 0x561f61232600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 --2- 172.21.15.45:0/265156492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f6121ad20 0x561f61232b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 >> 172.21.15.45:0/265156492 conn(0x561f6118f260 msgr2=0x561f6122a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 shutdown_connections 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 1 -- 172.21.15.45:0/265156492 wait complete. 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.439+0000 7fed4f0eab80 20 client.25330 trim_cache size 0 max 16384 2023-09-23T09:30:20.444 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:30:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2023-09-23T09:30:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2023-09-23T09:30:20.445 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:30:20.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fed2c0a1980 msgr2=0x7fed2c0a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fed2c0a1980 0x7fed2c0a3e40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed30009cb0 tx=0x7fed30009260 comp rx=0 tx=0).stop 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c19db80 msgr2=0x7fed3c19e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c19db80 0x7fed3c19e000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b0a0 tx=0x7fed34007970 comp rx=0 tx=0).stop 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 shutdown_connections 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed3c19db80 0x7fed3c19e000 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed3c103210 0x7fed3c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fed2c0a1980 0x7fed2c0a3e40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 --2- 172.21.15.45:0/3536142001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed3c108330 0x7fed3c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.440+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 >> 172.21.15.45:0/3536142001 conn(0x7fed3c0fea90 msgr2=0x7fed3c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.441+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 shutdown_connections 2023-09-23T09:30:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.441+0000 7fed42f00700 1 -- 172.21.15.45:0/3536142001 wait complete. 2023-09-23T09:30:20.465 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T09:30:20.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 -- 172.21.15.45:0/1097449478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f5a0 msgr2=0x7fc6d0107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/1097449478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f5a0 0x7fc6d0107e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009990 tx=0x7fc6cc009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 -- 172.21.15.45:0/1097449478 shutdown_connections 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/1097449478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6d01089d0 0x7fc6d011c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/1097449478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f5a0 0x7fc6d0107e20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/1097449478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6d010f980 0x7fc6d0108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 -- 172.21.15.45:0/1097449478 >> 172.21.15.45:0/1097449478 conn(0x7fc6d006be60 msgr2=0x7fc6d006c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 -- 172.21.15.45:0/1097449478 shutdown_connections 2023-09-23T09:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.688+0000 7fc6d7af3700 1 -- 172.21.15.45:0/1097449478 wait complete. 2023-09-23T09:30:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 Processor -- start 2023-09-23T09:30:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 -- start start 2023-09-23T09:30:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6d01089d0 0x7fc6d01a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6d010f5a0 0x7fc6d01a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f980 0x7fc6d01a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d01a9bb0 con 0x7fc6d01089d0 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d01a9d20 con 0x7fc6d010f5a0 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d7af3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d01a9ee0 con 0x7fc6d010f980 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.689+0000 7fc6d508e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6d010f5a0 0x7fc6d01a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d508e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6d010f5a0 0x7fc6d01a6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54526/0 (socket says 172.21.15.45:54526) 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d508e700 1 -- 172.21.15.45:0/3567734225 learned_addr learned my addr 172.21.15.45:0/3567734225 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d6090700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f980 0x7fc6d01a95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d588f700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6d01089d0 0x7fc6d01a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d6090700 1 -- 172.21.15.45:0/3567734225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6d010f5a0 msgr2=0x7fc6d01a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d6090700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6d010f5a0 0x7fc6d01a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d6090700 1 -- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6d01089d0 msgr2=0x7fc6d01a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d6090700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6d01089d0 0x7fc6d01a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d6090700 1 -- 172.21.15.45:0/3567734225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc0095f0 con 0x7fc6d010f980 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6d6090700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f980 0x7fc6d01a95b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800efc0 tx=0x7fc6c800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.690+0000 7fc6c6ffd700 1 -- 172.21.15.45:0/3567734225 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8018070 con 0x7fc6d010f980 2023-09-23T09:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.691+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d01aa160 con 0x7fc6d010f980 2023-09-23T09:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.691+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d01b2b20 con 0x7fc6d010f980 2023-09-23T09:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.692+0000 7fc6c6ffd700 1 -- 172.21.15.45:0/3567734225 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6c800f040 con 0x7fc6d010f980 2023-09-23T09:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.692+0000 7fc6c6ffd700 1 -- 172.21.15.45:0/3567734225 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c800cdf0 con 0x7fc6d010f980 2023-09-23T09:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.693+0000 7fc6c6ffd700 1 -- 172.21.15.45:0/3567734225 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7fc6c8004000 con 0x7fc6d010f980 2023-09-23T09:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.693+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d0067c10 con 0x7fc6d010f980 2023-09-23T09:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.694+0000 7fc6c6ffd700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6bc0a1930 0x7fc6bc0a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.694+0000 7fc6c6ffd700 1 -- 172.21.15.45:0/3567734225 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6c80cd0f0 con 0x7fc6d010f980 2023-09-23T09:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.694+0000 7fc6d588f700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6bc0a1930 0x7fc6bc0a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.695+0000 7fc6d588f700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6bc0a1930 0x7fc6bc0a3df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006010 tx=0x7fc6cc005c00 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:20.699+0000 7fc6c6ffd700 1 -- 172.21.15.45:0/3567734225 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc6c8096270 con 0x7fc6d010f980 2023-09-23T09:30:21.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.162+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6d011fb70 con 0x7fc6d010f980 2023-09-23T09:30:21.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.163+0000 7fc6c6ffd700 1 -- 172.21.15.45:0/3567734225 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11311 (secure 0 0 0) 0x7fc6c8099890 con 0x7fc6d010f980 2023-09-23T09:30:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:30:21.165 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:29:57.203083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2272642085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2272642085},{"type":"v1","addr":"172.21.15.123:6837","nonce":2272642085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6bc0a1930 msgr2=0x7fc6bc0a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6bc0a1930 0x7fc6bc0a3df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006010 tx=0x7fc6cc005c00 comp rx=0 tx=0).stop 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f980 msgr2=0x7fc6d01a95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f980 0x7fc6d01a95b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800efc0 tx=0x7fc6c800c3a0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 shutdown_connections 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d010f980 0x7fc6d01a95b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6bc0a1930 0x7fc6bc0a3df0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6d01089d0 0x7fc6d01a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 --2- 172.21.15.45:0/3567734225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6d010f5a0 0x7fc6d01a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 >> 172.21.15.45:0/3567734225 conn(0x7fc6d006be60 msgr2=0x7fc6d010d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 shutdown_connections 2023-09-23T09:30:21.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.169+0000 7fc6d7af3700 1 -- 172.21.15.45:0/3567734225 wait complete. 2023-09-23T09:30:21.172 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-09-23T09:30:21.191 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T09:30:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.314+0000 7f55220f9700 1 -- 172.21.15.45:0/4121158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 msgr2=0x7f551c079910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.314+0000 7f55220f9700 1 --2- 172.21.15.45:0/4121158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 0x7f551c079910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f550c009990 tx=0x7f550c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.315+0000 7f55220f9700 1 -- 172.21.15.45:0/4121158767 shutdown_connections 2023-09-23T09:30:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.315+0000 7f55220f9700 1 --2- 172.21.15.45:0/4121158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 0x7f551c079910 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.315+0000 7f55220f9700 1 --2- 172.21.15.45:0/4121158767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10d880 0x7f551c107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.315+0000 7f55220f9700 1 --2- 172.21.15.45:0/4121158767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10dc60 0x7f551c108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.315+0000 7f55220f9700 1 -- 172.21.15.45:0/4121158767 >> 172.21.15.45:0/4121158767 conn(0x7f551c06cee0 msgr2=0x7f551c06d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.315+0000 7f55220f9700 1 -- 172.21.15.45:0/4121158767 shutdown_connections 2023-09-23T09:30:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.315+0000 7f55220f9700 1 -- 172.21.15.45:0/4121158767 wait complete. 2023-09-23T09:30:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.316+0000 7f55220f9700 1 Processor -- start 2023-09-23T09:30:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.316+0000 7f55220f9700 1 -- start start 2023-09-23T09:30:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f55220f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 0x7f551c083690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f55220f9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10d880 0x7f551c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f55220f9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10dc60 0x7f551c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f55220f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c076220 con 0x7f551c10d210 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f55220f9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c076390 con 0x7f551c10d880 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f55220f9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c076550 con 0x7f551c10dc60 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f551affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10d880 0x7f551c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f551affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10d880 0x7f551c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54540/0 (socket says 172.21.15.45:54540) 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f551affd700 1 -- 172.21.15.45:0/4055860034 learned_addr learned my addr 172.21.15.45:0/4055860034 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f551bfff700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10dc60 0x7f551c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.317+0000 7f551b7fe700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 0x7f551c083690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f551affd700 1 -- 172.21.15.45:0/4055860034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10dc60 msgr2=0x7f551c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f551affd700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10dc60 0x7f551c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f551affd700 1 -- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 msgr2=0x7f551c083690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f551affd700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 0x7f551c083690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f551affd700 1 -- 172.21.15.45:0/4055860034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0095f0 con 0x7f551c10d880 2023-09-23T09:30:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f551b7fe700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 0x7f551c083690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:30:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f551affd700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10d880 0x7f551c0757a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f550400b550 tx=0x7f550400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f5518ff9700 1 -- 172.21.15.45:0/4055860034 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504011830 con 0x7f551c10d880 2023-09-23T09:30:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f5518ff9700 1 -- 172.21.15.45:0/4055860034 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5504011e50 con 0x7f551c10d880 2023-09-23T09:30:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f5518ff9700 1 -- 172.21.15.45:0/4055860034 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400f2e0 con 0x7f551c10d880 2023-09-23T09:30:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c1bb000 con 0x7f551c10d880 2023-09-23T09:30:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.318+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c1bb510 con 0x7f551c10d880 2023-09-23T09:30:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.324+0000 7f5518ff9700 1 -- 172.21.15.45:0/4055860034 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7f5504011990 con 0x7f551c10d880 2023-09-23T09:30:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.325+0000 7f5518ff9700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55080a1980 0x7f55080a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.325+0000 7f5518ff9700 1 -- 172.21.15.45:0/4055860034 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f55040cce10 con 0x7f551c10d880 2023-09-23T09:30:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.325+0000 7f552e2e3b80 1 Processor -- start 2023-09-23T09:30:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.325+0000 7f552e2e3b80 1 -- start start 2023-09-23T09:30:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde2f40 0x56173bde3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bdee710 0x56173bdf0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bd4c400 con 0x56173bde3860 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bd3dbc0 con 0x56173bde2f40 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bd3b4e0 con 0x56173bdee710 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f551bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bdee710 0x56173bdf0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f551bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bdee710 0x56173bdf0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35302/0 (socket says 172.21.15.45:35302) 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.326+0000 7f551bfff700 1 -- 172.21.15.45:0/784179828 learned_addr learned my addr 172.21.15.45:0/784179828 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551affd700 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551b7fe700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55080a1980 0x7f55080a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551bfff700 1 -- 172.21.15.45:0/784179828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde2f40 msgr2=0x56173bde3320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551bfff700 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde2f40 0x56173bde3320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551bfff700 1 -- 172.21.15.45:0/784179828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 msgr2=0x56173bdee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551bfff700 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551bfff700 1 -- 172.21.15.45:0/784179828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173bd99870 con 0x56173bdee710 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f551bfff700 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bdee710 0x56173bdf0b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f550c005c90 tx=0x7f550c005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5fb01ea8cc4bfd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.327+0000 7f5511ffb700 1 -- 172.21.15.45:0/784179828 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c01d030 con 0x56173bdee710 2023-09-23T09:30:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.328+0000 7f5511ffb700 1 -- 172.21.15.45:0/784179828 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f550c01e040 con 0x56173bdee710 2023-09-23T09:30:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.328+0000 7f5511ffb700 1 -- 172.21.15.45:0/784179828 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c005440 con 0x56173bdee710 2023-09-23T09:30:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 -- 172.21.15.45:0/784179828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bdee710 msgr2=0x56173bdf0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f551b7fe700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55080a1980 0x7f55080a3e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5514009cc0 tx=0x7f5514009040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bdee710 0x56173bdf0b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f550c005c90 tx=0x7f550c005da0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 -- 172.21.15.45:0/784179828 shutdown_connections 2023-09-23T09:30:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bdee710 0x56173bdf0b00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde2f40 0x56173bde3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/784179828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 -- 172.21.15.45:0/784179828 >> 172.21.15.45:0/784179828 conn(0x56173bd58200 msgr2=0x56173bddf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.329+0000 7f552e2e3b80 1 -- 172.21.15.45:0/784179828 shutdown_connections 2023-09-23T09:30:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.330+0000 7f552e2e3b80 1 -- 172.21.15.45:0/784179828 wait complete. 2023-09-23T09:30:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.331+0000 7f552e2e3b80 1 -- start start 2023-09-23T09:30:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.331+0000 7f552e2e3b80 1 Processor -- start 2023-09-23T09:30:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 0x56173bdee150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde3860 0x56173bdeb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bdee710 0x56173bdebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bd36dd0 con 0x56173bdee710 2023-09-23T09:30:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bcd36b0 con 0x56173bde3860 2023-09-23T09:30:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bcd3350 con 0x56173bde2f40 2023-09-23T09:30:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f552e2e3b80 10 client.0 _ll_register_callbacks cb 0x56173bb307a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f551affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde3860 0x56173bdeb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f551affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde3860 0x56173bdeb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54556/0 (socket says 172.21.15.45:54556) 2023-09-23T09:30:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f551affd700 1 -- 172.21.15.45:0/3901460521 learned_addr learned my addr 172.21.15.45:0/3901460521 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.332+0000 7f551b7fe700 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 0x56173bdee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f551bfff700 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bdee710 0x56173bdebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f551affd700 1 -- 172.21.15.45:0/3901460521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 msgr2=0x56173bdee150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f551affd700 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 0x56173bdee150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f551affd700 1 -- 172.21.15.45:0/3901460521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bdee710 msgr2=0x56173bdebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f551affd700 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bdee710 0x56173bdebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f551affd700 1 -- 172.21.15.45:0/3901460521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c0773a0 con 0x56173bde3860 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f551affd700 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde3860 0x56173bdeb3d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f550400bc60 tx=0x7f550409a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33542073e2ed757b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f54faffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f54faffd700 1 -- 172.21.15.45:0/3901460521 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550409dd80 con 0x56173bde3860 2023-09-23T09:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f54faffd700 1 -- 172.21.15.45:0/3901460521 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f55040a1fc0 con 0x56173bde3860 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.333+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56173bdd7090 con 0x56173bde3860 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.334+0000 7f54faffd700 1 -- 172.21.15.45:0/3901460521 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5504099e90 con 0x56173bde3860 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.334+0000 7f552e2e3b80 10 client.25328 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.334+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56173bb61d80 con 0x56173bde3860 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f54faffd700 1 -- 172.21.15.45:0/3901460521 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55040a9080 con 0x56173bde3860 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f54faffd700 1 -- 172.21.15.45:0/3901460521 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f55040a1b30 con 0x56173bde3860 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f552e2e3b80 10 client.25328 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f552e2e3b80 10 client.25328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f552e2e3b80 20 client.25328 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173bdfe120 0x56173be00540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f552e2e3b80 4 client.25328 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56173bd2fdb0 con 0x56173bdfe120 2023-09-23T09:30:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.335+0000 7f551b7fe700 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173bdfe120 0x56173be00540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.336+0000 7f551b7fe700 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173bdfe120 0x56173be00540 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6395fdd00a3e7c85 server_cookie=5ecd9ec71fb16403 in_seq=0 out_seq=0 2023-09-23T09:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.336+0000 7f54faffd700 10 client.25328 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.337+0000 7f54faffd700 1 -- 172.21.15.45:0/3901460521 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56173bd2fdb0 con 0x56173bdfe120 2023-09-23T09:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.337+0000 7f54faffd700 10 client.25328 handle_command_reply: tid=0 2023-09-23T09:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.337+0000 7f552e2e3b80 1 client.25328 shutdown 2023-09-23T09:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde3860 msgr2=0x56173bdeb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde3860 0x56173bdeb3d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f550400bc60 tx=0x7f550409a330 comp rx=0 tx=0).stop 2023-09-23T09:30:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 shutdown_connections 2023-09-23T09:30:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173bdfe120 0x56173be00540 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bdee710 0x56173bdebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 0x56173bdee150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/3901460521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bde3860 0x56173bdeb3d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 >> 172.21.15.45:0/3901460521 conn(0x56173bd58200 msgr2=0x56173bddef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.338+0000 7f54faffd700 0 client.25328 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.340+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 shutdown_connections 2023-09-23T09:30:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.340+0000 7f552e2e3b80 1 -- 172.21.15.45:0/3901460521 wait complete. 2023-09-23T09:30:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.341+0000 7f552e2e3b80 20 client.25328 trim_cache size 0 max 16384 2023-09-23T09:30:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 Processor -- start 2023-09-23T09:30:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 -- start start 2023-09-23T09:30:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 0x56173bdf51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdeab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bdfa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bbb78c0 con 0x56173bde3860 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bde0100 con 0x56173bdee710 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.370+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bdf5730 con 0x56173bde2f40 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bdfa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdeab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdeab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33210/0 (socket says 172.21.15.45:33210) 2023-09-23T09:30:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551affd700 1 -- 172.21.15.45:0/2102356233 learned_addr learned my addr 172.21.15.45:0/2102356233 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551bfff700 1 -- 172.21.15.45:0/2102356233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 msgr2=0x56173bdf51f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:30:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551bfff700 1 --2- 172.21.15.45:0/2102356233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 0x56173bdf51f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551bfff700 1 -- 172.21.15.45:0/2102356233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 msgr2=0x56173bdeab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551bfff700 1 --2- 172.21.15.45:0/2102356233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdeab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551bfff700 1 -- 172.21.15.45:0/2102356233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173bb4ac00 con 0x56173bdee710 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.371+0000 7f551bfff700 1 --2- 172.21.15.45:0/2102356233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bdfa3f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f550c0098e0 tx=0x7f550c00b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d6a5ed24f6f7e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.372+0000 7f55127fc700 1 -- 172.21.15.45:0/2102356233 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c01d040 con 0x56173bdee710 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.372+0000 7f55127fc700 1 -- 172.21.15.45:0/2102356233 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f550c003d60 con 0x56173bdee710 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.372+0000 7f55127fc700 1 -- 172.21.15.45:0/2102356233 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c005610 con 0x56173bdee710 2023-09-23T09:30:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.373+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2102356233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 msgr2=0x56173bdfa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.373+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2102356233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bdfa3f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f550c0098e0 tx=0x7f550c00b6b0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.373+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2102356233 shutdown_connections 2023-09-23T09:30:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.373+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2102356233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bdfa3f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.373+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2102356233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde2f40 0x56173bdf51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.373+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2102356233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde3860 0x56173bdeab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.373+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2102356233 >> 172.21.15.45:0/2102356233 conn(0x56173bd58200 msgr2=0x56173bdddaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.375+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2102356233 shutdown_connections 2023-09-23T09:30:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.375+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2102356233 wait complete. 2023-09-23T09:30:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.375+0000 7f552e2e3b80 1 -- start start 2023-09-23T09:30:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 1 Processor -- start 2023-09-23T09:30:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde2f40 0x56173bde7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde3860 0x56173bde7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bde8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bde8840 con 0x56173bde2f40 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bde89b0 con 0x56173bdee710 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173bde8b70 con 0x56173bde3860 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f552e2e3b80 10 client.0 _ll_register_callbacks cb 0x7f55040995b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f551b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde2f40 0x56173bde7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f551b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde2f40 0x56173bde7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33214/0 (socket says 172.21.15.45:33214) 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f551b7fe700 1 -- 172.21.15.45:0/2676044404 learned_addr learned my addr 172.21.15.45:0/2676044404 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f551bfff700 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bde8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.376+0000 7f551affd700 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde3860 0x56173bde7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f551bfff700 1 -- 172.21.15.45:0/2676044404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde3860 msgr2=0x56173bde7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f551bfff700 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde3860 0x56173bde7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f551bfff700 1 -- 172.21.15.45:0/2676044404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde2f40 msgr2=0x56173bde7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f551bfff700 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde2f40 0x56173bde7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f551bfff700 1 -- 172.21.15.45:0/2676044404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56173bd99870 con 0x56173bdee710 2023-09-23T09:30:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f551bfff700 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bde8130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f550c003760 tx=0x7f550c00bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559edd378b20b6f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f54faffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:30:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.377+0000 7f54faffd700 1 -- 172.21.15.45:0/2676044404 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c01d040 con 0x56173bdee710 2023-09-23T09:30:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f54faffd700 1 -- 172.21.15.45:0/2676044404 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f550c00b720 con 0x56173bdee710 2023-09-23T09:30:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f550c01e040 con 0x56173bdee710 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f54faffd700 1 -- 172.21.15.45:0/2676044404 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f550c00b920 con 0x56173bdee710 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f552e2e3b80 10 client.25334 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56173bb881f0 con 0x56173bdee710 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f54faffd700 1 -- 172.21.15.45:0/2676044404 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f550c01e040 con 0x56173bdee710 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f54faffd700 1 -- 172.21.15.45:0/2676044404 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f550c040070 con 0x56173bdee710 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f552e2e3b80 10 client.25334 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f552e2e3b80 10 client.25334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f552e2e3b80 20 client.25334 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.378+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173be526d0 0x56173be54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.379+0000 7f552e2e3b80 4 client.25334 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.379+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56173bd2fad0 con 0x56173be526d0 2023-09-23T09:30:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.379+0000 7f551b7fe700 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173be526d0 0x56173be54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.380+0000 7f54faffd700 10 client.25334 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.380+0000 7f551b7fe700 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173be526d0 0x56173be54ac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca5b5fe140a434b server_cookie=3e003bcd8aac7672 in_seq=0 out_seq=0 2023-09-23T09:30:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.405+0000 7f54faffd700 1 -- 172.21.15.45:0/2676044404 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56173bd2fad0 con 0x56173be526d0 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.405+0000 7f54faffd700 10 client.25334 handle_command_reply: tid=0 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 client.25334 shutdown 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 msgr2=0x56173bde8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bde8130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f550c003760 tx=0x7f550c00bf40 comp rx=0 tx=0).stop 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 shutdown_connections 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56173be526d0 0x56173be54ac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56173bdee710 0x56173bde8130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173bde2f40 0x56173bde7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f54faffd700 0 client.25334 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 --2- 172.21.15.45:0/2676044404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56173bde3860 0x56173bde7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 >> 172.21.15.45:0/2676044404 conn(0x56173bd58200 msgr2=0x56173bdfa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 shutdown_connections 2023-09-23T09:30:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 1 -- 172.21.15.45:0/2676044404 wait complete. 2023-09-23T09:30:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.406+0000 7f552e2e3b80 20 client.25334 trim_cache size 0 max 16384 2023-09-23T09:30:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:30:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:30:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:30:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55080a1980 msgr2=0x7f55080a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55080a1980 0x7f55080a3e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5514009cc0 tx=0x7f5514009040 comp rx=0 tx=0).stop 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10d880 msgr2=0x7f551c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10d880 0x7f551c0757a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f550400b550 tx=0x7f550400b860 comp rx=0 tx=0).stop 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 shutdown_connections 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10dc60 0x7f551c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55080a1980 0x7f55080a3e40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f551c10d210 0x7f551c083690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 --2- 172.21.15.45:0/4055860034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f551c10d880 0x7f551c0757a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 >> 172.21.15.45:0/4055860034 conn(0x7f551c06cee0 msgr2=0x7f551c0717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 shutdown_connections 2023-09-23T09:30:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:21.412+0000 7f55220f9700 1 -- 172.21.15.45:0/4055860034 wait complete. 2023-09-23T09:30:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.467+0000 7f70ed528700 1 -- 172.21.15.45:0/4051000119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e8107e20 msgr2=0x7f70e8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.467+0000 7f70ed528700 1 --2- 172.21.15.45:0/4051000119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e8107e20 0x7f70e8108200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009990 tx=0x7f70d8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.469+0000 7f70ed528700 1 -- 172.21.15.45:0/4051000119 shutdown_connections 2023-09-23T09:30:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.469+0000 7f70ed528700 1 --2- 172.21.15.45:0/4051000119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8113f70 0x7f70e8116360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.469+0000 7f70ed528700 1 --2- 172.21.15.45:0/4051000119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e8107e20 0x7f70e8108200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.469+0000 7f70ed528700 1 --2- 172.21.15.45:0/4051000119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8108740 0x7f70e8113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.469+0000 7f70ed528700 1 -- 172.21.15.45:0/4051000119 >> 172.21.15.45:0/4051000119 conn(0x7f70e806d0e0 msgr2=0x7f70e806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.469+0000 7f70ed528700 1 -- 172.21.15.45:0/4051000119 shutdown_connections 2023-09-23T09:30:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.469+0000 7f70ed528700 1 -- 172.21.15.45:0/4051000119 wait complete. 2023-09-23T09:30:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.470+0000 7f70ed528700 1 Processor -- start 2023-09-23T09:30:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.470+0000 7f70ed528700 1 -- start start 2023-09-23T09:30:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70ed528700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8107e20 0x7f70e8109890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70ed528700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8108740 0x7f70e8109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70ed528700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e8113f70 0x7f70e810a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70ed528700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e81b2130 con 0x7f70e8107e20 2023-09-23T09:30:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70ed528700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e81b2270 con 0x7f70e8108740 2023-09-23T09:30:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70ed528700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e81b23b0 con 0x7f70e8113f70 2023-09-23T09:30:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8108740 0x7f70e8109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8108740 0x7f70e8109dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54578/0 (socket says 172.21.15.45:54578) 2023-09-23T09:30:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 -- 172.21.15.45:0/1707320616 learned_addr learned my addr 172.21.15.45:0/1707320616 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8107e20 0x7f70e8109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 -- 172.21.15.45:0/1707320616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e8113f70 msgr2=0x7f70e810a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e8113f70 0x7f70e810a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 -- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8107e20 msgr2=0x7f70e8109890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8107e20 0x7f70e8109890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 -- 172.21.15.45:0/1707320616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80095f0 con 0x7f70e8108740 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.471+0000 7f70e67fc700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8108740 0x7f70e8109dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00b550 tx=0x7f70dc00b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.472+0000 7f70cffff700 1 -- 172.21.15.45:0/1707320616 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc011830 con 0x7f70e8108740 2023-09-23T09:30:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.472+0000 7f70cffff700 1 -- 172.21.15.45:0/1707320616 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70dc011e50 con 0x7f70e8108740 2023-09-23T09:30:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.472+0000 7f70cffff700 1 -- 172.21.15.45:0/1707320616 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc010e70 con 0x7f70e8108740 2023-09-23T09:30:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.472+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e81b2690 con 0x7f70e8108740 2023-09-23T09:30:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.472+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e81b2b50 con 0x7f70e8108740 2023-09-23T09:30:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.475+0000 7f70cffff700 1 -- 172.21.15.45:0/1707320616 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7f70dc011990 con 0x7f70e8108740 2023-09-23T09:30:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.476+0000 7f70cffff700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f70d00a1980 0x7f70d00a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.476+0000 7f70cffff700 1 -- 172.21.15.45:0/1707320616 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f70dc0ccd60 con 0x7f70e8108740 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.477+0000 7f70f9712b80 1 Processor -- start 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.477+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f70d00a1980 0x7f70d00a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.477+0000 7f70f9712b80 1 -- start start 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 0x5611d09ab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70f9712b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d0909460 con 0x5611d09a0400 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70f9712b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d08fac20 con 0x5611d09abbd0 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70f9712b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d08f8540 con 0x5611d09a0d20 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54584/0 (socket says 172.21.15.45:54584) 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 -- 172.21.15.45:0/2831968528 learned_addr learned my addr 172.21.15.45:0/2831968528 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e67fc700 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 0x5611d09ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 -- 172.21.15.45:0/2831968528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 msgr2=0x5611d09ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 0x5611d09ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 -- 172.21.15.45:0/2831968528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 msgr2=0x5611d09a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e77fe700 1 -- 172.21.15.45:0/2831968528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611d09568d0 con 0x5611d09abbd0 2023-09-23T09:30:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.478+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f70d00a1980 0x7f70d00a3e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b610 tx=0x7f70d80058e0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.479+0000 7f70e77fe700 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09adfc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f70e000e490 tx=0x7f70e000e7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c025a546ce782f76 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.479+0000 7f70cd7fa700 1 -- 172.21.15.45:0/2831968528 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0018070 con 0x5611d09abbd0 2023-09-23T09:30:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.479+0000 7f70cd7fa700 1 -- 172.21.15.45:0/2831968528 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e000f040 con 0x5611d09abbd0 2023-09-23T09:30:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.479+0000 7f70cd7fa700 1 -- 172.21.15.45:0/2831968528 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0004280 con 0x5611d09abbd0 2023-09-23T09:30:21.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.480+0000 7f70f9712b80 1 -- 172.21.15.45:0/2831968528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 msgr2=0x5611d09adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.480+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09adfc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f70e000e490 tx=0x7f70e000e7a0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.480+0000 7f70f9712b80 1 -- 172.21.15.45:0/2831968528 shutdown_connections 2023-09-23T09:30:21.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.480+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09adfc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.480+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.480+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2831968528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 0x5611d09ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.480+0000 7f70f9712b80 1 -- 172.21.15.45:0/2831968528 >> 172.21.15.45:0/2831968528 conn(0x5611d0915260 msgr2=0x5611d099c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.481+0000 7f70f9712b80 1 -- 172.21.15.45:0/2831968528 shutdown_connections 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.481+0000 7f70f9712b80 1 -- 172.21.15.45:0/2831968528 wait complete. 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 -- start start 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 Processor -- start 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0400 0x5611d09ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0d20 0x5611d09a8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d08f3e30 con 0x5611d09a0d20 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d0890710 con 0x5611d09a0400 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d08903b0 con 0x5611d09abbd0 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70f9712b80 10 client.0 _ll_register_callbacks cb 0x5611d06ed800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70e67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0d20 0x5611d09a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70e77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70e6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0400 0x5611d09ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70e67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0d20 0x5611d09a8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33244/0 (socket says 172.21.15.45:33244) 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70e6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0400 0x5611d09ab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54590/0 (socket says 172.21.15.45:54590) 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70e77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09a8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35348/0 (socket says 172.21.15.45:35348) 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.482+0000 7f70e67fc700 1 -- 172.21.15.45:0/29576073 learned_addr learned my addr 172.21.15.45:0/29576073 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.484+0000 7f70e77fe700 1 -- 172.21.15.45:0/29576073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0400 msgr2=0x5611d09ab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.484+0000 7f70e77fe700 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0400 0x5611d09ab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.484+0000 7f70e77fe700 1 -- 172.21.15.45:0/29576073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0d20 msgr2=0x5611d09a8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.484+0000 7f70e77fe700 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0d20 0x5611d09a8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.484+0000 7f70e77fe700 1 -- 172.21.15.45:0/29576073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e000e140 con 0x5611d09abbd0 2023-09-23T09:30:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.487+0000 7f70e77fe700 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09a8e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009960 tx=0x7f70e000efc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7802f45133e21768 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.487+0000 7f70caff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:21.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.491+0000 7f70caff5700 1 -- 172.21.15.45:0/29576073 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0022070 con 0x5611d09abbd0 2023-09-23T09:30:21.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.491+0000 7f70caff5700 1 -- 172.21.15.45:0/29576073 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f70e0003e00 con 0x5611d09abbd0 2023-09-23T09:30:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.498+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611d0994250 con 0x5611d09abbd0 2023-09-23T09:30:21.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.502+0000 7f70caff5700 1 -- 172.21.15.45:0/29576073 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70e0018070 con 0x5611d09abbd0 2023-09-23T09:30:21.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.504+0000 7f70f9712b80 10 client.25357 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:21.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.504+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5611d071ede0 con 0x5611d09abbd0 2023-09-23T09:30:21.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70caff5700 1 -- 172.21.15.45:0/29576073 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611d0994250 con 0x5611d09abbd0 2023-09-23T09:30:21.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70caff5700 1 -- 172.21.15.45:0/29576073 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f70e0025020 con 0x5611d09abbd0 2023-09-23T09:30:21.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70f9712b80 10 client.25357 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:21.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70f9712b80 10 client.25357 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-23T09:30:21.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70f9712b80 20 client.25357 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:21.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70f9712b80 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d09dd550 0x5611d09df970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70f9712b80 4 client.25357 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:21.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611d08ece10 con 0x5611d09dd550 2023-09-23T09:30:21.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.505+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d09dd550 0x5611d09df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.506+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d09dd550 0x5611d09df970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91c113a9eee20e31 server_cookie=d8b97a808f509970 in_seq=0 out_seq=0 2023-09-23T09:30:21.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.506+0000 7f70caff5700 10 client.25357 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.506+0000 7f70caff5700 1 -- 172.21.15.45:0/29576073 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611d08ece10 con 0x5611d09dd550 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.506+0000 7f70caff5700 10 client.25357 handle_command_reply: tid=0 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.506+0000 7f70f9712b80 1 client.25357 shutdown 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.506+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 msgr2=0x5611d09a8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.506+0000 7f70f9712b80 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09a8e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009960 tx=0x7f70e000efc0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 shutdown_connections 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d09dd550 0x5611d09df970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09a8e30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0400 0x5611d09ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 --2- 172.21.15.45:0/29576073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0d20 0x5611d09a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 >> 172.21.15.45:0/29576073 conn(0x5611d0915260 msgr2=0x5611d099c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70caff5700 0 client.25357 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:30:21.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 shutdown_connections 2023-09-23T09:30:21.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 1 -- 172.21.15.45:0/29576073 wait complete. 2023-09-23T09:30:21.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.507+0000 7f70f9712b80 20 client.25357 trim_cache size 0 max 16384 2023-09-23T09:30:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 Processor -- start 2023-09-23T09:30:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 -- start start 2023-09-23T09:30:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0d20 0x5611d09a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d0774920 con 0x5611d09a0400 2023-09-23T09:30:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d0993d90 con 0x5611d09a0d20 2023-09-23T09:30:21.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.531+0000 7f70f9712b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d09b2c50 con 0x5611d09abbd0 2023-09-23T09:30:21.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33250/0 (socket says 172.21.15.45:33250) 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e6ffd700 1 -- 172.21.15.45:0/2937716664 learned_addr learned my addr 172.21.15.45:0/2937716664 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e77fe700 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e6ffd700 1 -- 172.21.15.45:0/2937716664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 msgr2=0x5611d09ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e6ffd700 1 -- 172.21.15.45:0/2937716664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0d20 msgr2=0x5611d09a7fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.532+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0d20 0x5611d09a7fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.533+0000 7f70e6ffd700 1 -- 172.21.15.45:0/2937716664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611d0707c60 con 0x5611d09a0400 2023-09-23T09:30:21.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.533+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b2710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f70d80098e0 tx=0x7f70d8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9942900b5d2ccf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.534+0000 7f70cdffb700 1 -- 172.21.15.45:0/2937716664 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8037070 con 0x5611d09a0400 2023-09-23T09:30:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.534+0000 7f70cdffb700 1 -- 172.21.15.45:0/2937716664 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d801e040 con 0x5611d09a0400 2023-09-23T09:30:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.534+0000 7f70cdffb700 1 -- 172.21.15.45:0/2937716664 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8004500 con 0x5611d09a0400 2023-09-23T09:30:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 -- 172.21.15.45:0/2937716664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 msgr2=0x5611d09b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b2710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f70d80098e0 tx=0x7f70d8020000 comp rx=0 tx=0).stop 2023-09-23T09:30:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 -- 172.21.15.45:0/2937716664 shutdown_connections 2023-09-23T09:30:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09abbd0 0x5611d09ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b2710 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 --2- 172.21.15.45:0/2937716664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09a0d20 0x5611d09a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 -- 172.21.15.45:0/2937716664 >> 172.21.15.45:0/2937716664 conn(0x5611d0915260 msgr2=0x5611d09b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 -- 172.21.15.45:0/2937716664 shutdown_connections 2023-09-23T09:30:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.535+0000 7f70f9712b80 1 -- 172.21.15.45:0/2937716664 wait complete. 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 -- start start 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 Processor -- start 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 0x5611d09ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d09b2b00 con 0x5611d09a0400 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d09af170 con 0x5611d09abbd0 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d09af330 con 0x5611d09a0d20 2023-09-23T09:30:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.536+0000 7f70f9712b80 10 client.0 _ll_register_callbacks cb 0x7f70e000e360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54608/0 (socket says 172.21.15.45:54608) 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 -- 172.21.15.45:0/4097887420 learned_addr learned my addr 172.21.15.45:0/4097887420 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 -- 172.21.15.45:0/4097887420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 msgr2=0x5611d09ae6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 0x5611d09ae6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 -- 172.21.15.45:0/4097887420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 msgr2=0x5611d09b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 -- 172.21.15.45:0/4097887420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e8116ce0 con 0x5611d09abbd0 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.537+0000 7f70e77fe700 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09aec30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009250 tx=0x7f70e0007b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6221a22393e4f8d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.538+0000 7f70caff5700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.538+0000 7f70caff5700 1 -- 172.21.15.45:0/4097887420 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0022070 con 0x5611d09abbd0 2023-09-23T09:30:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.538+0000 7f70caff5700 1 -- 172.21.15.45:0/4097887420 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f70e0007c00 con 0x5611d09abbd0 2023-09-23T09:30:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.538+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70e000f040 con 0x5611d09abbd0 2023-09-23T09:30:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.538+0000 7f70caff5700 1 -- 172.21.15.45:0/4097887420 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70e0018040 con 0x5611d09abbd0 2023-09-23T09:30:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.538+0000 7f70f9712b80 10 client.25349 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.538+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5611d0745250 con 0x5611d09abbd0 2023-09-23T09:30:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.539+0000 7f70caff5700 1 -- 172.21.15.45:0/4097887420 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e003d020 con 0x5611d09abbd0 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.539+0000 7f70caff5700 1 -- 172.21.15.45:0/4097887420 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f70e0025050 con 0x5611d09abbd0 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.539+0000 7f70f9712b80 10 client.25349 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.539+0000 7f70f9712b80 10 client.25349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.539+0000 7f70f9712b80 20 client.25349 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.539+0000 7f70f9712b80 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d0a32500 0x5611d0a34920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.540+0000 7f70f9712b80 4 client.25349 mds_command: new command op to 24496 tid=0[{"prefix": "damage ls"}] 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.540+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 --> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5611d08ecb30 con 0x5611d0a32500 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.540+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d0a32500 0x5611d0a34920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70caff5700 10 client.25349 ms_handle_connect on v2:172.21.15.123:6838/4054887711 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70e6ffd700 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d0a32500 0x5611d0a34920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1d824770327399b server_cookie=dcbb26dd93004914 in_seq=0 out_seq=0 2023-09-23T09:30:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70caff5700 1 -- 172.21.15.45:0/4097887420 <== mds.2 v2:172.21.15.123:6838/4054887711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5611d08ecb30 con 0x5611d0a32500 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70caff5700 10 client.25349 handle_command_reply: tid=0 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 client.25349 shutdown 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 msgr2=0x5611d09aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09aec30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009250 tx=0x7f70e0007b30 comp rx=0 tx=0).stop 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 shutdown_connections 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d09abbd0 0x5611d09aec30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d09a0400 0x5611d09b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.123:6838/4054887711,v1:172.21.15.123:6839/4054887711] conn(0x5611d0a32500 0x5611d0a34920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 --2- 172.21.15.45:0/4097887420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d09a0d20 0x5611d09ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70caff5700 0 client.25349 ms_handle_reset on v2:172.21.15.123:6838/4054887711 2023-09-23T09:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 >> 172.21.15.45:0/4097887420 conn(0x5611d0915260 msgr2=0x5611d09a3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 shutdown_connections 2023-09-23T09:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.541+0000 7f70f9712b80 1 -- 172.21.15.45:0/4097887420 wait complete. 2023-09-23T09:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.542+0000 7f70f9712b80 20 client.25349 trim_cache size 0 max 16384 2023-09-23T09:30:21.546 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f70d00a1980 msgr2=0x7f70d00a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f70d00a1980 0x7f70d00a3e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b610 tx=0x7f70d80058e0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8108740 msgr2=0x7f70e8109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8108740 0x7f70e8109dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00b550 tx=0x7f70dc00b860 comp rx=0 tx=0).stop 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 shutdown_connections 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e8113f70 0x7f70e810a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f70d00a1980 0x7f70d00a3e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8107e20 0x7f70e8109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 --2- 172.21.15.45:0/1707320616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8108740 0x7f70e8109dd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.543+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 >> 172.21.15.45:0/1707320616 conn(0x7f70e806d0e0 msgr2=0x7f70e806f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.544+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 shutdown_connections 2023-09-23T09:30:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.544+0000 7f70ed528700 1 -- 172.21.15.45:0/1707320616 wait complete. 2023-09-23T09:30:21.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:21 smithi045 ceph-mon[98627]: pgmap v30: 97 pgs: 97 active+clean; 1.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 60 MiB/s wr, 2.30k op/s 2023-09-23T09:30:21.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:21 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3567734225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:30:21.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T09:30:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.780+0000 7fa13e3e4700 1 -- 172.21.15.45:0/1145116980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f940 msgr2=0x7fa13810d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.780+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/1145116980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f940 0x7fa13810d5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa128009990 tx=0x7fa128009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.780+0000 7fa13e3e4700 1 -- 172.21.15.45:0/1145116980 shutdown_connections 2023-09-23T09:30:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.780+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/1145116980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13811a600 0x7fa13811c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.780+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/1145116980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f4d0 0x7fa13810d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.780+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/1145116980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f940 0x7fa13810d5a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.780+0000 7fa13e3e4700 1 -- 172.21.15.45:0/1145116980 >> 172.21.15.45:0/1145116980 conn(0x7fa13806d0e0 msgr2=0x7fa13806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.781+0000 7fa13e3e4700 1 -- 172.21.15.45:0/1145116980 shutdown_connections 2023-09-23T09:30:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.782+0000 7fa13e3e4700 1 -- 172.21.15.45:0/1145116980 wait complete. 2023-09-23T09:30:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.782+0000 7fa13e3e4700 1 Processor -- start 2023-09-23T09:30:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.783+0000 7fa13e3e4700 1 -- start start 2023-09-23T09:30:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.783+0000 7fa13e3e4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f4d0 0x7fa1381165a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.783+0000 7fa13e3e4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f940 0x7fa138116ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.783+0000 7fa13e3e4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13811a600 0x7fa138111790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.783+0000 7fa13e3e4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa138111d60 con 0x7fa13811a600 2023-09-23T09:30:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.783+0000 7fa13e3e4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa138111ed0 con 0x7fa13810f940 2023-09-23T09:30:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.783+0000 7fa13e3e4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa138112090 con 0x7fa13810f4d0 2023-09-23T09:30:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f940 0x7fa138116ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f940 0x7fa138116ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54616/0 (socket says 172.21.15.45:54616) 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 -- 172.21.15.45:0/2477038844 learned_addr learned my addr 172.21.15.45:0/2477038844 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa137fff700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f4d0 0x7fa1381165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 -- 172.21.15.45:0/2477038844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f4d0 msgr2=0x7fa1381165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f4d0 0x7fa1381165a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 -- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13811a600 msgr2=0x7fa138111790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13811a600 0x7fa138111790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.784+0000 7fa1377fe700 1 -- 172.21.15.45:0/2477038844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280095f0 con 0x7fa13810f940 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.785+0000 7fa1377fe700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f940 0x7fa138116ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa12800b2e0 tx=0x7fa128005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.785+0000 7fa1357fa700 1 -- 172.21.15.45:0/2477038844 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12801d070 con 0x7fa13810f940 2023-09-23T09:30:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.785+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa138112310 con 0x7fa13810f940 2023-09-23T09:30:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.785+0000 7fa1357fa700 1 -- 172.21.15.45:0/2477038844 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa128021410 con 0x7fa13810f940 2023-09-23T09:30:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.785+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1381154e0 con 0x7fa13810f940 2023-09-23T09:30:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.785+0000 7fa1357fa700 1 -- 172.21.15.45:0/2477038844 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128017650 con 0x7fa13810f940 2023-09-23T09:30:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.788+0000 7fa1357fa700 1 -- 172.21.15.45:0/2477038844 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7fa12800fb00 con 0x7fa13810f940 2023-09-23T09:30:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.789+0000 7fa1357fa700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa1240a18b0 0x7fa1240a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.789+0000 7fa1357fa700 1 -- 172.21.15.45:0/2477038844 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa1280ce700 con 0x7fa13810f940 2023-09-23T09:30:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.789+0000 7fa137fff700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa1240a18b0 0x7fa1240a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa137fff700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa1240a18b0 0x7fa1240a3d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b450 tx=0x7fa13000d040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 Processor -- start 2023-09-23T09:30:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 -- start start 2023-09-23T09:30:21.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:21 smithi098 ceph-mon[102386]: pgmap v30: 97 pgs: 97 active+clean; 1.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 60 MiB/s wr, 2.30k op/s 2023-09-23T09:30:21.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:21 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3567734225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1419460 con 0x558fe14b0400 2023-09-23T09:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe140ac20 con 0x558fe14b0d20 2023-09-23T09:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.791+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1408540 con 0x558fe14bbbd0 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33274/0 (socket says 172.21.15.45:33274) 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 -- 172.21.15.45:0/840434565 learned_addr learned my addr 172.21.15.45:0/840434565 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa13c981700 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 -- 172.21.15.45:0/840434565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 msgr2=0x558fe14bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 -- 172.21.15.45:0/840434565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 msgr2=0x558fe14bb690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14bb690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.792+0000 7fa137fff700 1 -- 172.21.15.45:0/840434565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fe14668d0 con 0x558fe14b0400 2023-09-23T09:30:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.793+0000 7fa137fff700 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14b07e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa130016760 tx=0x7fa130011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c290737c2a3760 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.793+0000 7fa12e7fc700 1 -- 172.21.15.45:0/840434565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130010040 con 0x558fe14b0400 2023-09-23T09:30:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.793+0000 7fa12e7fc700 1 -- 172.21.15.45:0/840434565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa13001c040 con 0x558fe14b0400 2023-09-23T09:30:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.793+0000 7fa12e7fc700 1 -- 172.21.15.45:0/840434565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130020a00 con 0x558fe14b0400 2023-09-23T09:30:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/840434565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 msgr2=0x558fe14b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14b07e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa130016760 tx=0x7fa130011000 comp rx=0 tx=0).stop 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/840434565 shutdown_connections 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14b07e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/840434565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/840434565 >> 172.21.15.45:0/840434565 conn(0x558fe1425260 msgr2=0x558fe14ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.794+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/840434565 shutdown_connections 2023-09-23T09:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.795+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/840434565 wait complete. 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.795+0000 7fa14a5ceb80 1 -- start start 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.795+0000 7fa14a5ceb80 1 Processor -- start 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14bb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1403e30 con 0x558fe14b0400 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe13a0710 con 0x558fe14b0d20 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe13a03b0 con 0x558fe14bbbd0 2023-09-23T09:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa14a5ceb80 10 client.0 _ll_register_callbacks cb 0x558fe11fd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa1377fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14bb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14bb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33280/0 (socket says 172.21.15.45:33280) 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa137fff700 1 -- 172.21.15.45:0/1534622684 learned_addr learned my addr 172.21.15.45:0/1534622684 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa1377fe700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54632/0 (socket says 172.21.15.45:54632) 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa13c981700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa13c981700 1 -- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 msgr2=0x558fe14b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa13c981700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa13c981700 1 -- 172.21.15.45:0/1534622684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 msgr2=0x558fe14bb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.796+0000 7fa13c981700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14bb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.797+0000 7fa13c981700 1 -- 172.21.15.45:0/1534622684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa130016410 con 0x558fe14bbbd0 2023-09-23T09:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.797+0000 7fa137fff700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14bb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.797+0000 7fa13c981700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14b8e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa120010650 tx=0x7fa120010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0442c98d68af7d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.798+0000 7fa1177fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.798+0000 7fa1177fe700 1 -- 172.21.15.45:0/1534622684 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200096d0 con 0x558fe14bbbd0 2023-09-23T09:30:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.798+0000 7fa1177fe700 1 -- 172.21.15.45:0/1534622684 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa120017070 con 0x558fe14bbbd0 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.798+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fe14a4250 con 0x558fe14bbbd0 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.798+0000 7fa1177fe700 1 -- 172.21.15.45:0/1534622684 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa120009e50 con 0x558fe14bbbd0 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.798+0000 7fa14a5ceb80 10 client.25372 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.798+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558fe122ede0 con 0x558fe14bbbd0 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa1177fe700 1 -- 172.21.15.45:0/1534622684 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fe14a4250 con 0x558fe14bbbd0 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa1177fe700 1 -- 172.21.15.45:0/1534622684 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa12001cbb0 con 0x558fe14bbbd0 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa14a5ceb80 10 client.25372 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa14a5ceb80 10 client.25372 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-23T09:30:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa14a5ceb80 20 client.25372 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe14cb690 0x558fe14cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa14a5ceb80 4 client.25372 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fe13fce10 con 0x558fe14cb690 2023-09-23T09:30:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.799+0000 7fa137fff700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe14cb690 0x558fe14cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.802+0000 7fa137fff700 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe14cb690 0x558fe14cdab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5e4e8f41762e9e3 server_cookie=bced02f7650a5106 in_seq=0 out_seq=0 2023-09-23T09:30:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.802+0000 7fa1177fe700 10 client.25372 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:30:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.802+0000 7fa1177fe700 1 -- 172.21.15.45:0/1534622684 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fe13fce10 con 0x558fe14cb690 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.802+0000 7fa1177fe700 10 client.25372 handle_command_reply: tid=0 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.803+0000 7fa14a5ceb80 1 client.25372 shutdown 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.803+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 msgr2=0x558fe14b8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.803+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14b8e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa120010650 tx=0x7fa120010960 comp rx=0 tx=0).stop 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 shutdown_connections 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe14cb690 0x558fe14cdab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe14b8e20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14bb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/1534622684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0d20 0x558fe14b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 >> 172.21.15.45:0/1534622684 conn(0x558fe1425260 msgr2=0x558fe14ac320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa1177fe700 0 client.25372 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:30:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 shutdown_connections 2023-09-23T09:30:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/1534622684 wait complete. 2023-09-23T09:30:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.804+0000 7fa14a5ceb80 20 client.25372 trim_cache size 0 max 16384 2023-09-23T09:30:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.825+0000 7fa14a5ceb80 1 Processor -- start 2023-09-23T09:30:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.825+0000 7fa14a5ceb80 1 -- start start 2023-09-23T09:30:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14b0d20 0x558fe14b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14bbbd0 0x558fe14b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1284920 con 0x558fe14b0400 2023-09-23T09:30:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe14b1c50 con 0x558fe14bbbd0 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe14c2d40 con 0x558fe14b0d20 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33288/0 (socket says 172.21.15.45:33288) 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa137fff700 1 -- 172.21.15.45:0/697525660 learned_addr learned my addr 172.21.15.45:0/697525660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa1377fe700 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14b0d20 0x558fe14b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa13c981700 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14bbbd0 0x558fe14b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa1377fe700 1 -- 172.21.15.45:0/697525660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14bbbd0 msgr2=0x558fe14b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa1377fe700 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14bbbd0 0x558fe14b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa1377fe700 1 -- 172.21.15.45:0/697525660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 msgr2=0x558fe14c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa1377fe700 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14c2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa1377fe700 1 -- 172.21.15.45:0/697525660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fe14a8480 con 0x558fe14b0d20 2023-09-23T09:30:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.826+0000 7fa137fff700 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.827+0000 7fa1377fe700 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14b0d20 0x558fe14b3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1280cbcc0 tx=0x7fa128005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10513b3248c6467d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.827+0000 7fa12effd700 1 -- 172.21.15.45:0/697525660 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12809bba0 con 0x558fe14b0d20 2023-09-23T09:30:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.827+0000 7fa12effd700 1 -- 172.21.15.45:0/697525660 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1280b2030 con 0x558fe14b0d20 2023-09-23T09:30:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.827+0000 7fa12effd700 1 -- 172.21.15.45:0/697525660 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12809f3d0 con 0x558fe14b0d20 2023-09-23T09:30:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/697525660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14b0d20 msgr2=0x558fe14b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14b0d20 0x558fe14b3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1280cbcc0 tx=0x7fa128005c00 comp rx=0 tx=0).stop 2023-09-23T09:30:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/697525660 shutdown_connections 2023-09-23T09:30:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14bbbd0 0x558fe14b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0400 0x558fe14c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/697525660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14b0d20 0x558fe14b3b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/697525660 >> 172.21.15.45:0/697525660 conn(0x558fe1425260 msgr2=0x558fe14c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/697525660 shutdown_connections 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.828+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/697525660 wait complete. 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.829+0000 7fa14a5ceb80 1 -- start start 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.829+0000 7fa14a5ceb80 1 Processor -- start 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.830+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0400 0x558fe15177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.830+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0d20 0x558fe1517d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.830+0000 7fa14a5ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe1518530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.830+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1518c40 con 0x558fe14b0d20 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.830+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1518db0 con 0x558fe14b0400 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.830+0000 7fa14a5ceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1518f70 con 0x558fe14bbbd0 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.831+0000 7fa14a5ceb80 10 client.0 _ll_register_callbacks cb 0x7fa120009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.831+0000 7fa137fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0400 0x558fe15177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.831+0000 7fa13c981700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe1518530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.831+0000 7fa137fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0400 0x558fe15177c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54648/0 (socket says 172.21.15.45:54648) 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.831+0000 7fa137fff700 1 -- 172.21.15.45:0/3333737636 learned_addr learned my addr 172.21.15.45:0/3333737636 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa1377fe700 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0d20 0x558fe1517d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa137fff700 1 -- 172.21.15.45:0/3333737636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 msgr2=0x558fe1518530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa137fff700 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe1518530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa137fff700 1 -- 172.21.15.45:0/3333737636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0d20 msgr2=0x558fe1517d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa137fff700 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0d20 0x558fe1517d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa137fff700 1 -- 172.21.15.45:0/3333737636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fe14668d0 con 0x558fe14b0400 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa137fff700 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0400 0x558fe15177c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa130009b00 tx=0x7fa1300203f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c77b41dfbcbf12 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa1177fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa1177fe700 1 -- 172.21.15.45:0/3333737636 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130010040 con 0x558fe14b0400 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa1177fe700 1 -- 172.21.15.45:0/3333737636 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa13002e040 con 0x558fe14b0400 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.832+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa13001c040 con 0x558fe14b0400 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.833+0000 7fa1177fe700 1 -- 172.21.15.45:0/3333737636 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa130032540 con 0x558fe14b0400 2023-09-23T09:30:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.833+0000 7fa14a5ceb80 10 client.25361 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.833+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fa1381154e0 con 0x558fe14b0400 2023-09-23T09:30:21.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:21 smithi123 ceph-mon[103577]: pgmap v30: 97 pgs: 97 active+clean; 1.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 60 MiB/s wr, 2.30k op/s 2023-09-23T09:30:21.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:21 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3567734225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa1177fe700 1 -- 172.21.15.45:0/3333737636 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa13001c040 con 0x558fe14b0400 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa1177fe700 1 -- 172.21.15.45:0/3333737636 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa130032050 con 0x558fe14b0400 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa14a5ceb80 10 client.25361 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa14a5ceb80 10 client.25361 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa14a5ceb80 20 client.25361 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe1520770 0x558fe1522b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa14a5ceb80 4 client.25361 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 --> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558fe13fcb30 con 0x558fe1520770 2023-09-23T09:30:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.834+0000 7fa1377fe700 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe1520770 0x558fe1522b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.836+0000 7fa1177fe700 10 client.25361 ms_handle_connect on v2:172.21.15.98:6836/767375327 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.836+0000 7fa1377fe700 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe1520770 0x558fe1522b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e66ac8c5bf9ed813 server_cookie=a371dd61033555e2 in_seq=0 out_seq=0 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.836+0000 7fa1177fe700 1 -- 172.21.15.45:0/3333737636 <== mds.1 v2:172.21.15.98:6836/767375327 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558fe13fcb30 con 0x558fe1520770 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.836+0000 7fa1177fe700 10 client.25361 handle_command_reply: tid=0 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.836+0000 7fa14a5ceb80 1 client.25361 shutdown 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0400 msgr2=0x558fe15177c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0400 0x558fe15177c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa130009b00 tx=0x7fa1300203f0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 shutdown_connections 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:6836/767375327,v1:172.21.15.98:6837/767375327] conn(0x558fe1520770 0x558fe1522b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fe14bbbd0 0x558fe1518530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fe14b0400 0x558fe15177c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 --2- 172.21.15.45:0/3333737636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558fe14b0d20 0x558fe1517d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 >> 172.21.15.45:0/3333737636 conn(0x558fe1425260 msgr2=0x558fe14b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa1177fe700 0 client.25361 ms_handle_reset on v2:172.21.15.98:6836/767375327 2023-09-23T09:30:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 shutdown_connections 2023-09-23T09:30:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.837+0000 7fa14a5ceb80 1 -- 172.21.15.45:0/3333737636 wait complete. 2023-09-23T09:30:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa14a5ceb80 20 client.25361 trim_cache size 0 max 16384 2023-09-23T09:30:21.843 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-09-23T09:30:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa1240a18b0 msgr2=0x7fa1240a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa1240a18b0 0x7fa1240a3d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b450 tx=0x7fa13000d040 comp rx=0 tx=0).stop 2023-09-23T09:30:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f940 msgr2=0x7fa138116ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f940 0x7fa138116ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa12800b2e0 tx=0x7fa128005720 comp rx=0 tx=0).stop 2023-09-23T09:30:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 shutdown_connections 2023-09-23T09:30:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa1240a18b0 0x7fa1240a3d70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13811a600 0x7fa138111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.838+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa13810f4d0 0x7fa1381165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.839+0000 7fa13e3e4700 1 --2- 172.21.15.45:0/2477038844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa13810f940 0x7fa138116ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.839+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 >> 172.21.15.45:0/2477038844 conn(0x7fa13806d0e0 msgr2=0x7fa13806f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.839+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 shutdown_connections 2023-09-23T09:30:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:21.839+0000 7fa13e3e4700 1 -- 172.21.15.45:0/2477038844 wait complete. 2023-09-23T09:30:21.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T09:30:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 35.0 s, 4352.0 tps, lat 23.002 ms stddev 26.390 2023-09-23T09:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.064+0000 7fcc57291700 1 -- 172.21.15.45:0/2038202854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010dbc0 msgr2=0x7fcc5010dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.064+0000 7fcc57291700 1 --2- 172.21.15.45:0/2038202854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010dbc0 0x7fcc5010dfa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0099c0 tx=0x7fcc4c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.064+0000 7fcc57291700 1 -- 172.21.15.45:0/2038202854 shutdown_connections 2023-09-23T09:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.064+0000 7fcc57291700 1 --2- 172.21.15.45:0/2038202854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010d230 0x7fcc50114ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.064+0000 7fcc57291700 1 --2- 172.21.15.45:0/2038202854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010dbc0 0x7fcc5010dfa0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.064+0000 7fcc57291700 1 --2- 172.21.15.45:0/2038202854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010c870 0x7fcc5010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.064+0000 7fcc57291700 1 -- 172.21.15.45:0/2038202854 >> 172.21.15.45:0/2038202854 conn(0x7fcc5006d0e0 msgr2=0x7fcc5006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.066+0000 7fcc57291700 1 -- 172.21.15.45:0/2038202854 shutdown_connections 2023-09-23T09:30:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.066+0000 7fcc57291700 1 -- 172.21.15.45:0/2038202854 wait complete. 2023-09-23T09:30:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.067+0000 7fcc57291700 1 Processor -- start 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.067+0000 7fcc57291700 1 -- start start 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc57291700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010c870 0x7fcc5019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc57291700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010d230 0x7fcc501a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc57291700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010dbc0 0x7fcc501a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc57291700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a35c0 con 0x7fcc5010c870 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc57291700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a3730 con 0x7fcc5010d230 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc57291700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a38f0 con 0x7fcc5010dbc0 2023-09-23T09:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5482c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010d230 0x7fcc501a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5502d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010c870 0x7fcc5019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5482c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010d230 0x7fcc501a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54660/0 (socket says 172.21.15.45:54660) 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5482c700 1 -- 172.21.15.45:0/3616997323 learned_addr learned my addr 172.21.15.45:0/3616997323 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5582e700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010dbc0 0x7fcc501a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5582e700 1 -- 172.21.15.45:0/3616997323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010d230 msgr2=0x7fcc501a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5582e700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010d230 0x7fcc501a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5582e700 1 -- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010c870 msgr2=0x7fcc5019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5582e700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010c870 0x7fcc5019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.068+0000 7fcc5582e700 1 -- 172.21.15.45:0/3616997323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0095f0 con 0x7fcc5010dbc0 2023-09-23T09:30:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.069+0000 7fcc5582e700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010dbc0 0x7fcc501a2fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800efc0 tx=0x7fcc4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.070+0000 7fcc467fc700 1 -- 172.21.15.45:0/3616997323 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48018070 con 0x7fcc5010dbc0 2023-09-23T09:30:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.070+0000 7fcc467fc700 1 -- 172.21.15.45:0/3616997323 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4800f040 con 0x7fcc5010dbc0 2023-09-23T09:30:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.070+0000 7fcc467fc700 1 -- 172.21.15.45:0/3616997323 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800cc80 con 0x7fcc5010dbc0 2023-09-23T09:30:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.070+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc501a3b70 con 0x7fcc5010dbc0 2023-09-23T09:30:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.070+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc501ac530 con 0x7fcc5010dbc0 2023-09-23T09:30:22.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.076+0000 7fcc6347bb80 1 Processor -- start 2023-09-23T09:30:22.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.077+0000 7fcc467fc700 1 -- 172.21.15.45:0/3616997323 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7fcc48003a00 con 0x7fcc5010dbc0 2023-09-23T09:30:22.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.077+0000 7fcc6347bb80 1 -- start start 2023-09-23T09:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.078+0000 7fcc467fc700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a18b0 0x7fcc3c0a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.078+0000 7fcc467fc700 1 -- 172.21.15.45:0/3616997323 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc480cc7a0 con 0x7fcc5010dbc0 2023-09-23T09:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc5482c700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a18b0 0x7fcc3c0a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bc0940 0x55df08bc0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bd4270 0x55df08bd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08b29460 con 0x55df08bc1f90 2023-09-23T09:30:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08b1ac20 con 0x55df08bd4270 2023-09-23T09:30:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08b18540 con 0x55df08bc0940 2023-09-23T09:30:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc5502d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc5502d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33316/0 (socket says 172.21.15.45:33316) 2023-09-23T09:30:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc5502d700 1 -- 172.21.15.45:0/4246267726 learned_addr learned my addr 172.21.15.45:0/4246267726 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.079+0000 7fcc5482c700 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bc0940 0x55df08bc0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.080+0000 7fcc5502d700 1 -- 172.21.15.45:0/4246267726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bc0940 msgr2=0x55df08bc0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.080+0000 7fcc5502d700 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bc0940 0x55df08bc0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.080+0000 7fcc5502d700 1 -- 172.21.15.45:0/4246267726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bd4270 msgr2=0x55df08bd6660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:30:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.080+0000 7fcc5502d700 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bd4270 0x55df08bd6660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.080+0000 7fcc5502d700 1 -- 172.21.15.45:0/4246267726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df08b768d0 con 0x55df08bc1f90 2023-09-23T09:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.080+0000 7fcc5482c700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a18b0 0x7fcc3c0a3d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc50115960 tx=0x7fcc400119c0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.080+0000 7fcc5502d700 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc0400 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009fd0 tx=0x7fcc4c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1432f7f3ec0224 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.081+0000 7fcc3b7fe700 1 -- 172.21.15.45:0/4246267726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00b990 con 0x55df08bc1f90 2023-09-23T09:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.081+0000 7fcc3b7fe700 1 -- 172.21.15.45:0/4246267726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4c01e040 con 0x55df08bc1f90 2023-09-23T09:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.081+0000 7fcc3b7fe700 1 -- 172.21.15.45:0/4246267726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00fdf0 con 0x55df08bc1f90 2023-09-23T09:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.082+0000 7fcc6347bb80 1 -- 172.21.15.45:0/4246267726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 msgr2=0x55df08bc0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.082+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc0400 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009fd0 tx=0x7fcc4c005f00 comp rx=0 tx=0).stop 2023-09-23T09:30:22.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.084+0000 7fcc6347bb80 1 -- 172.21.15.45:0/4246267726 shutdown_connections 2023-09-23T09:30:22.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.084+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bd4270 0x55df08bd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.084+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc0400 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.084+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/4246267726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bc0940 0x55df08bc0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.084+0000 7fcc6347bb80 1 -- 172.21.15.45:0/4246267726 >> 172.21.15.45:0/4246267726 conn(0x55df08b35260 msgr2=0x55df08bbc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:22.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.084+0000 7fcc6347bb80 1 -- 172.21.15.45:0/4246267726 shutdown_connections 2023-09-23T09:30:22.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.084+0000 7fcc6347bb80 1 -- 172.21.15.45:0/4246267726 wait complete. 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 -- start start 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 Processor -- start 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 0x55df08bc71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08bc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08b13e30 con 0x55df08bc1f90 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08ab0710 con 0x55df08bc0940 2023-09-23T09:30:22.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08ab03b0 con 0x55df08bd4270 2023-09-23T09:30:22.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.085+0000 7fcc6347bb80 10 client.0 _ll_register_callbacks cb 0x55df0890d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:22.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.086+0000 7fcc5482c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.086+0000 7fcc5482c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33324/0 (socket says 172.21.15.45:33324) 2023-09-23T09:30:22.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.086+0000 7fcc5482c700 1 -- 172.21.15.45:0/2498247628 learned_addr learned my addr 172.21.15.45:0/2498247628 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:22.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.088+0000 7fcc5482c700 1 -- 172.21.15.45:0/2498247628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 msgr2=0x55df08bc4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:22.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.088+0000 7fcc5482c700 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08bc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.088+0000 7fcc5482c700 1 -- 172.21.15.45:0/2498247628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 msgr2=0x55df08bc71b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:30:22.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.088+0000 7fcc5482c700 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 0x55df08bc71b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.088+0000 7fcc5482c700 1 -- 172.21.15.45:0/2498247628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc50107e20 con 0x55df08bc1f90 2023-09-23T09:30:22.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.088+0000 7fcc5482c700 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc43c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcc40020a90 tx=0x7fcc400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f454ee60168bd482 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:22.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.092+0000 7fcc38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:30:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.093+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2498247628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4000fb60 con 0x55df08bc1f90 2023-09-23T09:30:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.093+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2498247628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc4000fcc0 con 0x55df08bc1f90 2023-09-23T09:30:22.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.094+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df08bb4250 con 0x55df08bc1f90 2023-09-23T09:30:22.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.094+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2498247628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc40009da0 con 0x55df08bc1f90 2023-09-23T09:30:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.094+0000 7fcc6347bb80 10 client.15720 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.094+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55df0893ede0 con 0x55df08bc1f90 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.094+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2498247628 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df08bb4250 con 0x55df08bc1f90 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2498247628 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcc40026070 con 0x55df08bc1f90 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc6347bb80 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc6347bb80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc6347bb80 20 client.15720 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08be3f00 0x55df08be6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc6347bb80 4 client.15720 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df08b0ce10 con 0x55df08be3f00 2023-09-23T09:30:22.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.095+0000 7fcc5502d700 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08be3f00 0x55df08be6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.096+0000 7fcc38ff9700 10 client.15720 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:22.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.096+0000 7fcc5502d700 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08be3f00 0x55df08be6320 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4391ba411aa5395 server_cookie=8633eb5d5c74f515 in_seq=0 out_seq=0 2023-09-23T09:30:22.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.096+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2498247628 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df08b0ce10 con 0x55df08be3f00 2023-09-23T09:30:22.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.096+0000 7fcc38ff9700 10 client.15720 handle_command_reply: tid=0 2023-09-23T09:30:22.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.096+0000 7fcc6347bb80 1 client.15720 shutdown 2023-09-23T09:30:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.098+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 msgr2=0x55df08bc43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.098+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc43c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcc40020a90 tx=0x7fcc400047c0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 shutdown_connections 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08be3f00 0x55df08be6320 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08bc4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc38ff9700 0 client.15720 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 0x55df08bc71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2498247628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08bc43c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 >> 172.21.15.45:0/2498247628 conn(0x55df08b35260 msgr2=0x55df08bbc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 shutdown_connections 2023-09-23T09:30:22.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2498247628 wait complete. 2023-09-23T09:30:22.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.099+0000 7fcc6347bb80 20 client.15720 trim_cache size 0 max 16384 2023-09-23T09:30:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 Processor -- start 2023-09-23T09:30:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 -- start start 2023-09-23T09:30:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 0x55df08bdaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08be0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08bbaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08994920 con 0x55df08bc1f90 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08bc5100 con 0x55df08bc0940 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08bb4100 con 0x55df08bd4270 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc5482c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08be0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc5482c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08be0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33330/0 (socket says 172.21.15.45:33330) 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.112+0000 7fcc5482c700 1 -- 172.21.15.45:0/1264181237 learned_addr learned my addr 172.21.15.45:0/1264181237 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5582e700 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08bbaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5502d700 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 0x55df08bdaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5482c700 1 -- 172.21.15.45:0/1264181237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 msgr2=0x55df08bbaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5482c700 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08bbaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5482c700 1 -- 172.21.15.45:0/1264181237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 msgr2=0x55df08bdaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5482c700 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 0x55df08bdaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5482c700 1 -- 172.21.15.45:0/1264181237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df08927c60 con 0x55df08bc1f90 2023-09-23T09:30:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc5482c700 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08be0210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcc400054c0 tx=0x7fcc400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3d580743c9b373 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc3bfff700 1 -- 172.21.15.45:0/1264181237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40016040 con 0x55df08bc1f90 2023-09-23T09:30:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc3bfff700 1 -- 172.21.15.45:0/1264181237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4000fe20 con 0x55df08bc1f90 2023-09-23T09:30:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.113+0000 7fcc3bfff700 1 -- 172.21.15.45:0/1264181237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40004a90 con 0x55df08bc1f90 2023-09-23T09:30:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 -- 172.21.15.45:0/1264181237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 msgr2=0x55df08be0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08be0210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcc400054c0 tx=0x7fcc400046b0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 -- 172.21.15.45:0/1264181237 shutdown_connections 2023-09-23T09:30:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08bbaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc0940 0x55df08bdaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/1264181237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc1f90 0x55df08be0210 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 -- 172.21.15.45:0/1264181237 >> 172.21.15.45:0/1264181237 conn(0x55df08b35260 msgr2=0x55df08bc0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 -- 172.21.15.45:0/1264181237 shutdown_connections 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 -- 172.21.15.45:0/1264181237 wait complete. 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 -- start start 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.114+0000 7fcc6347bb80 1 Processor -- start 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc0940 0x55df08bd9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc1f90 0x55df08bd9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08c30d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08c312b0 con 0x55df08bc0940 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08c31420 con 0x55df08bc1f90 2023-09-23T09:30:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08c315e0 con 0x55df08bd4270 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 10 client.0 _ll_register_callbacks cb 0x55df0890d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc0940 0x55df08bd9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc0940 0x55df08bd9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33336/0 (socket says 172.21.15.45:33336) 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 -- 172.21.15.45:0/2878159463 learned_addr learned my addr 172.21.15.45:0/2878159463 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5582e700 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08c30d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5482c700 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc1f90 0x55df08bd9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 -- 172.21.15.45:0/2878159463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 msgr2=0x55df08c30d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08c30d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 -- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc1f90 msgr2=0x55df08bd9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc1f90 0x55df08bd9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 -- 172.21.15.45:0/2878159463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df0893ede0 con 0x55df08bc0940 2023-09-23T09:30:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc5502d700 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc0940 0x55df08bd9110 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009fd0 tx=0x7fcc4c00f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353e17beaa1a127a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2878159463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c031040 con 0x55df08bc0940 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc4c01e040 con 0x55df08bc0940 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.115+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2878159463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc4c00fdb0 con 0x55df08bc0940 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2878159463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc4c00b730 con 0x55df08bc0940 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 10 client.15726 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55df08965250 con 0x55df08bc0940 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2878159463 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4c042020 con 0x55df08bc0940 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2878159463 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcc4c021070 con 0x55df08bc0940 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 20 client.15726 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08c38920 0x55df08c3ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 4 client.15726 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df08b0cb30 con 0x55df08c38920 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.116+0000 7fcc5482c700 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08c38920 0x55df08c3ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc38ff9700 10 client.15726 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc5482c700 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08c38920 0x55df08c3ad40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b25e2105ba279d4 server_cookie=5a90ac0235d863b8 in_seq=0 out_seq=0 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc38ff9700 1 -- 172.21.15.45:0/2878159463 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df08b0cb30 con 0x55df08c38920 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc38ff9700 10 client.15726 handle_command_reply: tid=0 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 client.15726 shutdown 2023-09-23T09:30:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc0940 msgr2=0x55df08bd9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc0940 0x55df08bd9110 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009fd0 tx=0x7fcc4c00f930 comp rx=0 tx=0).stop 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 shutdown_connections 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55df08c38920 0x55df08c3ad40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df08bd4270 0x55df08c30d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc38ff9700 0 client.15726 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df08bc0940 0x55df08bd9110 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 --2- 172.21.15.45:0/2878159463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df08bc1f90 0x55df08bd9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 >> 172.21.15.45:0/2878159463 conn(0x55df08b35260 msgr2=0x55df08bdf690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 shutdown_connections 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 1 -- 172.21.15.45:0/2878159463 wait complete. 2023-09-23T09:30:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc6347bb80 20 client.15726 trim_cache size 0 max 16384 2023-09-23T09:30:22.127 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-09-23T09:30:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a18b0 msgr2=0x7fcc3c0a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.117+0000 7fcc57291700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a18b0 0x7fcc3c0a3d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc50115960 tx=0x7fcc400119c0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010dbc0 msgr2=0x7fcc501a2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010dbc0 0x7fcc501a2fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800efc0 tx=0x7fcc4800c3a0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 shutdown_connections 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010dbc0 0x7fcc501a2fc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a18b0 0x7fcc3c0a3d70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010c870 0x7fcc5019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 --2- 172.21.15.45:0/3616997323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010d230 0x7fcc501a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 >> 172.21.15.45:0/3616997323 conn(0x7fcc5006d0e0 msgr2=0x7fcc50109840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 shutdown_connections 2023-09-23T09:30:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:22.118+0000 7fcc57291700 1 -- 172.21.15.45:0/3616997323 wait complete. 2023-09-23T09:30:23.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T09:30:23.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T09:30:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 -- 172.21.15.45:0/1601120001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae341019a0 msgr2=0x7fae341059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 --2- 172.21.15.45:0/1601120001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae341019a0 0x7fae341059e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 -- 172.21.15.45:0/1601120001 shutdown_connections 2023-09-23T09:30:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 --2- 172.21.15.45:0/1601120001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae34106110 0x7fae3410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 --2- 172.21.15.45:0/1601120001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae34100ff0 0x7fae341013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 --2- 172.21.15.45:0/1601120001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae341019a0 0x7fae341059e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 -- 172.21.15.45:0/1601120001 >> 172.21.15.45:0/1601120001 conn(0x7fae340fc870 msgr2=0x7fae340fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 -- 172.21.15.45:0/1601120001 shutdown_connections 2023-09-23T09:30:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.338+0000 7fae3a209700 1 -- 172.21.15.45:0/1601120001 wait complete. 2023-09-23T09:30:23.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.339+0000 7fae3a209700 1 Processor -- start 2023-09-23T09:30:23.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.339+0000 7fae3a209700 1 -- start start 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae3a209700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae34100ff0 0x7fae34196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae3a209700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae34106110 0x7fae34197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae3a209700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae3419b870 0x7fae3419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae3a209700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419c2f0 con 0x7fae34100ff0 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae3a209700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419c460 con 0x7fae34106110 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae3a209700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419c620 con 0x7fae3419b870 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae34100ff0 0x7fae34196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae34100ff0 0x7fae34196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33350/0 (socket says 172.21.15.45:33350) 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae2affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae34106110 0x7fae34197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 -- 172.21.15.45:0/3543439104 learned_addr learned my addr 172.21.15.45:0/3543439104 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae33fff700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae3419b870 0x7fae3419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 -- 172.21.15.45:0/3543439104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae3419b870 msgr2=0x7fae3419bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae3419b870 0x7fae3419bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 -- 172.21.15.45:0/3543439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae34106110 msgr2=0x7fae34197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae34106110 0x7fae34197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 -- 172.21.15.45:0/3543439104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24009520 con 0x7fae34100ff0 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae2affd700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae34106110 0x7fae34197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae33fff700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae3419b870 0x7fae3419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae337fe700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae34100ff0 0x7fae34196ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae2400eb30 tx=0x7fae2400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.340+0000 7fae317fa700 1 -- 172.21.15.45:0/3543439104 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24009880 con 0x7fae34100ff0 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.341+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c0095f0 con 0x7fae34100ff0 2023-09-23T09:30:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.341+0000 7fae317fa700 1 -- 172.21.15.45:0/3543439104 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae240044e0 con 0x7fae34100ff0 2023-09-23T09:30:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.341+0000 7fae317fa700 1 -- 172.21.15.45:0/3543439104 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24003e90 con 0x7fae34100ff0 2023-09-23T09:30:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.341+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae341a3f20 con 0x7fae34100ff0 2023-09-23T09:30:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.343+0000 7fae317fa700 1 -- 172.21.15.45:0/3543439104 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7fae24004000 con 0x7fae34100ff0 2023-09-23T09:30:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.343+0000 7fae317fa700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae200a18b0 0x7fae200a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.343+0000 7fae317fa700 1 -- 172.21.15.45:0/3543439104 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae24014070 con 0x7fae34100ff0 2023-09-23T09:30:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.343+0000 7fae2affd700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae200a18b0 0x7fae200a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae2affd700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae200a18b0 0x7fae200a3d70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00b2e0 tx=0x7fae1c005960 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 Processor -- start 2023-09-23T09:30:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 -- start start 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb7892480 0x55beb7892860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb78a2060 0x55beb78a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb77fb4e0 con 0x55beb7892da0 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb77ecca0 con 0x55beb7892480 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.344+0000 7fae463f3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb77ea5c0 con 0x55beb78a2060 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33360/0 (socket says 172.21.15.45:33360) 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 -- 172.21.15.45:0/1088076905 learned_addr learned my addr 172.21.15.45:0/1088076905 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae337fe700 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb7892480 0x55beb7892860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae33fff700 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb78a2060 0x55beb78a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 -- 172.21.15.45:0/1088076905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb78a2060 msgr2=0x55beb78a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb78a2060 0x55beb78a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 -- 172.21.15.45:0/1088076905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb7892480 msgr2=0x55beb7892860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb7892480 0x55beb7892860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 -- 172.21.15.45:0/1088076905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beb7848950 con 0x55beb7892da0 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae337fe700 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb7892480 0x55beb7892860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:23.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae2affd700 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a1b20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5542a8fcc1bfae30 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae29ffb700 1 -- 172.21.15.45:0/1088076905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c02b070 con 0x55beb7892da0 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae29ffb700 1 -- 172.21.15.45:0/1088076905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae1c01e040 con 0x55beb7892da0 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.345+0000 7fae29ffb700 1 -- 172.21.15.45:0/1088076905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c024e60 con 0x55beb7892da0 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 -- 172.21.15.45:0/1088076905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 msgr2=0x55beb78a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a1b20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c020000 comp rx=0 tx=0).stop 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 -- 172.21.15.45:0/1088076905 shutdown_connections 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb78a2060 0x55beb78a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb7892480 0x55beb7892860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1088076905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a1b20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 -- 172.21.15.45:0/1088076905 >> 172.21.15.45:0/1088076905 conn(0x55beb78072e0 msgr2=0x55beb788e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 -- 172.21.15.45:0/1088076905 shutdown_connections 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 -- 172.21.15.45:0/1088076905 wait complete. 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.346+0000 7fae463f3b80 1 -- start start 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 1 Processor -- start 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb789eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb789f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb77e5eb0 con 0x55beb7892da0 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb7782790 con 0x55beb78a2060 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb7782430 con 0x55beb7892480 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae463f3b80 10 client.0 _ll_register_callbacks cb 0x55beb75df880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae2affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb789f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb789eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae2affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb789f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33366/0 (socket says 172.21.15.45:33366) 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae2affd700 1 -- 172.21.15.45:0/2380879897 learned_addr learned my addr 172.21.15.45:0/2380879897 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.347+0000 7fae33fff700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae337fe700 1 -- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 msgr2=0x55beb789f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae337fe700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae337fe700 1 -- 172.21.15.45:0/2380879897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 msgr2=0x55beb789f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae337fe700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb789f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae337fe700 1 -- 172.21.15.45:0/2380879897 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae341a3f20 con 0x55beb7892480 2023-09-23T09:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae2affd700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb789f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae33fff700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae337fe700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb789eea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae240cafe0 tx=0x7fae2409ae30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d8a1a539567ba0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae12ffd700 1 -- 172.21.15.45:0/2380879897 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2409a730 con 0x55beb7892480 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55beb78862d0 con 0x55beb7892480 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae12ffd700 1 -- 172.21.15.45:0/2380879897 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae2409e880 con 0x55beb7892480 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae12ffd700 1 -- 172.21.15.45:0/2380879897 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae240a2ac0 con 0x55beb7892480 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 10 client.25405 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fae3410e500 con 0x55beb7892480 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae12ffd700 1 -- 172.21.15.45:0/2380879897 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55beb78862d0 con 0x55beb7892480 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae12ffd700 1 -- 172.21.15.45:0/2380879897 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fae240a2630 con 0x55beb7892480 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 10 client.25405 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 20 client.25405 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb78b1b50 0x55beb78b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 4 client.25405 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.348+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55beb77dee90 con 0x55beb78b1b50 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.349+0000 7fae2affd700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb78b1b50 0x55beb78b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.349+0000 7fae12ffd700 10 client.25405 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.349+0000 7fae2affd700 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb78b1b50 0x55beb78b3f70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbe0a1224187a16 server_cookie=e3dacbef539dfbf1 in_seq=0 out_seq=0 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae12ffd700 1 -- 172.21.15.45:0/2380879897 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55beb77dee90 con 0x55beb78b1b50 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae12ffd700 10 client.25405 handle_command_reply: tid=0 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 client.25405 shutdown 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 msgr2=0x55beb789eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb789eea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae240cafe0 tx=0x7fae2409ae30 comp rx=0 tx=0).stop 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 shutdown_connections 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb78b1b50 0x55beb78b3f70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae12ffd700 0 client.25405 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb789eea0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2380879897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb789f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 >> 172.21.15.45:0/2380879897 conn(0x55beb78072e0 msgr2=0x55beb788e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 shutdown_connections 2023-09-23T09:30:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 1 -- 172.21.15.45:0/2380879897 wait complete. 2023-09-23T09:30:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.350+0000 7fae463f3b80 20 client.25405 trim_cache size 0 max 16384 2023-09-23T09:30:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 Processor -- start 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 -- start start 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892480 0x55beb789e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892da0 0x55beb789a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb76669a0 con 0x55beb7892480 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb789d420 con 0x55beb78a2060 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae463f3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb789ad30 con 0x55beb7892da0 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892480 0x55beb789e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892480 0x55beb789e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33372/0 (socket says 172.21.15.45:33372) 2023-09-23T09:30:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.367+0000 7fae337fe700 1 -- 172.21.15.45:0/2414435847 learned_addr learned my addr 172.21.15.45:0/2414435847 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae33fff700 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae2affd700 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892da0 0x55beb789a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae337fe700 1 -- 172.21.15.45:0/2414435847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892da0 msgr2=0x55beb789a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae337fe700 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892da0 0x55beb789a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae337fe700 1 -- 172.21.15.45:0/2414435847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 msgr2=0x55beb789a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae337fe700 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae337fe700 1 -- 172.21.15.45:0/2414435847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beb78b8b60 con 0x55beb7892480 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae33fff700 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae337fe700 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892480 0x55beb789e680 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c5b0 tx=0x7fae24096470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a768298ab171c8f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae2a7fc700 1 -- 172.21.15.45:0/2414435847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24099c20 con 0x55beb7892480 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae2a7fc700 1 -- 172.21.15.45:0/2414435847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae240b6060 con 0x55beb7892480 2023-09-23T09:30:23.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.368+0000 7fae2a7fc700 1 -- 172.21.15.45:0/2414435847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240ba130 con 0x55beb7892480 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- 172.21.15.45:0/2414435847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892480 msgr2=0x55beb789e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892480 0x55beb789e680 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c5b0 tx=0x7fae24096470 comp rx=0 tx=0).stop 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- 172.21.15.45:0/2414435847 shutdown_connections 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb789a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892480 0x55beb789e680 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 --2- 172.21.15.45:0/2414435847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892da0 0x55beb789a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- 172.21.15.45:0/2414435847 >> 172.21.15.45:0/2414435847 conn(0x55beb78072e0 msgr2=0x55beb78ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- 172.21.15.45:0/2414435847 shutdown_connections 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- 172.21.15.45:0/2414435847 wait complete. 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- start start 2023-09-23T09:30:23.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 Processor -- start 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb78a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb78a7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb78a7860 con 0x55beb7892da0 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb78a79d0 con 0x55beb78a2060 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.369+0000 7fae463f3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb78a7b90 con 0x55beb7892480 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae463f3b80 10 client.0 _ll_register_callbacks cb 0x7fae2409a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33380/0 (socket says 172.21.15.45:33380) 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 -- 172.21.15.45:0/1723217536 learned_addr learned my addr 172.21.15.45:0/1723217536 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae33fff700 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb78a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae337fe700 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb78a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 -- 172.21.15.45:0/1723217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 msgr2=0x55beb78a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb78a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 -- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 msgr2=0x55beb78a7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb78a7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 -- 172.21.15.45:0/1723217536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3410e500 con 0x55beb7892da0 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae2affd700 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a6d50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c00b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3201fe8179d08c87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:30:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae12ffd700 1 -- 172.21.15.45:0/1723217536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c02b040 con 0x55beb7892da0 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae1c01e040 con 0x55beb7892da0 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae12ffd700 1 -- 172.21.15.45:0/1723217536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae1c024a90 con 0x55beb7892da0 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae12ffd700 1 -- 172.21.15.45:0/1723217536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae1c024c90 con 0x55beb7892da0 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae463f3b80 10 client.15744 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.370+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55beb7610e60 con 0x55beb7892da0 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae12ffd700 1 -- 172.21.15.45:0/1723217536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae1c047020 con 0x55beb7892da0 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae12ffd700 1 -- 172.21.15.45:0/1723217536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fae1c00f460 con 0x55beb7892da0 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae463f3b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae463f3b80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae463f3b80 20 client.15744 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb7906b40 0x55beb7908f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae463f3b80 4 client.15744 mds_command: new command op to 24530 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55beb77debb0 con 0x55beb7906b40 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae337fe700 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb7906b40 0x55beb7908f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae12ffd700 10 client.15744 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.371+0000 7fae337fe700 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb7906b40 0x55beb7908f60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c34a3a71706cd1 server_cookie=9218966b5016defa in_seq=0 out_seq=0 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae12ffd700 1 -- 172.21.15.45:0/1723217536 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55beb77debb0 con 0x55beb7906b40 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae12ffd700 10 client.15744 handle_command_reply: tid=0 2023-09-23T09:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 client.15744 shutdown 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 msgr2=0x55beb78a6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a6d50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c00b600 comp rx=0 tx=0).stop 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 shutdown_connections 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55beb7906b40 0x55beb7908f60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55beb78a2060 0x55beb78a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beb7892480 0x55beb78a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae12ffd700 0 client.15744 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 --2- 172.21.15.45:0/1723217536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55beb7892da0 0x55beb78a6d50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 >> 172.21.15.45:0/1723217536 conn(0x55beb78072e0 msgr2=0x55beb7894310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 shutdown_connections 2023-09-23T09:30:23.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.372+0000 7fae463f3b80 1 -- 172.21.15.45:0/1723217536 wait complete. 2023-09-23T09:30:23.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae463f3b80 20 client.15744 trim_cache size 0 max 16384 2023-09-23T09:30:23.382 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:30:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2023-09-23T09:30:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:30:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2023-09-23T09:30:23.383 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae200a18b0 msgr2=0x7fae200a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae200a18b0 0x7fae200a3d70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00b2e0 tx=0x7fae1c005960 comp rx=0 tx=0).stop 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae34100ff0 msgr2=0x7fae34196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae34100ff0 0x7fae34196ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae2400eb30 tx=0x7fae2400ee40 comp rx=0 tx=0).stop 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 shutdown_connections 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae3419b870 0x7fae3419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae200a18b0 0x7fae200a3d70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae34100ff0 0x7fae34196ed0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 --2- 172.21.15.45:0/3543439104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae34106110 0x7fae34197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 >> 172.21.15.45:0/3543439104 conn(0x7fae340fc870 msgr2=0x7fae340fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 shutdown_connections 2023-09-23T09:30:23.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.373+0000 7fae3a209700 1 -- 172.21.15.45:0/3543439104 wait complete. 2023-09-23T09:30:23.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:30:23.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:23 smithi045 ceph-mon[98627]: pgmap v31: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 74 MiB/s wr, 2.73k op/s 2023-09-23T09:30:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 -- 172.21.15.45:0/2726917437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75680ff6b0 msgr2=0x7f75680ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 --2- 172.21.15.45:0/2726917437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75680ff6b0 0x7f75680ffb30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f755c009990 tx=0x7f755c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 -- 172.21.15.45:0/2726917437 shutdown_connections 2023-09-23T09:30:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 --2- 172.21.15.45:0/2726917437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 0x7f756810c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 --2- 172.21.15.45:0/2726917437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75681054e0 0x7f75681058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 --2- 172.21.15.45:0/2726917437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75680ff6b0 0x7f75680ffb30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 -- 172.21.15.45:0/2726917437 >> 172.21.15.45:0/2726917437 conn(0x7f7568076e10 msgr2=0x7f75680fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 -- 172.21.15.45:0/2726917437 shutdown_connections 2023-09-23T09:30:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.593+0000 7f756cf27700 1 -- 172.21.15.45:0/2726917437 wait complete. 2023-09-23T09:30:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 Processor -- start 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 -- start start 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75680ff6b0 0x7f7568196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 0x7f7568197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75681054e0 0x7f756819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756819bee0 con 0x7f75680ff6b0 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756819c050 con 0x7f7568100070 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f756cf27700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756819c210 con 0x7f75681054e0 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f7565d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 0x7f7568197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f7565d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 0x7f7568197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54742/0 (socket says 172.21.15.45:54742) 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.594+0000 7f7565d9b700 1 -- 172.21.15.45:0/2158574790 learned_addr learned my addr 172.21.15.45:0/2158574790 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.595+0000 7f7566d9d700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75681054e0 0x7f756819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.595+0000 7f7565d9b700 1 -- 172.21.15.45:0/2158574790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75681054e0 msgr2=0x7f756819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.595+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75681054e0 0x7f756819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.595+0000 7f7565d9b700 1 -- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75680ff6b0 msgr2=0x7f7568196f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.595+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75680ff6b0 0x7f7568196f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.595+0000 7f7565d9b700 1 -- 172.21.15.45:0/2158574790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c0095f0 con 0x7f7568100070 2023-09-23T09:30:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.595+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 0x7f7568197440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f755c005910 tx=0x7f755c004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.596+0000 7f75577fe700 1 -- 172.21.15.45:0/2158574790 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c01d070 con 0x7f7568100070 2023-09-23T09:30:23.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.596+0000 7f75577fe700 1 -- 172.21.15.45:0/2158574790 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f755c022470 con 0x7f7568100070 2023-09-23T09:30:23.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.596+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756819c490 con 0x7f7568100070 2023-09-23T09:30:23.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.596+0000 7f75577fe700 1 -- 172.21.15.45:0/2158574790 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c017650 con 0x7f7568100070 2023-09-23T09:30:23.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.596+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75681011a0 con 0x7f7568100070 2023-09-23T09:30:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.599+0000 7f75577fe700 1 -- 172.21.15.45:0/2158574790 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7f755c0225e0 con 0x7f7568100070 2023-09-23T09:30:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.600+0000 7f75577fe700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f75500a1980 0x7f75500a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.600+0000 7f75577fe700 1 -- 172.21.15.45:0/2158574790 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f755c0ce7b0 con 0x7f7568100070 2023-09-23T09:30:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.600+0000 7f7579111b80 1 Processor -- start 2023-09-23T09:30:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7579111b80 1 -- start start 2023-09-23T09:30:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7579111b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac704e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7579111b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7059690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7579111b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac705bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7579111b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac6fb7460 con 0x558ac704ed20 2023-09-23T09:30:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7579111b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac6fa8c20 con 0x558ac7059bd0 2023-09-23T09:30:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7579111b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac6fa6540 con 0x558ac704e400 2023-09-23T09:30:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f756659c700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f75500a1980 0x7f75500a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7566d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac705bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7566d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac705bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54750/0 (socket says 172.21.15.45:54750) 2023-09-23T09:30:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.601+0000 7f7566d9d700 1 -- 172.21.15.45:0/2747499014 learned_addr learned my addr 172.21.15.45:0/2747499014 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.602+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7059690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.602+0000 7f756659c700 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac704e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.602+0000 7f7566d9d700 1 -- 172.21.15.45:0/2747499014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 msgr2=0x558ac704e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.602+0000 7f7566d9d700 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac704e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.602+0000 7f7566d9d700 1 -- 172.21.15.45:0/2747499014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 msgr2=0x558ac7059690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.602+0000 7f7566d9d700 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7059690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.602+0000 7f7566d9d700 1 -- 172.21.15.45:0/2747499014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ac70048d0 con 0x558ac7059bd0 2023-09-23T09:30:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.603+0000 7f7566d9d700 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac705bfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f756000a610 tx=0x7f75600088f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a82df57e21f56c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.603+0000 7f756659c700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f75500a1980 0x7f75500a3e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f75580136c0 tx=0x7f7558009610 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.603+0000 7f7554ff9700 1 -- 172.21.15.45:0/2747499014 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600077f0 con 0x558ac7059bd0 2023-09-23T09:30:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.603+0000 7f7554ff9700 1 -- 172.21.15.45:0/2747499014 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7560010040 con 0x558ac7059bd0 2023-09-23T09:30:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.603+0000 7f7554ff9700 1 -- 172.21.15.45:0/2747499014 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560016ea0 con 0x558ac7059bd0 2023-09-23T09:30:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 -- 172.21.15.45:0/2747499014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 msgr2=0x558ac705bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac705bfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f756000a610 tx=0x7f75600088f0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 -- 172.21.15.45:0/2747499014 shutdown_connections 2023-09-23T09:30:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac705bfc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac704e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 --2- 172.21.15.45:0/2747499014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7059690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 -- 172.21.15.45:0/2747499014 >> 172.21.15.45:0/2747499014 conn(0x558ac6fc3260 msgr2=0x558ac704a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 -- 172.21.15.45:0/2747499014 shutdown_connections 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.604+0000 7f7579111b80 1 -- 172.21.15.45:0/2747499014 wait complete. 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 -- start start 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 Processor -- start 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac7056960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7056ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac70573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac6fa1e30 con 0x558ac704ed20 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac6f3e710 con 0x558ac7059bd0 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac6f3e3b0 con 0x558ac704e400 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7579111b80 10 client.0 _ll_register_callbacks cb 0x558ac6d9b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f7565d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7056ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.605+0000 7f756659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac7056960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f756659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac7056960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35510/0 (socket says 172.21.15.45:35510) 2023-09-23T09:30:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f756659c700 1 -- 172.21.15.45:0/2465210061 learned_addr learned my addr 172.21.15.45:0/2465210061 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f7566d9d700 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac70573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f756659c700 1 -- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 msgr2=0x558ac70573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f756659c700 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac70573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f756659c700 1 -- 172.21.15.45:0/2465210061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 msgr2=0x558ac7056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f756659c700 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7056ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f756659c700 1 -- 172.21.15.45:0/2465210061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756000a2f0 con 0x558ac704e400 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.606+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7056ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f756659c700 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac7056960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75580094f0 tx=0x7f7558009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5cf655e4f0fb72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f754dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f754dffb700 1 -- 172.21.15.45:0/2465210061 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558013ec0 con 0x558ac704e400 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f754dffb700 1 -- 172.21.15.45:0/2465210061 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f755801e3f0 con 0x558ac704e400 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ac7042250 con 0x558ac704e400 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f754dffb700 1 -- 172.21.15.45:0/2465210061 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755801e5b0 con 0x558ac704e400 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f7579111b80 10 client.25423 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558ac6dccde0 con 0x558ac704e400 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.608+0000 7f754dffb700 1 -- 172.21.15.45:0/2465210061 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ac7042250 con 0x558ac704e400 2023-09-23T09:30:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f754dffb700 1 -- 172.21.15.45:0/2465210061 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f75580288a0 con 0x558ac704e400 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f7579111b80 10 client.25423 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f7579111b80 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f7579111b80 20 client.25423 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f7579111b80 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac707a7b0 0x558ac707cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f7579111b80 4 client.25423 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ac6f9ae10 con 0x558ac707a7b0 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.609+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac707a7b0 0x558ac707cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.610+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac707a7b0 0x558ac707cbd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88c0c4762c7e3d7 server_cookie=b3a72d557a88a0af in_seq=0 out_seq=0 2023-09-23T09:30:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.610+0000 7f754dffb700 10 client.25423 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f754dffb700 1 -- 172.21.15.45:0/2465210061 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558ac6f9ae10 con 0x558ac707a7b0 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f754dffb700 10 client.25423 handle_command_reply: tid=0 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 client.25423 shutdown 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 msgr2=0x558ac7056960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac7056960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75580094f0 tx=0x7f7558009260 comp rx=0 tx=0).stop 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 shutdown_connections 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac707a7b0 0x558ac707cbd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac70573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704e400 0x558ac7056960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 --2- 172.21.15.45:0/2465210061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7056ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.611+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 >> 172.21.15.45:0/2465210061 conn(0x558ac6fc3260 msgr2=0x558ac704a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.612+0000 7f754dffb700 0 client.25423 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.612+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 shutdown_connections 2023-09-23T09:30:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.612+0000 7f7579111b80 1 -- 172.21.15.45:0/2465210061 wait complete. 2023-09-23T09:30:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.613+0000 7f7579111b80 20 client.25423 trim_cache size 0 max 16384 2023-09-23T09:30:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 Processor -- start 2023-09-23T09:30:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 -- start start 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac704e400 0x558ac7057210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7053c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac7059bd0 0x558ac7054170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac6e22920 con 0x558ac704ed20 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac70765a0 con 0x558ac704e400 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.637+0000 7f7579111b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac704c0d0 con 0x558ac7059bd0 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac7059bd0 0x558ac7054170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac7059bd0 0x558ac7054170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35520/0 (socket says 172.21.15.45:35520) 2023-09-23T09:30:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 -- 172.21.15.45:0/3106224116 learned_addr learned my addr 172.21.15.45:0/3106224116 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7565d9b700 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7053c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 -- 172.21.15.45:0/3106224116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac704e400 msgr2=0x558ac7057210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac704e400 0x558ac7057210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 -- 172.21.15.45:0/3106224116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 msgr2=0x558ac7053c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7053c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.638+0000 7f7566d9d700 1 -- 172.21.15.45:0/3106224116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ac6db5c60 con 0x558ac7059bd0 2023-09-23T09:30:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.639+0000 7f7566d9d700 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac7059bd0 0x558ac7054170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f756000a660 tx=0x7f7560016bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=267be368852ee07b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.639+0000 7f75557fa700 1 -- 172.21.15.45:0/3106224116 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560016640 con 0x558ac7059bd0 2023-09-23T09:30:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.639+0000 7f75557fa700 1 -- 172.21.15.45:0/3106224116 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75600167a0 con 0x558ac7059bd0 2023-09-23T09:30:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.639+0000 7f75557fa700 1 -- 172.21.15.45:0/3106224116 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600050e0 con 0x558ac7059bd0 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- 172.21.15.45:0/3106224116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac7059bd0 msgr2=0x558ac7054170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac7059bd0 0x558ac7054170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f756000a660 tx=0x7f7560016bc0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- 172.21.15.45:0/3106224116 shutdown_connections 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac7059bd0 0x558ac7054170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac704e400 0x558ac7057210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 --2- 172.21.15.45:0/3106224116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704ed20 0x558ac7053c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- 172.21.15.45:0/3106224116 >> 172.21.15.45:0/3106224116 conn(0x558ac6fc3260 msgr2=0x558ac7055790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- 172.21.15.45:0/3106224116 shutdown_connections 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- 172.21.15.45:0/3106224116 wait complete. 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- start start 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 Processor -- start 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704e400 0x558ac7071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704ed20 0x558ac706fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac7070200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac70708d0 con 0x558ac704e400 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac70ce5f0 con 0x558ac7059bd0 2023-09-23T09:30:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac70ce730 con 0x558ac704ed20 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.640+0000 7f7579111b80 10 client.0 _ll_register_callbacks cb 0x7f755800f750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.641+0000 7f756659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704e400 0x558ac7071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.641+0000 7f756659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704e400 0x558ac7071870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33418/0 (socket says 172.21.15.45:33418) 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.641+0000 7f756659c700 1 -- 172.21.15.45:0/2008583750 learned_addr learned my addr 172.21.15.45:0/2008583750 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f7566d9d700 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac7070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f756659c700 1 -- 172.21.15.45:0/2008583750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704ed20 msgr2=0x558ac706fb60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f756659c700 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704ed20 0x558ac706fb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f756659c700 1 -- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 msgr2=0x558ac7070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f756659c700 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac7070200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f756659c700 1 -- 172.21.15.45:0/2008583750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ac70048d0 con 0x558ac704e400 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f756659c700 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704e400 0x558ac7071870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7558028c00 tx=0x7f7558028f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5060b544bbf76f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f754dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:30:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f754dffb700 1 -- 172.21.15.45:0/2008583750 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558005c80 con 0x558ac704e400 2023-09-23T09:30:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f754dffb700 1 -- 172.21.15.45:0/2008583750 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7558028480 con 0x558ac704e400 2023-09-23T09:30:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.642+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7560010040 con 0x558ac704e400 2023-09-23T09:30:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.643+0000 7f754dffb700 1 -- 172.21.15.45:0/2008583750 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7558028640 con 0x558ac704e400 2023-09-23T09:30:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.643+0000 7f7579111b80 10 client.15759 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.643+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558ac6df3250 con 0x558ac704e400 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.643+0000 7f754dffb700 1 -- 172.21.15.45:0/2008583750 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7558045020 con 0x558ac704e400 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f754dffb700 1 -- 172.21.15.45:0/2008583750 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f75580047c0 con 0x558ac704e400 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f7579111b80 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f7579111b80 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f7579111b80 20 client.15759 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f7579111b80 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac70cfe50 0x558ac70d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f7579111b80 4 client.15759 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ac6f9ab30 con 0x558ac70cfe50 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.644+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac70cfe50 0x558ac70d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.645+0000 7f754dffb700 10 client.15759 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.645+0000 7f7565d9b700 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac70cfe50 0x558ac70d2270 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bbe4bb465e71bb server_cookie=5de7407afad8fb79 in_seq=0 out_seq=0 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f754dffb700 1 -- 172.21.15.45:0/2008583750 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558ac6f9ab30 con 0x558ac70cfe50 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f754dffb700 10 client.15759 handle_command_reply: tid=0 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 client.15759 shutdown 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704e400 msgr2=0x558ac7071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704e400 0x558ac7071870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7558028c00 tx=0x7f7558028f20 comp rx=0 tx=0).stop 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 shutdown_connections 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558ac70cfe50 0x558ac70d2270 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ac7059bd0 0x558ac7070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558ac704e400 0x558ac7071870 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f754dffb700 0 client.15759 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 --2- 172.21.15.45:0/2008583750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac704ed20 0x558ac706fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 >> 172.21.15.45:0/2008583750 conn(0x558ac6fc3260 msgr2=0x558ac7056f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 shutdown_connections 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:30:23.652 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:30:23.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:30:23.653 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:30:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 1 -- 172.21.15.45:0/2008583750 wait complete. 2023-09-23T09:30:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.646+0000 7f7579111b80 20 client.15759 trim_cache size 0 max 16384 2023-09-23T09:30:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.647+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f75500a1980 msgr2=0x7f75500a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f75500a1980 0x7f75500a3e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f75580136c0 tx=0x7f7558009610 comp rx=0 tx=0).stop 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 msgr2=0x7f7568197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 0x7f7568197440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f755c005910 tx=0x7f755c004710 comp rx=0 tx=0).stop 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 shutdown_connections 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75681054e0 0x7f756819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f75500a1980 0x7f75500a3e40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75680ff6b0 0x7f7568196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 --2- 172.21.15.45:0/2158574790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7568100070 0x7f7568197440 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 >> 172.21.15.45:0/2158574790 conn(0x7f7568076e10 msgr2=0x7f75680fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 shutdown_connections 2023-09-23T09:30:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:23.648+0000 7f756cf27700 1 -- 172.21.15.45:0/2158574790 wait complete. 2023-09-23T09:30:23.668 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:30:23.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:23 smithi098 ceph-mon[102386]: pgmap v31: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 74 MiB/s wr, 2.73k op/s 2023-09-23T09:30:23.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:23 smithi123 ceph-mon[103577]: pgmap v31: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 74 MiB/s wr, 2.73k op/s 2023-09-23T09:30:24.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub queued for path: / 2023-09-23T09:30:24.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:30:24.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:30:24.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:30:24.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:30:24.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:24 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub queued for path: / 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:30:24.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:24 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub queued for path: / 2023-09-23T09:30:24.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:30:24.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:30:24.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:30:24.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:30:24.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:24.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:24 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:30:25.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:25 smithi098 ceph-mon[102386]: pgmap v32: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 63 MiB/s wr, 2.52k op/s 2023-09-23T09:30:25.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:25 smithi045 ceph-mon[98627]: pgmap v32: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 63 MiB/s wr, 2.52k op/s 2023-09-23T09:30:25.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:25 smithi123 ceph-mon[103577]: pgmap v32: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 63 MiB/s wr, 2.52k op/s 2023-09-23T09:30:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 40.0 s, 4178.1 tps, lat 23.880 ms stddev 25.996 2023-09-23T09:30:27.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:27 smithi098 ceph-mon[102386]: pgmap v33: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 47 MiB/s wr, 2.20k op/s 2023-09-23T09:30:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:27 smithi045 ceph-mon[98627]: pgmap v33: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 47 MiB/s wr, 2.20k op/s 2023-09-23T09:30:27.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:27 smithi123 ceph-mon[103577]: pgmap v33: 97 pgs: 97 active+clean; 2.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 47 MiB/s wr, 2.20k op/s 2023-09-23T09:30:29.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:29 smithi098 ceph-mon[102386]: pgmap v34: 97 pgs: 97 active+clean; 2.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 58 MiB/s wr, 2.73k op/s 2023-09-23T09:30:29.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:29 smithi045 ceph-mon[98627]: pgmap v34: 97 pgs: 97 active+clean; 2.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 58 MiB/s wr, 2.73k op/s 2023-09-23T09:30:29.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:29 smithi123 ceph-mon[103577]: pgmap v34: 97 pgs: 97 active+clean; 2.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 58 MiB/s wr, 2.73k op/s 2023-09-23T09:30:31.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:31 smithi098 ceph-mon[102386]: pgmap v35: 97 pgs: 97 active+clean; 2.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 27 MiB/s wr, 1.03k op/s 2023-09-23T09:30:31.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:31 smithi045 ceph-mon[98627]: pgmap v35: 97 pgs: 97 active+clean; 2.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 27 MiB/s wr, 1.03k op/s 2023-09-23T09:30:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:31 smithi123 ceph-mon[103577]: pgmap v35: 97 pgs: 97 active+clean; 2.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 27 MiB/s wr, 1.03k op/s 2023-09-23T09:30:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 45.0 s, 3920.8 tps, lat 25.472 ms stddev 25.909 2023-09-23T09:30:32.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:32 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:30:32.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:32 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:30:32.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:32 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:30:33.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:33 smithi098 ceph-mon[102386]: pgmap v36: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 41 MiB/s wr, 1.46k op/s 2023-09-23T09:30:33.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:33 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:30:33.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:33 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:30:33.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:33 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:30:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:33 smithi045 ceph-mon[98627]: pgmap v36: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 41 MiB/s wr, 1.46k op/s 2023-09-23T09:30:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:33 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:30:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:33 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:30:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:33 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:30:33.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:33 smithi123 ceph-mon[103577]: pgmap v36: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 41 MiB/s wr, 1.46k op/s 2023-09-23T09:30:33.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:33 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:30:33.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:33 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:30:33.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:33 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:30:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.604+0000 7f44a01f4700 1 -- 172.21.15.45:0/304915004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44980ffe20 msgr2=0x7f449810fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.604+0000 7f44a01f4700 1 --2- 172.21.15.45:0/304915004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44980ffe20 0x7f449810fa50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f449000b0a0 tx=0x7f449000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.604+0000 7f44a01f4700 1 -- 172.21.15.45:0/304915004 shutdown_connections 2023-09-23T09:30:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.604+0000 7f44a01f4700 1 --2- 172.21.15.45:0/304915004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44980ffe20 0x7f449810fa50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.604+0000 7f44a01f4700 1 --2- 172.21.15.45:0/304915004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4498101ce0 0x7f44980ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.604+0000 7f44a01f4700 1 --2- 172.21.15.45:0/304915004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44981020c0 0x7f44980ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.604+0000 7f44a01f4700 1 -- 172.21.15.45:0/304915004 >> 172.21.15.45:0/304915004 conn(0x7f44980771f0 msgr2=0x7f44980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.606+0000 7f44a01f4700 1 -- 172.21.15.45:0/304915004 shutdown_connections 2023-09-23T09:30:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.606+0000 7f44a01f4700 1 -- 172.21.15.45:0/304915004 wait complete. 2023-09-23T09:30:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.607+0000 7f44a01f4700 1 Processor -- start 2023-09-23T09:30:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f44a01f4700 1 -- start start 2023-09-23T09:30:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f44a01f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44980ffe20 0x7f44981026b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f44a01f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4498101ce0 0x7f4498102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f44a01f4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44981020c0 0x7f44981a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f44a01f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4498103450 con 0x7f4498101ce0 2023-09-23T09:30:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f44a01f4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44981a9d70 con 0x7f44981020c0 2023-09-23T09:30:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f44a01f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44981a9eb0 con 0x7f44980ffe20 2023-09-23T09:30:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f449df90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44980ffe20 0x7f44981026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f449df90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44980ffe20 0x7f44981026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35536/0 (socket says 172.21.15.45:35536) 2023-09-23T09:30:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.608+0000 7f449df90700 1 -- 172.21.15.45:0/2327688715 learned_addr learned my addr 172.21.15.45:0/2327688715 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449e791700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44981020c0 0x7f44981a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449df90700 1 -- 172.21.15.45:0/2327688715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44981020c0 msgr2=0x7f44981a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449df90700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44981020c0 0x7f44981a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449df90700 1 -- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4498101ce0 msgr2=0x7f4498102bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:30:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449df90700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4498101ce0 0x7f4498102bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449df90700 1 -- 172.21.15.45:0/2327688715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4490009d00 con 0x7f44980ffe20 2023-09-23T09:30:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449e791700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44981020c0 0x7f44981a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:30:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f449df90700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44980ffe20 0x7f44981026b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f449400b530 tx=0x7f449400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.609+0000 7f448effd700 1 -- 172.21.15.45:0/2327688715 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494011830 con 0x7f44980ffe20 2023-09-23T09:30:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.610+0000 7f448effd700 1 -- 172.21.15.45:0/2327688715 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4494011e50 con 0x7f44980ffe20 2023-09-23T09:30:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.610+0000 7f448effd700 1 -- 172.21.15.45:0/2327688715 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400f2e0 con 0x7f44980ffe20 2023-09-23T09:30:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.610+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44981aa110 con 0x7f44980ffe20 2023-09-23T09:30:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.610+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44981aa620 con 0x7f44980ffe20 2023-09-23T09:30:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.615+0000 7f44ac3deb80 1 Processor -- start 2023-09-23T09:30:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.615+0000 7f44ac3deb80 1 -- start start 2023-09-23T09:30:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.616+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5933320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.616+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.616+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5940b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.616+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a589c400 con 0x55b5a593e710 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.616+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a588dbc0 con 0x55b5a5933860 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.616+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a588b4e0 con 0x55b5a5932f40 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.616+0000 7f449e791700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5933320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449e791700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5933320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35542/0 (socket says 172.21.15.45:35542) 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449e791700 1 -- 172.21.15.45:0/3235864072 learned_addr learned my addr 172.21.15.45:0/3235864072 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449d78f700 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5940b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449d78f700 1 -- 172.21.15.45:0/3235864072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 msgr2=0x55b5a5933320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449d78f700 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5933320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449d78f700 1 -- 172.21.15.45:0/3235864072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 msgr2=0x55b5a593e1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:30:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449d78f700 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593e1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449d78f700 1 -- 172.21.15.45:0/3235864072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5a58e9870 con 0x55b5a593e710 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449e791700 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5933320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f448effd700 1 -- 172.21.15.45:0/2327688715 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7f4494011990 con 0x7f44980ffe20 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.617+0000 7f449d78f700 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5940b00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f44980fc110 tx=0x7f4488009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a9b39b77f9ec2e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.618+0000 7f4483fff700 1 -- 172.21.15.45:0/3235864072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488004230 con 0x55b5a593e710 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.618+0000 7f4483fff700 1 -- 172.21.15.45:0/3235864072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4488004cf0 con 0x55b5a593e710 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.618+0000 7f4483fff700 1 -- 172.21.15.45:0/3235864072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800eee0 con 0x55b5a593e710 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.618+0000 7f448effd700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f44840a1930 0x7f44840a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.618+0000 7f449e791700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f44840a1930 0x7f44840a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.618+0000 7f448effd700 1 -- 172.21.15.45:0/2327688715 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f44940ccfd0 con 0x7f44980ffe20 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 -- 172.21.15.45:0/3235864072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 msgr2=0x55b5a5940b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5940b00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f44980fc110 tx=0x7f4488009e10 comp rx=0 tx=0).stop 2023-09-23T09:30:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 -- 172.21.15.45:0/3235864072 shutdown_connections 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5940b00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5933320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/3235864072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 -- 172.21.15.45:0/3235864072 >> 172.21.15.45:0/3235864072 conn(0x55b5a58a8200 msgr2=0x55b5a592f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f449e791700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f44840a1930 0x7f44840a3df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4490009850 tx=0x7f4490009780 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 -- 172.21.15.45:0/3235864072 shutdown_connections 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.619+0000 7f44ac3deb80 1 -- 172.21.15.45:0/3235864072 wait complete. 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 -- start start 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 Processor -- start 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5932f40 0x55b5a593e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a593b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a593e710 0x55b5a593bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a5886dd0 con 0x55b5a5933860 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a58236b0 con 0x55b5a5932f40 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a5823350 con 0x55b5a593e710 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.620+0000 7f44ac3deb80 10 client.0 _ll_register_callbacks cb 0x55b5a56807a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449e791700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a593e710 0x55b5a593bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449e791700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a593e710 0x55b5a593bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35550/0 (socket says 172.21.15.45:35550) 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449e791700 1 -- 172.21.15.45:0/363483654 learned_addr learned my addr 172.21.15.45:0/363483654 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449df90700 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5932f40 0x55b5a593e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449d78f700 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a593b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449df90700 1 -- 172.21.15.45:0/363483654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a593e710 msgr2=0x55b5a593bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449df90700 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a593e710 0x55b5a593bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449df90700 1 -- 172.21.15.45:0/363483654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 msgr2=0x55b5a593b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449df90700 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a593b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.621+0000 7f449df90700 1 -- 172.21.15.45:0/363483654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4498110710 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.622+0000 7f449df90700 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5932f40 0x55b5a593e150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44940ca640 tx=0x7f44940999c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acd8424f20745652 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.622+0000 7f44817fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.622+0000 7f44817fa700 1 -- 172.21.15.45:0/363483654 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449409a490 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.622+0000 7f44817fa700 1 -- 172.21.15.45:0/363483654 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4494099d90 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.622+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5a5927090 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44817fa700 1 -- 172.21.15.45:0/363483654 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f449409dee0 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 10 client.25412 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f4488009810 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44817fa700 1 -- 172.21.15.45:0/363483654 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5a5927090 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44817fa700 1 -- 172.21.15.45:0/363483654 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f44940a1c90 con 0x55b5a5932f40 2023-09-23T09:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 10 client.25412 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 10 client.25412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 20 client.25412 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a594e1c0 0x55b5a59505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 4 client.25412 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.623+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5a587fdb0 con 0x55b5a594e1c0 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.625+0000 7f449d78f700 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a594e1c0 0x55b5a59505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.626+0000 7f449d78f700 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a594e1c0 0x55b5a59505e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbc204428e6fde server_cookie=78a8d5bbc62ec05b in_seq=0 out_seq=0 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.626+0000 7f44817fa700 10 client.25412 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44817fa700 1 -- 172.21.15.45:0/363483654 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b5a587fdb0 con 0x55b5a594e1c0 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44817fa700 10 client.25412 handle_command_reply: tid=0 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 client.25412 shutdown 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5932f40 msgr2=0x55b5a593e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5932f40 0x55b5a593e150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44940ca640 tx=0x7f44940999c0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 shutdown_connections 2023-09-23T09:30:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a594e1c0 0x55b5a59505e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a593e710 0x55b5a593bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5932f40 0x55b5a593e150 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/363483654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a593b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 >> 172.21.15.45:0/363483654 conn(0x55b5a58a8200 msgr2=0x55b5a592efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.629+0000 7f44817fa700 0 client.25412 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.630+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 shutdown_connections 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.630+0000 7f44ac3deb80 1 -- 172.21.15.45:0/363483654 wait complete. 2023-09-23T09:30:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.630+0000 7f44ac3deb80 20 client.25412 trim_cache size 0 max 16384 2023-09-23T09:30:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 Processor -- start 2023-09-23T09:30:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 -- start start 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5945250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5936770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a57078c0 con 0x55b5a593e710 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a5930100 con 0x55b5a5933860 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.663+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a5945790 con 0x55b5a5932f40 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449d78f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5945250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5945250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35556/0 (socket says 172.21.15.45:35556) 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449d78f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54800/0 (socket says 172.21.15.45:54800) 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 -- 172.21.15.45:0/2124843484 learned_addr learned my addr 172.21.15.45:0/2124843484 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449e791700 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5936770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 -- 172.21.15.45:0/2124843484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 msgr2=0x55b5a593dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 -- 172.21.15.45:0/2124843484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 msgr2=0x55b5a5936770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5936770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 -- 172.21.15.45:0/2124843484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5a56b1d80 con 0x55b5a5932f40 2023-09-23T09:30:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.664+0000 7f449df90700 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5945250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f449400bcf0 tx=0x7f4494004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447c4e4d6801561a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.665+0000 7f448cff9700 1 -- 172.21.15.45:0/2124843484 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400d040 con 0x55b5a5932f40 2023-09-23T09:30:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.665+0000 7f448cff9700 1 -- 172.21.15.45:0/2124843484 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4494099280 con 0x55b5a5932f40 2023-09-23T09:30:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.665+0000 7f448cff9700 1 -- 172.21.15.45:0/2124843484 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44940a2120 con 0x55b5a5932f40 2023-09-23T09:30:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 -- 172.21.15.45:0/2124843484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 msgr2=0x55b5a5945250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5945250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f449400bcf0 tx=0x7f4494004860 comp rx=0 tx=0).stop 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 -- 172.21.15.45:0/2124843484 shutdown_connections 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a593e710 0x55b5a5936770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5945250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/2124843484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a5933860 0x55b5a593dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 -- 172.21.15.45:0/2124843484 >> 172.21.15.45:0/2124843484 conn(0x55b5a58a8200 msgr2=0x55b5a59499b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 -- 172.21.15.45:0/2124843484 shutdown_connections 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.666+0000 7f44ac3deb80 1 -- 172.21.15.45:0/2124843484 wait complete. 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 -- start start 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 Processor -- start 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5941230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a5941770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a593e710 0x55b5a5941cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a59a1000 con 0x55b5a5933860 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a59a1140 con 0x55b5a593e710 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a59a1300 con 0x55b5a5932f40 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f44ac3deb80 10 client.0 _ll_register_callbacks cb 0x7f4494099b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f449d78f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a5941770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f449df90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5941230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f449d78f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a5941770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33458/0 (socket says 172.21.15.45:33458) 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f449d78f700 1 -- 172.21.15.45:0/1237501244 learned_addr learned my addr 172.21.15.45:0/1237501244 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.667+0000 7f449e791700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a593e710 0x55b5a5941cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.668+0000 7f449d78f700 1 -- 172.21.15.45:0/1237501244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 msgr2=0x55b5a5941230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.668+0000 7f449d78f700 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5941230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.668+0000 7f449d78f700 1 -- 172.21.15.45:0/1237501244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a593e710 msgr2=0x55b5a5941cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.668+0000 7f449d78f700 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a593e710 0x55b5a5941cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.668+0000 7f449d78f700 1 -- 172.21.15.45:0/1237501244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4488009810 con 0x55b5a5933860 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.669+0000 7f449d78f700 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a5941770 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f44880097b0 tx=0x7f4488009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ae722c54dad0bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.669+0000 7f44817fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.669+0000 7f44817fa700 1 -- 172.21.15.45:0/1237501244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448801f030 con 0x55b5a5933860 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.669+0000 7f44817fa700 1 -- 172.21.15.45:0/1237501244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4488014070 con 0x55b5a5933860 2023-09-23T09:30:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.669+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4488004cf0 con 0x55b5a5933860 2023-09-23T09:30:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44817fa700 1 -- 172.21.15.45:0/1237501244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4488005b00 con 0x55b5a5933860 2023-09-23T09:30:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 10 client.15774 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b5a569ac00 con 0x55b5a5933860 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44817fa700 1 -- 172.21.15.45:0/1237501244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4488004cf0 con 0x55b5a5933860 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44817fa700 1 -- 172.21.15.45:0/1237501244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4488028140 con 0x55b5a5933860 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 20 client.15774 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a59a2ad0 0x55b5a59a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 4 client.15774 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.671+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5a587fad0 con 0x55b5a59a2ad0 2023-09-23T09:30:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.672+0000 7f449df90700 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a59a2ad0 0x55b5a59a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.672+0000 7f44817fa700 10 client.15774 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.672+0000 7f449df90700 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a59a2ad0 0x55b5a59a4ef0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a54fbaaf96244c server_cookie=c8381704940152aa in_seq=0 out_seq=0 2023-09-23T09:30:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.700+0000 7f44817fa700 1 -- 172.21.15.45:0/1237501244 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5a587fad0 con 0x55b5a59a2ad0 2023-09-23T09:30:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.700+0000 7f44817fa700 10 client.15774 handle_command_reply: tid=0 2023-09-23T09:30:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.700+0000 7f44ac3deb80 1 client.15774 shutdown 2023-09-23T09:30:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 msgr2=0x55b5a5941770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a5941770 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f44880097b0 tx=0x7f4488009260 comp rx=0 tx=0).stop 2023-09-23T09:30:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 shutdown_connections 2023-09-23T09:30:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b5a59a2ad0 0x55b5a59a4ef0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b5a593e710 0x55b5a5941cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5a5932f40 0x55b5a5941230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 --2- 172.21.15.45:0/1237501244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a5933860 0x55b5a5941770 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 >> 172.21.15.45:0/1237501244 conn(0x55b5a58a8200 msgr2=0x55b5a5936770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44817fa700 0 client.15774 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 shutdown_connections 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 1 -- 172.21.15.45:0/1237501244 wait complete. 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.701+0000 7f44ac3deb80 20 client.15774 trim_cache size 0 max 16384 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:30:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:30:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:30:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f44840a1930 msgr2=0x7f44840a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f44840a1930 0x7f44840a3df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4490009850 tx=0x7f4490009780 comp rx=0 tx=0).stop 2023-09-23T09:30:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44980ffe20 msgr2=0x7f44981026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44980ffe20 0x7f44981026b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f449400b530 tx=0x7f449400b8f0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 shutdown_connections 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44981020c0 0x7f44981a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44980ffe20 0x7f44981026b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f44840a1930 0x7f44840a3df0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 --2- 172.21.15.45:0/2327688715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4498101ce0 0x7f4498102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 >> 172.21.15.45:0/2327688715 conn(0x7f44980771f0 msgr2=0x7f44980fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 shutdown_connections 2023-09-23T09:30:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:34.702+0000 7f44a01f4700 1 -- 172.21.15.45:0/2327688715 wait complete. 2023-09-23T09:30:35.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:35 smithi098 ceph-mon[102386]: pgmap v37: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 28 MiB/s wr, 1.06k op/s 2023-09-23T09:30:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:35 smithi045 ceph-mon[98627]: pgmap v37: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 28 MiB/s wr, 1.06k op/s 2023-09-23T09:30:35.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:35 smithi123 ceph-mon[103577]: pgmap v37: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 28 MiB/s wr, 1.06k op/s 2023-09-23T09:30:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 50.0 s, 2936.8 tps, lat 34.128 ms stddev 69.159 2023-09-23T09:30:37.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:37 smithi098 ceph-mon[102386]: pgmap v38: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 26 MiB/s wr, 991 op/s 2023-09-23T09:30:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:37 smithi045 ceph-mon[98627]: pgmap v38: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 26 MiB/s wr, 991 op/s 2023-09-23T09:30:37.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:37 smithi123 ceph-mon[103577]: pgmap v38: 97 pgs: 97 active+clean; 2.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 26 MiB/s wr, 991 op/s 2023-09-23T09:30:39.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:39 smithi098 ceph-mon[102386]: pgmap v39: 97 pgs: 97 active+clean; 2.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 60 MiB/s wr, 2.72k op/s 2023-09-23T09:30:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:39 smithi045 ceph-mon[98627]: pgmap v39: 97 pgs: 97 active+clean; 2.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 60 MiB/s wr, 2.72k op/s 2023-09-23T09:30:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:39 smithi123 ceph-mon[103577]: pgmap v39: 97 pgs: 97 active+clean; 2.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 60 MiB/s wr, 2.72k op/s 2023-09-23T09:30:41.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:41 smithi098 ceph-mon[102386]: pgmap v40: 97 pgs: 97 active+clean; 2.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 49 MiB/s wr, 2.19k op/s 2023-09-23T09:30:41.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:41 smithi045 ceph-mon[98627]: pgmap v40: 97 pgs: 97 active+clean; 2.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 49 MiB/s wr, 2.19k op/s 2023-09-23T09:30:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:41 smithi123 ceph-mon[103577]: pgmap v40: 97 pgs: 97 active+clean; 2.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 49 MiB/s wr, 2.19k op/s 2023-09-23T09:30:41.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 55.0 s, 4331.3 tps, lat 23.103 ms stddev 24.539 2023-09-23T09:30:43.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:43 smithi098 ceph-mon[102386]: pgmap v41: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 59 MiB/s wr, 2.58k op/s 2023-09-23T09:30:43.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:43 smithi045 ceph-mon[98627]: pgmap v41: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 59 MiB/s wr, 2.58k op/s 2023-09-23T09:30:43.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:43 smithi123 ceph-mon[103577]: pgmap v41: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 59 MiB/s wr, 2.58k op/s 2023-09-23T09:30:45.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:45 smithi098 ceph-mon[102386]: pgmap v42: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 45 MiB/s wr, 2.15k op/s 2023-09-23T09:30:45.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:45 smithi045 ceph-mon[98627]: pgmap v42: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 45 MiB/s wr, 2.15k op/s 2023-09-23T09:30:45.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:45 smithi123 ceph-mon[103577]: pgmap v42: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 45 MiB/s wr, 2.15k op/s 2023-09-23T09:30:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 60.0 s, 4614.3 tps, lat 21.607 ms stddev 23.218 2023-09-23T09:30:47.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:47 smithi098 ceph-mon[102386]: pgmap v43: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 44 MiB/s wr, 2.12k op/s 2023-09-23T09:30:47.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:47 smithi045 ceph-mon[98627]: pgmap v43: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 44 MiB/s wr, 2.12k op/s 2023-09-23T09:30:47.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:47 smithi123 ceph-mon[103577]: pgmap v43: 97 pgs: 97 active+clean; 2.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 44 MiB/s wr, 2.12k op/s 2023-09-23T09:30:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.891+0000 7fd08f9c6700 1 -- 172.21.15.45:0/807757049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0881092d0 msgr2=0x7fd08810c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.891+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/807757049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0881092d0 0x7fd08810c8e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c180 tx=0x7fd08000c490 comp rx=0 tx=0).stop 2023-09-23T09:30:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.892+0000 7fd08f9c6700 1 -- 172.21.15.45:0/807757049 shutdown_connections 2023-09-23T09:30:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.892+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/807757049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08810d010 0x7fd088114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.892+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/807757049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd088108920 0x7fd088108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.892+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/807757049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0881092d0 0x7fd08810c8e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.892+0000 7fd08f9c6700 1 -- 172.21.15.45:0/807757049 >> 172.21.15.45:0/807757049 conn(0x7fd08806cee0 msgr2=0x7fd08806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.894+0000 7fd08f9c6700 1 -- 172.21.15.45:0/807757049 shutdown_connections 2023-09-23T09:30:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.894+0000 7fd08f9c6700 1 -- 172.21.15.45:0/807757049 wait complete. 2023-09-23T09:30:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 Processor -- start 2023-09-23T09:30:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 -- start start 2023-09-23T09:30:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd088108920 0x7fd088116b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08810d010 0x7fd0881170c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd088117600 0x7fd0881b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881b2700 con 0x7fd088108920 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881b2870 con 0x7fd088117600 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08f9c6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881b2a30 con 0x7fd08810d010 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08810d010 0x7fd0881170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08df63700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd088117600 0x7fd0881b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08810d010 0x7fd0881170c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35578/0 (socket says 172.21.15.45:35578) 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08cf61700 1 -- 172.21.15.45:0/3145003432 learned_addr learned my addr 172.21.15.45:0/3145003432 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.895+0000 7fd08df63700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd088117600 0x7fd0881b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54824/0 (socket says 172.21.15.45:54824) 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08d762700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd088108920 0x7fd088116b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08cf61700 1 -- 172.21.15.45:0/3145003432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd088117600 msgr2=0x7fd0881b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd088117600 0x7fd0881b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08cf61700 1 -- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd088108920 msgr2=0x7fd088116b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd088108920 0x7fd088116b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08cf61700 1 -- 172.21.15.45:0/3145003432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780095f0 con 0x7fd08810d010 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08810d010 0x7fd0881170c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd080008c30 tx=0x7fd080008c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd07e7fc700 1 -- 172.21.15.45:0/3145003432 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000ce80 con 0x7fd08810d010 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd07e7fc700 1 -- 172.21.15.45:0/3145003432 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd080003a00 con 0x7fd08810d010 2023-09-23T09:30:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd07e7fc700 1 -- 172.21.15.45:0/3145003432 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080005570 con 0x7fd08810d010 2023-09-23T09:30:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08000ad50 con 0x7fd08810d010 2023-09-23T09:30:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.896+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0881a1a90 con 0x7fd08810d010 2023-09-23T09:30:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.901+0000 7fd07e7fc700 1 -- 172.21.15.45:0/3145003432 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7fd08001a070 con 0x7fd08810d010 2023-09-23T09:30:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.901+0000 7fd07e7fc700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd0740a18b0 0x7fd0740a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.902+0000 7fd07e7fc700 1 -- 172.21.15.45:0/3145003432 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd0800ce170 con 0x7fd08810d010 2023-09-23T09:30:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 Processor -- start 2023-09-23T09:30:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd08d762700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd0740a18b0 0x7fd0740a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 -- start start 2023-09-23T09:30:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342d0860 0x559f342db1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f34239400 con 0x559f342cff40 2023-09-23T09:30:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3422abc0 con 0x559f342db710 2023-09-23T09:30:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f342284e0 con 0x559f342d0860 2023-09-23T09:30:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342d0860 0x559f342db1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342d0860 0x559f342db1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35584/0 (socket says 172.21.15.45:35584) 2023-09-23T09:30:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.903+0000 7fd08cf61700 1 -- 172.21.15.45:0/3626667537 learned_addr learned my addr 172.21.15.45:0/3626667537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08df63700 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08d762700 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08d762700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd0740a18b0 0x7fd0740a3d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd078005fd0 tx=0x7fd078009310 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08cf61700 1 -- 172.21.15.45:0/3626667537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 msgr2=0x559f342ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342ddb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08cf61700 1 -- 172.21.15.45:0/3626667537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 msgr2=0x559f342d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08cf61700 1 -- 172.21.15.45:0/3626667537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f34286870 con 0x559f342d0860 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08df63700 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342d0860 0x559f342db1d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd08009abb0 tx=0x7fd08000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3614caf345ddffa9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.904+0000 7fd0737fe700 1 -- 172.21.15.45:0/3626667537 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080003dc0 con 0x559f342d0860 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.905+0000 7fd0737fe700 1 -- 172.21.15.45:0/3626667537 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08009b5b0 con 0x559f342d0860 2023-09-23T09:30:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.905+0000 7fd0737fe700 1 -- 172.21.15.45:0/3626667537 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08002ddb0 con 0x559f342d0860 2023-09-23T09:30:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.905+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3626667537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342d0860 msgr2=0x559f342db1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.905+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342d0860 0x559f342db1d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd08009abb0 tx=0x7fd08000c490 comp rx=0 tx=0).stop 2023-09-23T09:30:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.906+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3626667537 shutdown_connections 2023-09-23T09:30:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.906+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.906+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.906+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3626667537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342d0860 0x559f342db1d0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd08009abb0 tx=0x7fd08000c490 comp rx=0 tx=0).stop 2023-09-23T09:30:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.906+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3626667537 >> 172.21.15.45:0/3626667537 conn(0x559f34245200 msgr2=0x559f342cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.907+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3626667537 shutdown_connections 2023-09-23T09:30:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.907+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3626667537 wait complete. 2023-09-23T09:30:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.907+0000 7fd09bbb0b80 1 -- start start 2023-09-23T09:30:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.907+0000 7fd09bbb0b80 1 Processor -- start 2023-09-23T09:30:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f34223dd0 con 0x559f342db710 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f341c06b0 con 0x559f342cff40 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f341c0350 con 0x559f342e4b60 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd09bbb0b80 10 client.0 _ll_register_callbacks cb 0x559f3401d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd08df63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd08df63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342e6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35594/0 (socket says 172.21.15.45:35594) 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd08df63700 1 -- 172.21.15.45:0/3668712631 learned_addr learned my addr 172.21.15.45:0/3668712631 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342dc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33484/0 (socket says 172.21.15.45:33484) 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.908+0000 7fd08d762700 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.909+0000 7fd08d762700 1 -- 172.21.15.45:0/3668712631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 msgr2=0x559f342e6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.909+0000 7fd08d762700 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342e6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.909+0000 7fd08d762700 1 -- 172.21.15.45:0/3668712631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 msgr2=0x559f342dc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.909+0000 7fd08d762700 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342dc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.909+0000 7fd08d762700 1 -- 172.21.15.45:0/3668712631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0840098c0 con 0x559f342cff40 2023-09-23T09:30:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.909+0000 7fd08df63700 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.909+0000 7fd08d762700 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342dbce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd07800d950 tx=0x7fd078004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8184c4de6120d291 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd070ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:30:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd070ff9700 1 -- 172.21.15.45:0/3668712631 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078036630 con 0x559f342cff40 2023-09-23T09:30:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd070ff9700 1 -- 172.21.15.45:0/3668712631 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd078036790 con 0x559f342cff40 2023-09-23T09:30:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f342c4090 con 0x559f342cff40 2023-09-23T09:30:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd070ff9700 1 -- 172.21.15.45:0/3668712631 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd078036950 con 0x559f342cff40 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd09bbb0b80 10 client.25430 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fd0881a1a90 con 0x559f342cff40 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.910+0000 7fd070ff9700 1 -- 172.21.15.45:0/3668712631 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f342c4090 con 0x559f342cff40 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd070ff9700 1 -- 172.21.15.45:0/3668712631 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd07803fbd0 con 0x559f342cff40 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd09bbb0b80 10 client.25430 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd09bbb0b80 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd09bbb0b80 20 client.25430 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f342eb4b0 0x559f342ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd09bbb0b80 4 client.25430 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f3421cdb0 con 0x559f342eb4b0 2023-09-23T09:30:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.911+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f342eb4b0 0x559f342ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.912+0000 7fd08cf61700 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f342eb4b0 0x559f342ed8d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83134d9ec479f24 server_cookie=8619c8e0f5418923 in_seq=0 out_seq=0 2023-09-23T09:30:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.912+0000 7fd070ff9700 10 client.25430 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.913+0000 7fd070ff9700 1 -- 172.21.15.45:0/3668712631 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559f3421cdb0 con 0x559f342eb4b0 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.913+0000 7fd070ff9700 10 client.25430 handle_command_reply: tid=0 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.913+0000 7fd09bbb0b80 1 client.25430 shutdown 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 msgr2=0x559f342dbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342dbce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd07800d950 tx=0x7fd078004750 comp rx=0 tx=0).stop 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 shutdown_connections 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f342eb4b0 0x559f342ed8d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342dbce0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/3668712631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342dc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 >> 172.21.15.45:0/3668712631 conn(0x559f34245200 msgr2=0x559f342cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd070ff9700 0 client.25430 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 shutdown_connections 2023-09-23T09:30:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/3668712631 wait complete. 2023-09-23T09:30:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.914+0000 7fd09bbb0b80 20 client.25430 trim_cache size 0 max 16384 2023-09-23T09:30:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 Processor -- start 2023-09-23T09:30:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 -- start start 2023-09-23T09:30:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342e4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f340a48c0 con 0x559f342cff40 2023-09-23T09:30:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f342dc8e0 con 0x559f342db710 2023-09-23T09:30:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f342db110 con 0x559f342e4b60 2023-09-23T09:30:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd08d762700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd08d762700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342e4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33492/0 (socket says 172.21.15.45:33492) 2023-09-23T09:30:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd08cf61700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342d5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54842/0 (socket says 172.21.15.45:54842) 2023-09-23T09:30:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd08d762700 1 -- 172.21.15.45:0/1069571179 learned_addr learned my addr 172.21.15.45:0/1069571179 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.940+0000 7fd08df63700 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd08d762700 1 -- 172.21.15.45:0/1069571179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 msgr2=0x559f342d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd08d762700 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342d6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd08d762700 1 -- 172.21.15.45:0/1069571179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 msgr2=0x559f342d5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd08d762700 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342d5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd08d762700 1 -- 172.21.15.45:0/1069571179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f3404ed80 con 0x559f342cff40 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd08cf61700 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342d5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd08d762700 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342e4230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd078005da0 tx=0x7fd078005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2306d1d61d59d116 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd073fff700 1 -- 172.21.15.45:0/1069571179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0780295b0 con 0x559f342cff40 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd073fff700 1 -- 172.21.15.45:0/1069571179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07802e070 con 0x559f342cff40 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.941+0000 7fd073fff700 1 -- 172.21.15.45:0/1069571179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07803f970 con 0x559f342cff40 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.942+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/1069571179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 msgr2=0x559f342e4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.942+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342e4230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd078005da0 tx=0x7fd078005dd0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.942+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/1069571179 shutdown_connections 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.942+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.942+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342cff40 0x559f342e4230 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.942+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/1069571179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342db710 0x559f342d5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.942+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/1069571179 >> 172.21.15.45:0/1069571179 conn(0x559f34245200 msgr2=0x559f342d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.943+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/1069571179 shutdown_connections 2023-09-23T09:30:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.943+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/1069571179 wait complete. 2023-09-23T09:30:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.943+0000 7fd09bbb0b80 1 -- start start 2023-09-23T09:30:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.943+0000 7fd09bbb0b80 1 Processor -- start 2023-09-23T09:30:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd09bbb0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3433e3f0 con 0x559f342db710 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3433e560 con 0x559f342cff40 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd09bbb0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3433e720 con 0x559f342e4b60 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd09bbb0b80 10 client.0 _ll_register_callbacks cb 0x7fd07800de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd08df63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.944+0000 7fd08df63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342ded40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35608/0 (socket says 172.21.15.45:35608) 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08df63700 1 -- 172.21.15.45:0/2259154075 learned_addr learned my addr 172.21.15.45:0/2259154075 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08cf61700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08d762700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08df63700 1 -- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 msgr2=0x559f342de2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08df63700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342de2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08df63700 1 -- 172.21.15.45:0/2259154075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 msgr2=0x559f342de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08df63700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08df63700 1 -- 172.21.15.45:0/2259154075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f34286870 con 0x559f342e4b60 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08d762700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd08df63700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342ded40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd084010e90 tx=0x7fd08400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf0036685223347 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.945+0000 7fd070ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd070ff9700 1 -- 172.21.15.45:0/2259154075 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084004030 con 0x559f342e4b60 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd070ff9700 1 -- 172.21.15.45:0/2259154075 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd084004190 con 0x559f342e4b60 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd08009b5b0 con 0x559f342e4b60 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd070ff9700 1 -- 172.21.15.45:0/2259154075 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd084004390 con 0x559f342e4b60 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd09bbb0b80 10 client.25459 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fd0881a1a90 con 0x559f342e4b60 2023-09-23T09:30:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd070ff9700 1 -- 172.21.15.45:0/2259154075 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08402b020 con 0x559f342e4b60 2023-09-23T09:30:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.946+0000 7fd070ff9700 1 -- 172.21.15.45:0/2259154075 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd08401d790 con 0x559f342e4b60 2023-09-23T09:30:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd09bbb0b80 10 client.25459 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd09bbb0b80 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd09bbb0b80 20 client.25459 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f3433ff60 0x559f34342380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd09bbb0b80 4 client.25459 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f3421cad0 con 0x559f3433ff60 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd08d762700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f3433ff60 0x559f34342380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd08d762700 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f3433ff60 0x559f34342380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ecb3a8e45d77fe server_cookie=af80144a456abe3f in_seq=0 out_seq=0 2023-09-23T09:30:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.947+0000 7fd070ff9700 10 client.25459 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd070ff9700 1 -- 172.21.15.45:0/2259154075 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f3421cad0 con 0x559f3433ff60 2023-09-23T09:30:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd070ff9700 10 client.25459 handle_command_reply: tid=0 2023-09-23T09:30:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 client.25459 shutdown 2023-09-23T09:30:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 msgr2=0x559f342ded40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342ded40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd084010e90 tx=0x7fd08400f6a0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 shutdown_connections 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f3433ff60 0x559f34342380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f342e4b60 0x559f342ded40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f342cff40 0x559f342de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd070ff9700 0 client.25459 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 --2- 172.21.15.45:0/2259154075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f342db710 0x559f342de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 >> 172.21.15.45:0/2259154075 conn(0x559f34245200 msgr2=0x559f342d5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.974+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 shutdown_connections 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd09bbb0b80 1 -- 172.21.15.45:0/2259154075 wait complete. 2023-09-23T09:30:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd09bbb0b80 20 client.25459 trim_cache size 0 max 16384 2023-09-23T09:30:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:30:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:30:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:30:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:30:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd0740a18b0 msgr2=0x7fd0740a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd0740a18b0 0x7fd0740a3d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd078005fd0 tx=0x7fd078009310 comp rx=0 tx=0).stop 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08810d010 msgr2=0x7fd0881170c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08810d010 0x7fd0881170c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd080008c30 tx=0x7fd080008c60 comp rx=0 tx=0).stop 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 shutdown_connections 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd088117600 0x7fd0881b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fd0740a18b0 0x7fd0740a3d70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd088108920 0x7fd088116b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 --2- 172.21.15.45:0/3145003432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08810d010 0x7fd0881170c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.975+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 >> 172.21.15.45:0/3145003432 conn(0x7fd08806cee0 msgr2=0x7fd088070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.978+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 shutdown_connections 2023-09-23T09:30:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:30:47.978+0000 7fd08f9c6700 1 -- 172.21.15.45:0/3145003432 wait complete. 2023-09-23T09:30:49.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:49 smithi098 ceph-mon[102386]: pgmap v44: 97 pgs: 97 active+clean; 2.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 54 MiB/s wr, 2.56k op/s 2023-09-23T09:30:49.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:49 smithi045 ceph-mon[98627]: pgmap v44: 97 pgs: 97 active+clean; 2.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 54 MiB/s wr, 2.56k op/s 2023-09-23T09:30:49.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:49 smithi123 ceph-mon[103577]: pgmap v44: 97 pgs: 97 active+clean; 2.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 54 MiB/s wr, 2.56k op/s 2023-09-23T09:30:51.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:51 smithi098 ceph-mon[102386]: pgmap v45: 97 pgs: 97 active+clean; 2.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 20 MiB/s wr, 836 op/s 2023-09-23T09:30:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:51 smithi045 ceph-mon[98627]: pgmap v45: 97 pgs: 97 active+clean; 2.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 20 MiB/s wr, 836 op/s 2023-09-23T09:30:51.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:51 smithi123 ceph-mon[103577]: pgmap v45: 97 pgs: 97 active+clean; 2.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 20 MiB/s wr, 836 op/s 2023-09-23T09:30:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 65.0 s, 3435.5 tps, lat 28.542 ms stddev 31.350 2023-09-23T09:30:53.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:30:53.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:53 smithi098 ceph-mon[102386]: pgmap v46: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 36 MiB/s wr, 1.49k op/s 2023-09-23T09:30:53.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:53 smithi045 ceph-mon[98627]: pgmap v46: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 36 MiB/s wr, 1.49k op/s 2023-09-23T09:30:53.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:53 smithi123 ceph-mon[103577]: pgmap v46: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 36 MiB/s wr, 1.49k op/s 2023-09-23T09:30:53.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.909+0000 7f9b80208700 1 -- 172.21.15.45:0/217125680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 msgr2=0x7f9b7811c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.909+0000 7f9b80208700 1 --2- 172.21.15.45:0/217125680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b7811c9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009990 tx=0x7f9b68009ca0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.909+0000 7f9b80208700 1 -- 172.21.15.45:0/217125680 shutdown_connections 2023-09-23T09:30:53.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.909+0000 7f9b80208700 1 --2- 172.21.15.45:0/217125680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b7811c9f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.909+0000 7f9b80208700 1 --2- 172.21.15.45:0/217125680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b7810f4d0 0x7f9b7810d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.909+0000 7f9b80208700 1 --2- 172.21.15.45:0/217125680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b7810f940 0x7f9b7810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.909+0000 7f9b80208700 1 -- 172.21.15.45:0/217125680 >> 172.21.15.45:0/217125680 conn(0x7f9b7806d0e0 msgr2=0x7f9b7806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.912+0000 7f9b80208700 1 -- 172.21.15.45:0/217125680 shutdown_connections 2023-09-23T09:30:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.913+0000 7f9b80208700 1 -- 172.21.15.45:0/217125680 wait complete. 2023-09-23T09:30:53.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 Processor -- start 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 -- start start 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b7810f4d0 0x7f9b781a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b7810f940 0x7f9b781a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b781aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b781aae10 con 0x7f9b7810f940 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b781aaf80 con 0x7f9b7811a600 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b80208700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b781ab140 con 0x7f9b7810f4d0 2023-09-23T09:30:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b7e7a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b781aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b7dfa4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b7810f4d0 0x7f9b781a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b7e7a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b781aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54862/0 (socket says 172.21.15.45:54862) 2023-09-23T09:30:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.914+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/2559414167 learned_addr learned my addr 172.21.15.45:0/2559414167 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b7810f940 0x7f9b781a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/2559414167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b7810f4d0 msgr2=0x7f9b781a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b7810f4d0 0x7f9b781a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b7810f940 msgr2=0x7f9b781a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b7810f940 0x7f9b781a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/2559414167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b680095f0 con 0x7f9b7811a600 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b781aa7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b68006010 tx=0x7f9b680037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b6effd700 1 -- 172.21.15.45:0/2559414167 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6801d070 con 0x7f9b7811a600 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.915+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b781ab3c0 con 0x7f9b7811a600 2023-09-23T09:30:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.916+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b781b2c60 con 0x7f9b7811a600 2023-09-23T09:30:53.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.917+0000 7f9b6effd700 1 -- 172.21.15.45:0/2559414167 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b68005580 con 0x7f9b7811a600 2023-09-23T09:30:53.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.917+0000 7f9b6effd700 1 -- 172.21.15.45:0/2559414167 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800fcc0 con 0x7f9b7811a600 2023-09-23T09:30:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.920+0000 7f9b6effd700 1 -- 172.21.15.45:0/2559414167 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 136425+0+0 (secure 0 0 0) 0x7f9b6802b440 con 0x7f9b7811a600 2023-09-23T09:30:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.920+0000 7f9b6effd700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9b640a18b0 0x7f9b640a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.920+0000 7f9b8c3f2b80 1 Processor -- start 2023-09-23T09:30:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.921+0000 7f9b6effd700 1 -- 172.21.15.45:0/2559414167 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9b680ce490 con 0x7f9b7811a600 2023-09-23T09:30:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.921+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9b640a18b0 0x7f9b640a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.921+0000 7f9b8c3f2b80 1 -- start start 2023-09-23T09:30:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e44400 0x564fb9e447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e4f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9b640a18b0 0x7f9b640a3d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000a400 tx=0x7f9b7000d040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e4fbd0 0x564fb9e51fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9dad460 con 0x564fb9e44400 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9d9ec20 con 0x564fb9e4fbd0 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9d9c540 con 0x564fb9e44d20 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e44400 0x564fb9e447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e44400 0x564fb9e447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33520/0 (socket says 172.21.15.45:33520) 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 -- 172.21.15.45:0/862120495 learned_addr learned my addr 172.21.15.45:0/862120495 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e4fbd0 0x564fb9e51fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e4f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 -- 172.21.15.45:0/862120495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 msgr2=0x564fb9e4f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e4f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 -- 172.21.15.45:0/862120495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e4fbd0 msgr2=0x564fb9e51fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e4fbd0 0x564fb9e51fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.922+0000 7f9b7dfa4700 1 -- 172.21.15.45:0/862120495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fb9dfa8d0 con 0x564fb9e44400 2023-09-23T09:30:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.923+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e44400 0x564fb9e447e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9b70016730 tx=0x7f9b70011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f741be6a1d751866 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.924+0000 7f9b63fff700 1 -- 172.21.15.45:0/862120495 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70010040 con 0x564fb9e44400 2023-09-23T09:30:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.924+0000 7f9b63fff700 1 -- 172.21.15.45:0/862120495 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b7001c040 con 0x564fb9e44400 2023-09-23T09:30:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.924+0000 7f9b63fff700 1 -- 172.21.15.45:0/862120495 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b700208b0 con 0x564fb9e44400 2023-09-23T09:30:53.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/862120495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e44400 msgr2=0x564fb9e447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e44400 0x564fb9e447e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9b70016730 tx=0x7f9b70011000 comp rx=0 tx=0).stop 2023-09-23T09:30:53.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/862120495 shutdown_connections 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e4fbd0 0x564fb9e51fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e44400 0x564fb9e447e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/862120495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e4f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/862120495 >> 172.21.15.45:0/862120495 conn(0x564fb9db9260 msgr2=0x564fb9e40790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/862120495 shutdown_connections 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.925+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/862120495 wait complete. 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.926+0000 7f9b8c3f2b80 1 -- start start 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.926+0000 7f9b8c3f2b80 1 Processor -- start 2023-09-23T09:30:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 0x564fb9e4c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9d97e30 con 0x564fb9e4fbd0 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9d34710 con 0x564fb9e44d20 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9d343b0 con 0x564fb9e44400 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b8c3f2b80 10 client.0 _ll_register_callbacks cb 0x564fb9b91800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54876/0 (socket says 172.21.15.45:54876) 2023-09-23T09:30:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/131719573 learned_addr learned my addr 172.21.15.45:0/131719573 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/131719573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 msgr2=0x564fb9e4c990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:30:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 0x564fb9e4c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/131719573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 msgr2=0x564fb9e4d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.927+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/131719573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b70016410 con 0x564fb9e44d20 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4ced0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b74011290 tx=0x7f9b74011650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b7d6de841b5424 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b617fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b617fa700 1 -- 172.21.15.45:0/131719573 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74009630 con 0x564fb9e44d20 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b617fa700 1 -- 172.21.15.45:0/131719573 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9b74015660 con 0x564fb9e44d20 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fb9e38250 con 0x564fb9e44d20 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b617fa700 1 -- 172.21.15.45:0/131719573 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b74015860 con 0x564fb9e44d20 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b8c3f2b80 10 client.25448 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.928+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564fb9bc2de0 con 0x564fb9e44d20 2023-09-23T09:30:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b617fa700 1 -- 172.21.15.45:0/131719573 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564fb9e38250 con 0x564fb9e44d20 2023-09-23T09:30:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b617fa700 1 -- 172.21.15.45:0/131719573 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b7401ec90 con 0x564fb9e44d20 2023-09-23T09:30:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b8c3f2b80 10 client.25448 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b8c3f2b80 10 client.25448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b8c3f2b80 20 client.25448 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9e708a0 0x564fb9e72cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b8c3f2b80 4 client.25448 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:30:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.929+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fb9d90e10 con 0x564fb9e708a0 2023-09-23T09:30:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.931+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9e708a0 0x564fb9e72cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.932+0000 7f9b617fa700 10 client.25448 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.932+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9e708a0 0x564fb9e72cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e9d87f53629002 server_cookie=81522ca26e9b655 in_seq=0 out_seq=0 2023-09-23T09:30:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.932+0000 7f9b617fa700 1 -- 172.21.15.45:0/131719573 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564fb9d90e10 con 0x564fb9e708a0 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.932+0000 7f9b617fa700 10 client.25448 handle_command_reply: tid=0 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.932+0000 7f9b8c3f2b80 1 client.25448 shutdown 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 msgr2=0x564fb9e4ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4ced0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b74011290 tx=0x7f9b74011650 comp rx=0 tx=0).stop 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 shutdown_connections 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9e708a0 0x564fb9e72cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 0x564fb9e4c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/131719573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4ced0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b617fa700 0 client.25448 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 >> 172.21.15.45:0/131719573 conn(0x564fb9db9260 msgr2=0x564fb9e40380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 shutdown_connections 2023-09-23T09:30:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.933+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/131719573 wait complete. 2023-09-23T09:30:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.934+0000 7f9b8c3f2b80 20 client.25448 trim_cache size 0 max 16384 2023-09-23T09:30:53.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.961+0000 7f9b8c3f2b80 1 Processor -- start 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.961+0000 7f9b8c3f2b80 1 -- start start 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 0x564fb9e472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9c18920 con 0x564fb9e4fbd0 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9e48840 con 0x564fb9e44d20 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9e42080 con 0x564fb9e44400 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b7e7a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b7e7a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33532/0 (socket says 172.21.15.45:33532) 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/3903229105 learned_addr learned my addr 172.21.15.45:0/3903229105 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 0x564fb9e472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.962+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/3903229105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 msgr2=0x564fb9e472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:53.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 0x564fb9e472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/3903229105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 msgr2=0x564fb9e4d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b7e7a5700 1 -- 172.21.15.45:0/3903229105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fb9babc60 con 0x564fb9e4fbd0 2023-09-23T09:30:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4e940 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9b68097ca0 tx=0x7f9b68005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804cbb57da0e3cf0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b6cff9700 1 -- 172.21.15.45:0/3903229105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6809f170 con 0x564fb9e4fbd0 2023-09-23T09:30:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b6cff9700 1 -- 172.21.15.45:0/3903229105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b6800fe20 con 0x564fb9e4fbd0 2023-09-23T09:30:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.963+0000 7f9b6cff9700 1 -- 172.21.15.45:0/3903229105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6809b490 con 0x564fb9e4fbd0 2023-09-23T09:30:53.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.964+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/3903229105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 msgr2=0x564fb9e4e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.964+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4e940 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9b68097ca0 tx=0x7f9b68005040 comp rx=0 tx=0).stop 2023-09-23T09:30:53.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.965+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/3903229105 shutdown_connections 2023-09-23T09:30:53.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.965+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9e4e940 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.965+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44400 0x564fb9e472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.965+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/3903229105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44d20 0x564fb9e4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.965+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/3903229105 >> 172.21.15.45:0/3903229105 conn(0x564fb9db9260 msgr2=0x564fb9e4e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:53.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.965+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/3903229105 shutdown_connections 2023-09-23T09:30:53.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.965+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/3903229105 wait complete. 2023-09-23T09:30:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.966+0000 7f9b8c3f2b80 1 -- start start 2023-09-23T09:30:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.966+0000 7f9b8c3f2b80 1 Processor -- start 2023-09-23T09:30:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.966+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44400 0x564fb9e54950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.966+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e54e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.966+0000 7f9b8c3f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9ec36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9ec3c10 con 0x564fb9e4fbd0 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9ec3d50 con 0x564fb9e44400 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b8c3f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fb9ec3ec0 con 0x564fb9e44d20 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b8c3f2b80 10 client.0 _ll_register_callbacks cb 0x7f9b74009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e54e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7dfa4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44400 0x564fb9e54950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7e7a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9ec36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e54e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35644/0 (socket says 172.21.15.45:35644) 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7dfa4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44400 0x564fb9e54950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54888/0 (socket says 172.21.15.45:54888) 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/2262929601 learned_addr learned my addr 172.21.15.45:0/2262929601 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:30:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/2262929601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44400 msgr2=0x564fb9e54950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44400 0x564fb9e54950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/2262929601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 msgr2=0x564fb9ec36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:30:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9ec36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.967+0000 7f9b7d7a3700 1 -- 172.21.15.45:0/2262929601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b781b2c60 con 0x564fb9e44d20 2023-09-23T09:30:53.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.969+0000 7f9b7e7a5700 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9ec36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:30:53.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.969+0000 7f9b7d7a3700 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e54e90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b74000c00 tx=0x7f9b74009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63757800dc5e9fe8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:30:53.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.969+0000 7f9b617fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:30:53.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b617fa700 1 -- 172.21.15.45:0/2262929601 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74005e10 con 0x564fb9e44d20 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b617fa700 1 -- 172.21.15.45:0/2262929601 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9b74004040 con 0x564fb9e44d20 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b7001c040 con 0x564fb9e44d20 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b617fa700 1 -- 172.21.15.45:0/2262929601 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b74004200 con 0x564fb9e44d20 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 10 client.25471 fetch_fsmap learned FSMap version 26 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564fb9be9250 con 0x564fb9e44d20 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b617fa700 1 -- 172.21.15.45:0/2262929601 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b7001c040 con 0x564fb9e44d20 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b617fa700 1 -- 172.21.15.45:0/2262929601 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b74006040 con 0x564fb9e44d20 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 10 client.25471 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 10 client.25471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 20 client.25471 populate_metadata read hostname 'smithi045' 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9ec5730 0x564fb9ec7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:30:53.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 4 client.25471 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:30:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.970+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564fb9d90b30 con 0x564fb9ec5730 2023-09-23T09:30:53.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.971+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9ec5730 0x564fb9ec7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:30:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b617fa700 10 client.25471 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b7dfa4700 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9ec5730 0x564fb9ec7b50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94613c0225d2c252 server_cookie=a2277edb15576fd3 in_seq=0 out_seq=0 2023-09-23T09:30:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b617fa700 1 -- 172.21.15.45:0/2262929601 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564fb9d90b30 con 0x564fb9ec5730 2023-09-23T09:30:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b617fa700 10 client.25471 handle_command_reply: tid=0 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 client.25471 shutdown 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 msgr2=0x564fb9e54e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e54e90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b74000c00 tx=0x7f9b74009190 comp rx=0 tx=0).stop 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 shutdown_connections 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564fb9ec5730 0x564fb9ec7b50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb9e4fbd0 0x564fb9ec36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b617fa700 0 client.25471 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fb9e44400 0x564fb9e54950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 --2- 172.21.15.45:0/2262929601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fb9e44d20 0x564fb9e54e90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.973+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 >> 172.21.15.45:0/2262929601 conn(0x564fb9db9260 msgr2=0x564fb9e4eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.974+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 shutdown_connections 2023-09-23T09:30:53.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.974+0000 7f9b8c3f2b80 1 -- 172.21.15.45:0/2262929601 wait complete. 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.974+0000 7f9b8c3f2b80 20 client.25471 trim_cache size 0 max 16384 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:30:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:30:53.978 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:30:53.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9b640a18b0 msgr2=0x7f9b640a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9b640a18b0 0x7f9b640a3d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000a400 tx=0x7f9b7000d040 comp rx=0 tx=0).stop 2023-09-23T09:30:53.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 msgr2=0x7f9b781aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:30:53.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b781aa7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b68006010 tx=0x7f9b680037c0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 shutdown_connections 2023-09-23T09:30:53.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b7811a600 0x7f9b781aa7d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b7810f4d0 0x7f9b781a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9b640a18b0 0x7f9b640a3d70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 --2- 172.21.15.45:0/2559414167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b7810f940 0x7f9b781a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:30:53.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.975+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 >> 172.21.15.45:0/2559414167 conn(0x7f9b7806d0e0 msgr2=0x7f9b781185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:30:53.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.976+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 shutdown_connections 2023-09-23T09:30:53.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:30:53.976+0000 7f9b80208700 1 -- 172.21.15.45:0/2559414167 wait complete. 2023-09-23T09:30:53.995 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:30:55.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:55 smithi098 ceph-mon[102386]: pgmap v47: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2023-09-23T09:30:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:55 smithi045 ceph-mon[98627]: pgmap v47: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2023-09-23T09:30:55.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:55 smithi123 ceph-mon[103577]: pgmap v47: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2023-09-23T09:30:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 70.0 s, 2919.4 tps, lat 32.191 ms stddev 57.836 2023-09-23T09:30:57.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:56 smithi098 ceph-mon[102386]: pgmap v48: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2023-09-23T09:30:57.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:56 smithi045 ceph-mon[98627]: pgmap v48: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2023-09-23T09:30:57.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:56 smithi123 ceph-mon[103577]: pgmap v48: 97 pgs: 97 active+clean; 2.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2023-09-23T09:30:58.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:57 smithi098 ceph-mon[102386]: mgrmap e25: x(active, since 90s), standbys: z, y 2023-09-23T09:30:58.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:57 smithi045 ceph-mon[98627]: mgrmap e25: x(active, since 90s), standbys: z, y 2023-09-23T09:30:58.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:57 smithi123 ceph-mon[103577]: mgrmap e25: x(active, since 90s), standbys: z, y 2023-09-23T09:30:59.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:30:58 smithi098 ceph-mon[102386]: pgmap v49: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 MiB/s wr, 2.49k op/s 2023-09-23T09:30:59.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:30:58 smithi045 ceph-mon[98627]: pgmap v49: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 MiB/s wr, 2.49k op/s 2023-09-23T09:30:59.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:30:58 smithi123 ceph-mon[103577]: pgmap v49: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 MiB/s wr, 2.49k op/s 2023-09-23T09:31:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.176+0000 7f0533c21700 1 -- 172.21.15.45:0/1053318628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 msgr2=0x7f052c108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.176+0000 7f0533c21700 1 --2- 172.21.15.45:0/1053318628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 0x7f052c108360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0528009990 tx=0x7f0528009ca0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.185+0000 7f0533c21700 1 -- 172.21.15.45:0/1053318628 shutdown_connections 2023-09-23T09:31:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.185+0000 7f0533c21700 1 --2- 172.21.15.45:0/1053318628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f052c11a600 0x7f052c11c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.185+0000 7f0533c21700 1 --2- 172.21.15.45:0/1053318628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 0x7f052c107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.185+0000 7f0533c21700 1 --2- 172.21.15.45:0/1053318628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 0x7f052c108360 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.185+0000 7f0533c21700 1 -- 172.21.15.45:0/1053318628 >> 172.21.15.45:0/1053318628 conn(0x7f052c06cee0 msgr2=0x7f052c06d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.189+0000 7f0533c21700 1 -- 172.21.15.45:0/1053318628 shutdown_connections 2023-09-23T09:31:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.190+0000 7f0533c21700 1 -- 172.21.15.45:0/1053318628 wait complete. 2023-09-23T09:31:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.190+0000 7f0533c21700 1 Processor -- start 2023-09-23T09:31:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.191+0000 7f0533c21700 1 -- start start 2023-09-23T09:31:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.191+0000 7f0533c21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 0x7f052c1a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.191+0000 7f0533c21700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 0x7f052c1a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.191+0000 7f0533c21700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f052c11a600 0x7f052c1aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.191+0000 7f0533c21700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c1aaf80 con 0x7f052c10f570 2023-09-23T09:31:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.191+0000 7f0533c21700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c1ab0f0 con 0x7f052c11a600 2023-09-23T09:31:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.191+0000 7f0533c21700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c1ab2b0 con 0x7f052c10f9e0 2023-09-23T09:31:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 0x7f052c1a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05311bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 0x7f052c1a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 0x7f052c1a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33554/0 (socket says 172.21.15.45:33554) 2023-09-23T09:31:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 -- 172.21.15.45:0/3282138914 learned_addr learned my addr 172.21.15.45:0/3282138914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 -- 172.21.15.45:0/3282138914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 msgr2=0x7f052c1a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 0x7f052c1a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 -- 172.21.15.45:0/3282138914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f052c11a600 msgr2=0x7f052c1aa940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:31:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f052c11a600 0x7f052c1aa940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.192+0000 7f05319bd700 1 -- 172.21.15.45:0/3282138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05280095f0 con 0x7f052c10f570 2023-09-23T09:31:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.193+0000 7f05319bd700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 0x7f052c1a5d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f051c00b530 tx=0x7f051c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.193+0000 7f0522ffd700 1 -- 172.21.15.45:0/3282138914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c011830 con 0x7f052c10f570 2023-09-23T09:31:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.193+0000 7f0522ffd700 1 -- 172.21.15.45:0/3282138914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f051c011e50 con 0x7f052c10f570 2023-09-23T09:31:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.193+0000 7f0522ffd700 1 -- 172.21.15.45:0/3282138914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00f2e0 con 0x7f052c10f570 2023-09-23T09:31:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.193+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c1ab590 con 0x7f052c10f570 2023-09-23T09:31:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.193+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c110040 con 0x7f052c10f570 2023-09-23T09:31:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.197+0000 7f0522ffd700 1 -- 172.21.15.45:0/3282138914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f051c011990 con 0x7f052c10f570 2023-09-23T09:31:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.198+0000 7f0522ffd700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05180a1920 0x7f05180a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.198+0000 7f0522ffd700 1 -- 172.21.15.45:0/3282138914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f051c0ccc30 con 0x7f052c10f570 2023-09-23T09:31:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.199+0000 7f053fe0bb80 1 Processor -- start 2023-09-23T09:31:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.199+0000 7f053fe0bb80 1 -- start start 2023-09-23T09:31:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8488a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f848951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f84897b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f847f3400 con 0x555f84895710 2023-09-23T09:31:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f847e4bc0 con 0x555f84889f40 2023-09-23T09:31:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f847e24e0 con 0x555f8488a860 2023-09-23T09:31:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f05311bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f848951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f05311bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f848951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35668/0 (socket says 172.21.15.45:35668) 2023-09-23T09:31:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.200+0000 7f05311bc700 1 -- 172.21.15.45:0/3659498049 learned_addr learned my addr 172.21.15.45:0/3659498049 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.201+0000 7f05321be700 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f84897b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.201+0000 7f05321be700 1 -- 172.21.15.45:0/3659498049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 msgr2=0x555f848951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.201+0000 7f05321be700 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f848951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.201+0000 7f05321be700 1 -- 172.21.15.45:0/3659498049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 msgr2=0x555f8488a320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:31:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.201+0000 7f05321be700 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8488a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.201+0000 7f05321be700 1 -- 172.21.15.45:0/3659498049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f84840870 con 0x555f84895710 2023-09-23T09:31:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.201+0000 7f05321be700 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f84897b00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0524009c90 tx=0x7f052400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f6d05fa7fdbae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.202+0000 7f0517fff700 1 -- 172.21.15.45:0/3659498049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052400bd70 con 0x555f84895710 2023-09-23T09:31:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.202+0000 7f0517fff700 1 -- 172.21.15.45:0/3659498049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05240138d0 con 0x555f84895710 2023-09-23T09:31:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.202+0000 7f0517fff700 1 -- 172.21.15.45:0/3659498049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05240128e0 con 0x555f84895710 2023-09-23T09:31:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.202+0000 7f05311bc700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05180a1920 0x7f05180a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 -- 172.21.15.45:0/3659498049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 msgr2=0x555f84897b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f84897b00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0524009c90 tx=0x7f052400b410 comp rx=0 tx=0).stop 2023-09-23T09:31:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 -- 172.21.15.45:0/3659498049 shutdown_connections 2023-09-23T09:31:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f84897b00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8488a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/3659498049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f848951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 -- 172.21.15.45:0/3659498049 >> 172.21.15.45:0/3659498049 conn(0x555f847ff200 msgr2=0x555f848863c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 -- 172.21.15.45:0/3659498049 shutdown_connections 2023-09-23T09:31:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 -- 172.21.15.45:0/3659498049 wait complete. 2023-09-23T09:31:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.203+0000 7f053fe0bb80 1 -- start start 2023-09-23T09:31:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 1 Processor -- start 2023-09-23T09:31:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f05311bc700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05180a1920 0x7f05180a3de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f052801a040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848951a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f84892410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f84892b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f847dddd0 con 0x555f8488a860 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8477a6b0 con 0x555f84889f40 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8477a350 con 0x555f84895710 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f053fe0bb80 10 client.0 _ll_register_callbacks cb 0x555f845d77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f05319bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f05319bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54916/0 (socket says 172.21.15.45:54916) 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.204+0000 7f05319bd700 1 -- 172.21.15.45:0/257535960 learned_addr learned my addr 172.21.15.45:0/257535960 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05311bc700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f84892410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05321be700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f84892b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05319bd700 1 -- 172.21.15.45:0/257535960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 msgr2=0x555f84892b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05319bd700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f84892b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05319bd700 1 -- 172.21.15.45:0/257535960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 msgr2=0x555f84892410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05319bd700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f84892410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05319bd700 1 -- 172.21.15.45:0/257535960 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05240098f0 con 0x555f84889f40 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05321be700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f84892b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05319bd700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848951a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f051c0a1aa0 tx=0x7f051c09a3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a9f5e2b3f41d809 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05157fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05157fa700 1 -- 172.21.15.45:0/257535960 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c09a0a0 con 0x555f84889f40 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05157fa700 1 -- 172.21.15.45:0/257535960 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f051c0a1fd0 con 0x555f84889f40 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f8487e090 con 0x555f84889f40 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f05157fa700 1 -- 172.21.15.45:0/257535960 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f051c0b2200 con 0x555f84889f40 2023-09-23T09:31:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.205+0000 7f053fe0bb80 10 client.25457 fetch_fsmap learned FSMap version 26 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555f84608d80 con 0x555f84889f40 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f05157fa700 1 -- 172.21.15.45:0/257535960 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f8487e090 con 0x555f84889f40 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f05157fa700 1 -- 172.21.15.45:0/257535960 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f051c0a1bd0 con 0x555f84889f40 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f053fe0bb80 10 client.25457 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f053fe0bb80 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f053fe0bb80 20 client.25457 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848a52d0 0x555f848a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f053fe0bb80 4 client.25457 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f847d6db0 con 0x555f848a52d0 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.206+0000 7f05311bc700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848a52d0 0x555f848a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f05157fa700 10 client.25457 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f05311bc700 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848a52d0 0x555f848a76f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e42b4bf41cc327 server_cookie=c8a7ad984e50543f in_seq=0 out_seq=0 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f05157fa700 1 -- 172.21.15.45:0/257535960 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555f847d6db0 con 0x555f848a52d0 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f05157fa700 10 client.25457 handle_command_reply: tid=0 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 client.25457 shutdown 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 msgr2=0x555f848951a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848951a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f051c0a1aa0 tx=0x7f051c09a3c0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 shutdown_connections 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848a52d0 0x555f848a76f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f84892b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f05157fa700 0 client.25457 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848951a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/257535960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f84892410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.207+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 >> 172.21.15.45:0/257535960 conn(0x555f847ff200 msgr2=0x555f84885f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.208+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 shutdown_connections 2023-09-23T09:31:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.208+0000 7f053fe0bb80 1 -- 172.21.15.45:0/257535960 wait complete. 2023-09-23T09:31:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.208+0000 7f053fe0bb80 20 client.25457 trim_cache size 0 max 16384 2023-09-23T09:31:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 Processor -- start 2023-09-23T09:31:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 -- start start 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f8488d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f8488dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8465e8c0 con 0x555f8488a860 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f84894d40 con 0x555f84889f40 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f84887780 con 0x555f84895710 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05311bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f8488d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848a76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54924/0 (socket says 172.21.15.45:54924) 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 -- 172.21.15.45:0/1119690556 learned_addr learned my addr 172.21.15.45:0/1119690556 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 -- 172.21.15.45:0/1119690556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 msgr2=0x555f8488dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 --2- 172.21.15.45:0/1119690556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f8488dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 -- 172.21.15.45:0/1119690556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 msgr2=0x555f8488d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 --2- 172.21.15.45:0/1119690556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f8488d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.223+0000 7f05319bd700 1 -- 172.21.15.45:0/1119690556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f845f1c00 con 0x555f84889f40 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f05319bd700 1 --2- 172.21.15.45:0/1119690556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848a76c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f051c0966b0 tx=0x7f051c099a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc65fc68a5b246 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f0520d86700 1 -- 172.21.15.45:0/1119690556 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00d040 con 0x555f84889f40 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f0520d86700 1 -- 172.21.15.45:0/1119690556 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f051c098f50 con 0x555f84889f40 2023-09-23T09:31:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f0520d86700 1 -- 172.21.15.45:0/1119690556 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c099390 con 0x555f84889f40 2023-09-23T09:31:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1119690556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 msgr2=0x555f848a76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1119690556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848a76c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f051c0966b0 tx=0x7f051c099a10 comp rx=0 tx=0).stop 2023-09-23T09:31:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1119690556 shutdown_connections 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1119690556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f84895710 0x555f8488dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1119690556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f848a76c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1119690556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f8488a860 0x555f8488d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1119690556 >> 172.21.15.45:0/1119690556 conn(0x555f847ff200 msgr2=0x555f848a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.224+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1119690556 shutdown_connections 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1119690556 wait complete. 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 -- start start 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 Processor -- start 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8489a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f8489a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f848f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8489b0e0 con 0x555f84895710 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f848f8600 con 0x555f84889f40 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f848f8740 con 0x555f8488a860 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.225+0000 7f053fe0bb80 10 client.0 _ll_register_callbacks cb 0x7f051c099780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05319bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8489a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05321be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f848f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05319bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8489a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54930/0 (socket says 172.21.15.45:54930) 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05319bd700 1 -- 172.21.15.45:0/1747848226 learned_addr learned my addr 172.21.15.45:0/1747848226 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05321be700 1 -- 172.21.15.45:0/1747848226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 msgr2=0x555f8489a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05321be700 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f8489a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05321be700 1 -- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 msgr2=0x555f8489a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05321be700 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8489a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05321be700 1 -- 172.21.15.45:0/1747848226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f84608d80 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05319bd700 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8489a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05321be700 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f848f80c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0524000f80 tx=0x7f0524013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9f3d48e064d6a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05157fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05157fa700 1 -- 172.21.15.45:0/1747848226 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524012740 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05240138d0 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05157fa700 1 -- 172.21.15.45:0/1747848226 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0524006e40 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f05157fa700 1 -- 172.21.15.45:0/1747848226 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05240128a0 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f053fe0bb80 10 client.15819 fetch_fsmap learned FSMap version 26 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.226+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555f8462f1f0 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f05157fa700 1 -- 172.21.15.45:0/1747848226 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05240138d0 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f05157fa700 1 -- 172.21.15.45:0/1747848226 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0524026060 con 0x555f84895710 2023-09-23T09:31:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f053fe0bb80 10 client.15819 fetch_fsmap finished waiting for FSMap version 26 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f053fe0bb80 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f053fe0bb80 20 client.15819 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848f9e10 0x555f848fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f053fe0bb80 4 client.15819 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f847d6ad0 con 0x555f848f9e10 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f05319bd700 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848f9e10 0x555f848fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f05157fa700 10 client.15819 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:01.227+0000 7f05319bd700 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848f9e10 0x555f848fc230 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc5500590fc306cf server_cookie=d1a056f524dbe955 in_seq=0 out_seq=0 2023-09-23T09:31:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:01 smithi045 ceph-mon[98627]: pgmap v50: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 47 MiB/s wr, 2.04k op/s 2023-09-23T09:31:01.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:01 smithi123 ceph-mon[103577]: pgmap v50: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 47 MiB/s wr, 2.04k op/s 2023-09-23T09:31:01.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:01 smithi098 ceph-mon[102386]: pgmap v50: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 47 MiB/s wr, 2.04k op/s 2023-09-23T09:31:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 75.0 s, 3706.2 tps, lat 29.200 ms stddev 58.457 2023-09-23T09:31:03.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:03 smithi045 ceph-mon[98627]: pgmap v51: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 52 MiB/s wr, 2.39k op/s 2023-09-23T09:31:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:03 smithi123 ceph-mon[103577]: pgmap v51: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 52 MiB/s wr, 2.39k op/s 2023-09-23T09:31:03.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:03 smithi098 ceph-mon[102386]: pgmap v51: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 52 MiB/s wr, 2.39k op/s 2023-09-23T09:31:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.963+0000 7f05157fa700 1 -- 172.21.15.45:0/1747848226 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f847d6ad0 con 0x555f848f9e10 2023-09-23T09:31:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.963+0000 7f05157fa700 10 client.15819 handle_command_reply: tid=0 2023-09-23T09:31:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.963+0000 7f053fe0bb80 1 client.15819 shutdown 2023-09-23T09:31:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.964+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 msgr2=0x555f848f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.964+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f848f80c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0524000f80 tx=0x7f0524013c70 comp rx=0 tx=0).stop 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.964+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 shutdown_connections 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.964+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x555f848f9e10 0x555f848fc230 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f84895710 0x555f848f80c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f84889f40 0x555f8489a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f053fe0bb80 1 --2- 172.21.15.45:0/1747848226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f8488a860 0x555f8489a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 >> 172.21.15.45:0/1747848226 conn(0x555f847ff200 msgr2=0x555f8488d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f05157fa700 0 client.15819 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 shutdown_connections 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f053fe0bb80 1 -- 172.21.15.45:0/1747848226 wait complete. 2023-09-23T09:31:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.965+0000 7f053fe0bb80 20 client.15819 trim_cache size 0 max 16384 2023-09-23T09:31:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:31:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:31:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:31:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:31:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05180a1920 msgr2=0x7f05180a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05180a1920 0x7f05180a3de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f052801a040 comp rx=0 tx=0).stop 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 msgr2=0x7f052c1a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 0x7f052c1a5d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f051c00b530 tx=0x7f051c00b8f0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 shutdown_connections 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f052c11a600 0x7f052c1aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f05180a1920 0x7f05180a3de0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f052801a040 comp rx=0 tx=0).stop 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c10f570 0x7f052c1a5d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 --2- 172.21.15.45:0/3282138914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f052c10f9e0 0x7f052c1a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.966+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 >> 172.21.15.45:0/3282138914 conn(0x7f052c06cee0 msgr2=0x7f052c1185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.968+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 shutdown_connections 2023-09-23T09:31:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:03.968+0000 7f0533c21700 1 -- 172.21.15.45:0/3282138914 wait complete. 2023-09-23T09:31:04.339 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: *** Caught signal (Segmentation fault) ** 2023-09-23T09:31:04.339 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: in thread 7fddac4af700 thread_name:md_log_replay 2023-09-23T09:31:04.339 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T09:31:04.339 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fddbd05ecf0] 2023-09-23T09:31:04.339 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2: /usr/bin/ceph-mds(+0x4f1eff) [0x5587d9828eff] 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5587d982b133] 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 4: (MDLog::_replay_thread()+0x647) [0x5587d9886af7] 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5587d9890041] 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fddbd0541ca] 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 7: clone() 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2023-09-23T09:31:04.037+0000 7fddac4af700 -1 *** Caught signal (Segmentation fault) ** 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: in thread 7fddac4af700 thread_name:md_log_replay 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T09:31:04.340 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fddbd05ecf0] 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2: /usr/bin/ceph-mds(+0x4f1eff) [0x5587d9828eff] 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5587d982b133] 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 4: (MDLog::_replay_thread()+0x647) [0x5587d9886af7] 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5587d9890041] 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fddbd0541ca] 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 7: clone() 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 0> 2023-09-23T09:31:04.037+0000 7fddac4af700 -1 *** Caught signal (Segmentation fault) ** 2023-09-23T09:31:04.341 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: in thread 7fddac4af700 thread_name:md_log_replay 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fddbd05ecf0] 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2: /usr/bin/ceph-mds(+0x4f1eff) [0x5587d9828eff] 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5587d982b133] 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 4: (MDLog::_replay_thread()+0x647) [0x5587d9886af7] 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5587d9890041] 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fddbd0541ca] 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 7: clone() 2023-09-23T09:31:04.342 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 0> 2023-09-23T09:31:04.037+0000 7fddac4af700 -1 *** Caught signal (Segmentation fault) ** 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: in thread 7fddac4af700 thread_name:md_log_replay 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fddbd05ecf0] 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2: /usr/bin/ceph-mds(+0x4f1eff) [0x5587d9828eff] 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5587d982b133] 2023-09-23T09:31:04.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 4: (MDLog::_replay_thread()+0x647) [0x5587d9886af7] 2023-09-23T09:31:04.344 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5587d9890041] 2023-09-23T09:31:04.344 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fddbd0541ca] 2023-09-23T09:31:04.344 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 7: clone() 2023-09-23T09:31:04.344 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-23T09:31:04.344 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[139525]: 2023-09-23T09:31:04.809 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 systemd[1]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-23T09:31:04.838 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:04 smithi123 systemd[1]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.c.service: Failed with result 'exit-code'. 2023-09-23T09:31:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:05 smithi123 ceph-mon[103577]: pgmap v52: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 2.62k op/s 2023-09-23T09:31:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:05 smithi098 ceph-mon[102386]: pgmap v52: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 2.62k op/s 2023-09-23T09:31:05.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:05 smithi045 ceph-mon[98627]: pgmap v52: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 2.62k op/s 2023-09-23T09:31:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 80.0 s, 4165.4 tps, lat 24.004 ms stddev 25.828 2023-09-23T09:31:07.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:07 smithi098 ceph-mon[102386]: pgmap v53: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 2.62k op/s 2023-09-23T09:31:07.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:07 smithi045 ceph-mon[98627]: pgmap v53: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 2.62k op/s 2023-09-23T09:31:07.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:07 smithi123 ceph-mon[103577]: pgmap v53: 97 pgs: 97 active+clean; 2.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 2.62k op/s 2023-09-23T09:31:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:09 smithi098 ceph-mon[102386]: pgmap v54: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 57 MiB/s wr, 2.98k op/s 2023-09-23T09:31:09.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:09 smithi045 ceph-mon[98627]: pgmap v54: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 57 MiB/s wr, 2.98k op/s 2023-09-23T09:31:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:09 smithi123 ceph-mon[103577]: pgmap v54: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 57 MiB/s wr, 2.98k op/s 2023-09-23T09:31:11.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:11 smithi098 ceph-mon[102386]: pgmap v55: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2023-09-23T09:31:11.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:11 smithi045 ceph-mon[98627]: pgmap v55: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2023-09-23T09:31:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:11 smithi123 ceph-mon[103577]: pgmap v55: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2023-09-23T09:31:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 85.0 s, 4632.0 tps, lat 21.577 ms stddev 23.775 2023-09-23T09:31:13.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:13 smithi098 ceph-mon[102386]: pgmap v56: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 29 MiB/s wr, 1.70k op/s 2023-09-23T09:31:13.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:13 smithi045 ceph-mon[98627]: pgmap v56: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 29 MiB/s wr, 1.70k op/s 2023-09-23T09:31:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:13 smithi123 ceph-mon[103577]: pgmap v56: 97 pgs: 97 active+clean; 2.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 29 MiB/s wr, 1.70k op/s 2023-09-23T09:31:15.203 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:14 smithi123 systemd[1]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-23T09:31:15.204 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:14 smithi123 systemd[1]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.c.service: Scheduled restart job, restart counter is at 1. 2023-09-23T09:31:15.204 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:14 smithi123 systemd[1]: Stopped Ceph mds.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:31:15.204 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:14 smithi123 systemd[1]: Starting Ceph mds.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:31:15.533 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:15 smithi123 ceph-mon[103577]: pgmap v57: 97 pgs: 97 active+clean; 2.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2023-09-23T09:31:15.533 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:15 smithi123 podman[143182]: 2023-09-23T09:31:15.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:15 smithi098 ceph-mon[102386]: pgmap v57: 97 pgs: 97 active+clean; 2.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2023-09-23T09:31:15.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:15 smithi045 ceph-mon[98627]: pgmap v57: 97 pgs: 97 active+clean; 2.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2023-09-23T09:31:15.838 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:15 smithi123 bash[143182]: 590c7eced03302de25f38805cbba4b938d902fdd165954b0744bf28d4d96d1cc 2023-09-23T09:31:15.838 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:15 smithi123 ceph-mds[143222]: -- 172.21.15.123:0/3808104296 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55adbc3131e0 con 0x55adbc2e8800 2023-09-23T09:31:15.838 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[143197]: starting mds.c at 2023-09-23T09:31:15.839 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:31:15 smithi123 systemd[1]: Started Ceph mds.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:31:16.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:16 smithi098 ceph-mon[102386]: daemon mds.c restarted 2023-09-23T09:31:16.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:16 smithi045 ceph-mon[98627]: daemon mds.c restarted 2023-09-23T09:31:16.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:16 smithi123 ceph-mon[103577]: daemon mds.c restarted 2023-09-23T09:31:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 90.0 s, 3255.2 tps, lat 30.680 ms stddev 30.491 2023-09-23T09:31:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.128+0000 7f3ca61f2700 1 -- 172.21.15.45:0/1680492107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 msgr2=0x7f3ca010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.128+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/1680492107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca010cb30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009f90 tx=0x7f3c9800b330 comp rx=0 tx=0).stop 2023-09-23T09:31:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.128+0000 7f3ca61f2700 1 -- 172.21.15.45:0/1680492107 shutdown_connections 2023-09-23T09:31:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.128+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/1680492107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca010cb30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.129+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/1680492107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0105d90 0x7f3ca0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.129+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/1680492107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca00ffcd0 0x7f3ca0100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.129+0000 7f3ca61f2700 1 -- 172.21.15.45:0/1680492107 >> 172.21.15.45:0/1680492107 conn(0x7f3ca0076e20 msgr2=0x7f3ca0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.129+0000 7f3ca61f2700 1 -- 172.21.15.45:0/1680492107 shutdown_connections 2023-09-23T09:31:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.129+0000 7f3ca61f2700 1 -- 172.21.15.45:0/1680492107 wait complete. 2023-09-23T09:31:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.130+0000 7f3ca61f2700 1 Processor -- start 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.130+0000 7f3ca61f2700 1 -- start start 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3ca61f2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca00ffcd0 0x7f3ca006c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3ca61f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca006c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3ca61f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0105d90 0x7f3ca00710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3ca61f2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00716e0 con 0x7f3ca0100690 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3ca61f2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0071850 con 0x7f3ca00ffcd0 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3ca61f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0071a10 con 0x7f3ca0105d90 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca006c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca00ffcd0 0x7f3ca006c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca006c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33596/0 (socket says 172.21.15.45:33596) 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 -- 172.21.15.45:0/285778686 learned_addr learned my addr 172.21.15.45:0/285778686 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0105d90 0x7f3ca00710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 -- 172.21.15.45:0/285778686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0105d90 msgr2=0x7f3ca00710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0105d90 0x7f3ca00710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 -- 172.21.15.45:0/285778686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca00ffcd0 msgr2=0x7f3ca006c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca00ffcd0 0x7f3ca006c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9effd700 1 -- 172.21.15.45:0/285778686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98009c40 con 0x7f3ca0100690 2023-09-23T09:31:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.131+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0105d90 0x7f3ca00710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.132+0000 7f3c9effd700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca006c7b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400d950 tx=0x7f3c9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.133+0000 7f3c9cff9700 1 -- 172.21.15.45:0/285778686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c94009930 con 0x7f3ca0100690 2023-09-23T09:31:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.133+0000 7f3c9cff9700 1 -- 172.21.15.45:0/285778686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c94010460 con 0x7f3ca0100690 2023-09-23T09:31:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.133+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca0071cf0 con 0x7f3ca0100690 2023-09-23T09:31:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.133+0000 7f3c9cff9700 1 -- 172.21.15.45:0/285778686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400f620 con 0x7f3ca0100690 2023-09-23T09:31:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.133+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca0072250 con 0x7f3ca0100690 2023-09-23T09:31:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.136+0000 7f3c9cff9700 1 -- 172.21.15.45:0/285778686 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f3c940108d0 con 0x7f3ca0100690 2023-09-23T09:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.137+0000 7f3c9cff9700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c880a1790 0x7f3c880a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.137+0000 7f3cb23dcb80 1 Processor -- start 2023-09-23T09:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.137+0000 7f3cb23dcb80 1 -- start start 2023-09-23T09:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.137+0000 7f3c9cff9700 1 -- 172.21.15.45:0/285778686 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c940cd0f0 con 0x7f3ca0100690 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a13860 0x564db5a2f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a2f720 0x564db5a31b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db597c400 con 0x564db5a13860 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db596dbc0 con 0x564db5a12f40 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db596b4e0 con 0x564db5a2f720 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a2f720 0x564db5a31b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a2f720 0x564db5a31b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35712/0 (socket says 172.21.15.45:35712) 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1074087879 learned_addr learned my addr 172.21.15.45:0/1074087879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3c9effd700 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a13860 0x564db5a2f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.138+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c880a1790 0x7f3c880a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.139+0000 7f3c9f7fe700 1 -- 172.21.15.45:0/1074087879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a2f720 msgr2=0x564db5a31b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.139+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a2f720 0x564db5a31b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.139+0000 7f3c9f7fe700 1 -- 172.21.15.45:0/1074087879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a13860 msgr2=0x564db5a2f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.139+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a13860 0x564db5a2f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.139+0000 7f3c9f7fe700 1 -- 172.21.15.45:0/1074087879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db59c9870 con 0x564db5a12f40 2023-09-23T09:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.140+0000 7f3c9effd700 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a13860 0x564db5a2f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.140+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a13320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005ce0 tx=0x7f3c9003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e67fbd657edb6e69 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.140+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c880a1790 0x7f3c880a3c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005fd0 tx=0x7f3c90009b30 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.144+0000 7f3c85ffb700 1 -- 172.21.15.45:0/1074087879 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90042430 con 0x564db5a12f40 2023-09-23T09:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.144+0000 7f3c85ffb700 1 -- 172.21.15.45:0/1074087879 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c90042a50 con 0x564db5a12f40 2023-09-23T09:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.144+0000 7f3c85ffb700 1 -- 172.21.15.45:0/1074087879 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90040760 con 0x564db5a12f40 2023-09-23T09:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.145+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1074087879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 msgr2=0x564db5a13320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.145+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a13320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005ce0 tx=0x7f3c9003f000 comp rx=0 tx=0).stop 2023-09-23T09:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.149+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1074087879 shutdown_connections 2023-09-23T09:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.149+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a2f720 0x564db5a31b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.149+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a13320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.149+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1074087879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a13860 0x564db5a2f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.149+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1074087879 >> 172.21.15.45:0/1074087879 conn(0x564db5988200 msgr2=0x564db5a0f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.159+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1074087879 shutdown_connections 2023-09-23T09:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.160+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1074087879 wait complete. 2023-09-23T09:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.161+0000 7f3cb23dcb80 1 -- start start 2023-09-23T09:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.161+0000 7f3cb23dcb80 1 Processor -- start 2023-09-23T09:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a22840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a2f720 0x564db5a1db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5966dd0 con 0x564db5a2f720 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db59036b0 con 0x564db5a12f40 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5903350 con 0x564db5a13860 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3cb23dcb80 10 client.0 _ll_register_callbacks cb 0x564db57607a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a2f720 0x564db5a1db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a2f720 0x564db5a1db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33612/0 (socket says 172.21.15.45:33612) 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1311594746 learned_addr learned my addr 172.21.15.45:0/1311594746 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a22840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9effd700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1311594746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 msgr2=0x564db5a22d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 msgr2=0x564db5a22840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a22840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1311594746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca0072250 con 0x564db5a2f720 2023-09-23T09:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a22840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a2f720 0x564db5a1db10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009790 tx=0x7f3c98003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3050e92fdeb7acbc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.162+0000 7f3c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.163+0000 7f3c7effd700 1 -- 172.21.15.45:0/1311594746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800e040 con 0x564db5a2f720 2023-09-23T09:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.163+0000 7f3c7effd700 1 -- 172.21.15.45:0/1311594746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c9800be00 con 0x564db5a2f720 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.163+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564db5a07090 con 0x564db5a2f720 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3c7effd700 1 -- 172.21.15.45:0/1311594746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c98005040 con 0x564db5a2f720 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 10 client.15831 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3c90010e00 con 0x564db5a2f720 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3c7effd700 1 -- 172.21.15.45:0/1311594746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564db5a07090 con 0x564db5a2f720 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3c7effd700 1 -- 172.21.15.45:0/1311594746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3c98019030 con 0x564db5a2f720 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 20 client.15831 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a2de10 0x564db5a2e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 4 client.15831 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db595fdb0 con 0x564db5a2de10 2023-09-23T09:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.164+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a2de10 0x564db5a2e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.165+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a2de10 0x564db5a2e220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321362fce542cacc server_cookie=b674c1707cbf445d in_seq=0 out_seq=0 2023-09-23T09:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.165+0000 7f3c7effd700 10 client.15831 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3c7effd700 1 -- 172.21.15.45:0/1311594746 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564db595fdb0 con 0x564db5a2de10 2023-09-23T09:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3c7effd700 10 client.15831 handle_command_reply: tid=0 2023-09-23T09:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 client.15831 shutdown 2023-09-23T09:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a2f720 msgr2=0x564db5a1db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a2f720 0x564db5a1db10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009790 tx=0x7f3c98003c30 comp rx=0 tx=0).stop 2023-09-23T09:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 shutdown_connections 2023-09-23T09:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a2de10 0x564db5a2e220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a2f720 0x564db5a1db10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a12f40 0x564db5a22840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1311594746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 >> 172.21.15.45:0/1311594746 conn(0x564db5988200 msgr2=0x564db5a0ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3c7effd700 0 client.15831 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 shutdown_connections 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.167+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1311594746 wait complete. 2023-09-23T09:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.168+0000 7f3cb23dcb80 20 client.15831 trim_cache size 0 max 16384 2023-09-23T09:31:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.189+0000 7f3cb23dcb80 1 Processor -- start 2023-09-23T09:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3cb23dcb80 1 -- start start 2023-09-23T09:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a35b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db57e78c0 con 0x564db5a12f40 2023-09-23T09:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5a360a0 con 0x564db5a2f720 2023-09-23T09:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5a13350 con 0x564db5a13860 2023-09-23T09:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3c9f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3c9effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a35b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3c9f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a2a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33616/0 (socket says 172.21.15.45:33616) 2023-09-23T09:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.190+0000 7f3c9f7fe700 1 -- 172.21.15.45:0/2950270605 learned_addr learned my addr 172.21.15.45:0/2950270605 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c9ffff700 1 -- 172.21.15.45:0/2950270605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 msgr2=0x564db5a22d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c9ffff700 1 -- 172.21.15.45:0/2950270605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 msgr2=0x564db5a2a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a2a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c9ffff700 1 -- 172.21.15.45:0/2950270605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db5791d80 con 0x564db5a2f720 2023-09-23T09:31:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c9effd700 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a35b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d010 tx=0x7f3c98003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838431a6a9a0e013 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c867fc700 1 -- 172.21.15.45:0/2950270605 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800e070 con 0x564db5a2f720 2023-09-23T09:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c867fc700 1 -- 172.21.15.45:0/2950270605 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c9801d9f0 con 0x564db5a2f720 2023-09-23T09:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.191+0000 7f3c867fc700 1 -- 172.21.15.45:0/2950270605 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98005900 con 0x564db5a2f720 2023-09-23T09:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.192+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/2950270605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 msgr2=0x564db5a35b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.192+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a35b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d010 tx=0x7f3c98003ca0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.193+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/2950270605 shutdown_connections 2023-09-23T09:31:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.193+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a35b30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.193+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.193+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/2950270605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a22d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.193+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/2950270605 >> 172.21.15.45:0/2950270605 conn(0x564db5988200 msgr2=0x564db5a22630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.193+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/2950270605 shutdown_connections 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.193+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/2950270605 wait complete. 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 -- start start 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 Processor -- start 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a1a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a1a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a1acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5a80c10 con 0x564db5a12f40 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5a80d50 con 0x564db5a2f720 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5a80f10 con 0x564db5a13860 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3cb23dcb80 10 client.0 _ll_register_callbacks cb 0x564db57607a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.194+0000 7f3c9effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a1a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a1a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35728/0 (socket says 172.21.15.45:35728) 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9effd700 1 -- 172.21.15.45:0/1545073210 learned_addr learned my addr 172.21.15.45:0/1545073210 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a1acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a1a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1545073210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 msgr2=0x564db5a1a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a1a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1545073210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 msgr2=0x564db5a1a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a1a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9ffff700 1 -- 172.21.15.45:0/1545073210 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca0071cf0 con 0x564db5a2f720 2023-09-23T09:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a1acb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c98003c70 tx=0x7f3c98005860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54266cf7fce2d87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.195+0000 7f3c7effd700 1 -- 172.21.15.45:0/1545073210 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98039070 con 0x564db5a2f720 2023-09-23T09:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.196+0000 7f3c7effd700 1 -- 172.21.15.45:0/1545073210 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c98035440 con 0x564db5a2f720 2023-09-23T09:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.196+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c90042a50 con 0x564db5a2f720 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.197+0000 7f3c7effd700 1 -- 172.21.15.45:0/1545073210 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c9800e040 con 0x564db5a2f720 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.197+0000 7f3cb23dcb80 10 client.25484 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.197+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3c90010e00 con 0x564db5a2f720 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.197+0000 7f3c7effd700 1 -- 172.21.15.45:0/1545073210 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c90042a50 con 0x564db5a2f720 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3c7effd700 1 -- 172.21.15.45:0/1545073210 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3c9803c020 con 0x564db5a2f720 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3cb23dcb80 10 client.25484 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3cb23dcb80 10 client.25484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3cb23dcb80 20 client.25484 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a821c0 0x564db5a845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3cb23dcb80 4 client.25484 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564db595fad0 con 0x564db5a821c0 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a821c0 0x564db5a845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3c7effd700 10 client.25484 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.198+0000 7f3c9f7fe700 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a821c0 0x564db5a845b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465c8e9533b8c4d7 server_cookie=5f172424d430c9d5 in_seq=0 out_seq=0 2023-09-23T09:31:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.232+0000 7f3c7effd700 1 -- 172.21.15.45:0/1545073210 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564db595fad0 con 0x564db5a821c0 2023-09-23T09:31:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.232+0000 7f3c7effd700 10 client.25484 handle_command_reply: tid=0 2023-09-23T09:31:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.232+0000 7f3cb23dcb80 1 client.25484 shutdown 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 msgr2=0x564db5a1acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a1acb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c98003c70 tx=0x7f3c98005860 comp rx=0 tx=0).stop 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 shutdown_connections 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564db5a821c0 0x564db5a845b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564db5a2f720 0x564db5a1acb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564db5a12f40 0x564db5a1a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 --2- 172.21.15.45:0/1545073210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564db5a13860 0x564db5a1a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 >> 172.21.15.45:0/1545073210 conn(0x564db5988200 msgr2=0x564db5a21430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.234+0000 7f3c7effd700 0 client.25484 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.235+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 shutdown_connections 2023-09-23T09:31:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.235+0000 7f3cb23dcb80 1 -- 172.21.15.45:0/1545073210 wait complete. 2023-09-23T09:31:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.235+0000 7f3cb23dcb80 20 client.25484 trim_cache size 0 max 16384 2023-09-23T09:31:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:31:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:31:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:31:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:31:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c880a1790 msgr2=0x7f3c880a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c880a1790 0x7f3c880a3c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005fd0 tx=0x7f3c90009b30 comp rx=0 tx=0).stop 2023-09-23T09:31:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 msgr2=0x7f3ca006c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca006c7b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400d950 tx=0x7f3c9400dc60 comp rx=0 tx=0).stop 2023-09-23T09:31:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 shutdown_connections 2023-09-23T09:31:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0105d90 0x7f3ca00710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ca00ffcd0 0x7f3ca006c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c880a1790 0x7f3c880a3c50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 --2- 172.21.15.45:0/285778686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca0100690 0x7f3ca006c7b0 secure :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400d950 tx=0x7f3c9400dc60 comp rx=0 tx=0).stop 2023-09-23T09:31:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 >> 172.21.15.45:0/285778686 conn(0x7f3ca0076e20 msgr2=0x7f3ca00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 shutdown_connections 2023-09-23T09:31:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:17.237+0000 7f3ca61f2700 1 -- 172.21.15.45:0/285778686 wait complete. 2023-09-23T09:31:17.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:17 smithi098 ceph-mon[102386]: pgmap v58: 97 pgs: 97 active+clean; 2.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 719 op/s 2023-09-23T09:31:17.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:17 smithi098 ceph-mon[102386]: osdmap e87: 12 total, 12 up, 12 in 2023-09-23T09:31:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:17 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6836/2571161456,v1:172.21.15.123:6837/2571161456] up:boot 2023-09-23T09:31:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:17 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-23T09:31:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:17 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:31:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:17 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:31:17.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:17 smithi045 ceph-mon[98627]: pgmap v58: 97 pgs: 97 active+clean; 2.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 719 op/s 2023-09-23T09:31:17.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:17 smithi045 ceph-mon[98627]: osdmap e87: 12 total, 12 up, 12 in 2023-09-23T09:31:17.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:17 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6836/2571161456,v1:172.21.15.123:6837/2571161456] up:boot 2023-09-23T09:31:17.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:17 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-23T09:31:17.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:17 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:31:17.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:17 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:31:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:17 smithi123 ceph-mon[103577]: pgmap v58: 97 pgs: 97 active+clean; 2.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 719 op/s 2023-09-23T09:31:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:17 smithi123 ceph-mon[103577]: osdmap e87: 12 total, 12 up, 12 in 2023-09-23T09:31:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:17 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6836/2571161456,v1:172.21.15.123:6837/2571161456] up:boot 2023-09-23T09:31:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:17 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-23T09:31:17.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:17 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:31:17.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:17 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:31:19.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:19 smithi098 ceph-mon[102386]: pgmap v60: 97 pgs: 97 active+clean; 3.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.12k op/s 2023-09-23T09:31:19.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:19 smithi045 ceph-mon[98627]: pgmap v60: 97 pgs: 97 active+clean; 3.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.12k op/s 2023-09-23T09:31:19.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:19 smithi123 ceph-mon[103577]: pgmap v60: 97 pgs: 97 active+clean; 3.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.12k op/s 2023-09-23T09:31:21.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:21 smithi098 ceph-mon[102386]: pgmap v61: 97 pgs: 97 active+clean; 3.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.12k op/s 2023-09-23T09:31:21.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:21 smithi045 ceph-mon[98627]: pgmap v61: 97 pgs: 97 active+clean; 3.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.12k op/s 2023-09-23T09:31:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:21 smithi123 ceph-mon[103577]: pgmap v61: 97 pgs: 97 active+clean; 3.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.12k op/s 2023-09-23T09:31:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 95.0 s, 3549.0 tps, lat 28.098 ms stddev 32.189 2023-09-23T09:31:23.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:23 smithi098 ceph-mon[102386]: pgmap v62: 97 pgs: 97 active+clean; 3.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.15k op/s 2023-09-23T09:31:23.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:23 smithi045 ceph-mon[98627]: pgmap v62: 97 pgs: 97 active+clean; 3.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.15k op/s 2023-09-23T09:31:23.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:23 smithi123 ceph-mon[103577]: pgmap v62: 97 pgs: 97 active+clean; 3.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.15k op/s 2023-09-23T09:31:23.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:31:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.208+0000 7efd22e3c700 1 -- 172.21.15.45:0/1122693879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 msgr2=0x7efd1c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.208+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1122693879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 0x7efd1c0ffef0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009990 tx=0x7efd0c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.213+0000 7efd22e3c700 1 -- 172.21.15.45:0/1122693879 shutdown_connections 2023-09-23T09:31:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.213+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1122693879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1c10d6c0 0x7efd1c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.213+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1122693879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 0x7efd1c0ffef0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.213+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1122693879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 0x7efd1c10d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.213+0000 7efd22e3c700 1 -- 172.21.15.45:0/1122693879 >> 172.21.15.45:0/1122693879 conn(0x7efd1c0fb770 msgr2=0x7efd1c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.213+0000 7efd22e3c700 1 -- 172.21.15.45:0/1122693879 shutdown_connections 2023-09-23T09:31:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.214+0000 7efd22e3c700 1 -- 172.21.15.45:0/1122693879 wait complete. 2023-09-23T09:31:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.214+0000 7efd22e3c700 1 Processor -- start 2023-09-23T09:31:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.215+0000 7efd22e3c700 1 -- start start 2023-09-23T09:31:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.215+0000 7efd22e3c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 0x7efd1c1aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.215+0000 7efd22e3c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 0x7efd1c1aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.215+0000 7efd22e3c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1c10d6c0 0x7efd1c1ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.215+0000 7efd22e3c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c1abe90 con 0x7efd1c1004c0 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.215+0000 7efd22e3c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c1abfd0 con 0x7efd1c0ffb10 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.215+0000 7efd22e3c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c1ac160 con 0x7efd1c10d6c0 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.216+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 0x7efd1c1aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.216+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 0x7efd1c1aaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33636/0 (socket says 172.21.15.45:33636) 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.216+0000 7efd1bfff700 1 -- 172.21.15.45:0/1410845230 learned_addr learned my addr 172.21.15.45:0/1410845230 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.216+0000 7efd20bd8700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 0x7efd1c1aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.217+0000 7efd1bfff700 1 -- 172.21.15.45:0/1410845230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1c10d6c0 msgr2=0x7efd1c1ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.217+0000 7efd1bfff700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1c10d6c0 0x7efd1c1ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.217+0000 7efd1bfff700 1 -- 172.21.15.45:0/1410845230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 msgr2=0x7efd1c1aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.217+0000 7efd1bfff700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 0x7efd1c1aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.217+0000 7efd1bfff700 1 -- 172.21.15.45:0/1410845230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0095f0 con 0x7efd1c1004c0 2023-09-23T09:31:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.217+0000 7efd1bfff700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 0x7efd1c1aaef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd14007f10 tx=0x7efd1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.218+0000 7efd19ffb700 1 -- 172.21.15.45:0/1410845230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400de90 con 0x7efd1c1004c0 2023-09-23T09:31:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.218+0000 7efd19ffb700 1 -- 172.21.15.45:0/1410845230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1400f040 con 0x7efd1c1004c0 2023-09-23T09:31:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.218+0000 7efd19ffb700 1 -- 172.21.15.45:0/1410845230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14015400 con 0x7efd1c1004c0 2023-09-23T09:31:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.218+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c1ac3e0 con 0x7efd1c1004c0 2023-09-23T09:31:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.218+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c06aa20 con 0x7efd1c1004c0 2023-09-23T09:31:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.223+0000 7efd2f026b80 1 Processor -- start 2023-09-23T09:31:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.224+0000 7efd19ffb700 1 -- 172.21.15.45:0/1410845230 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7efd14004000 con 0x7efd1c1004c0 2023-09-23T09:31:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.225+0000 7efd19ffb700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efd040c3870 0x7efd040c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.225+0000 7efd19ffb700 1 -- 172.21.15.45:0/1410845230 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efd140ccbf0 con 0x7efd1c1004c0 2023-09-23T09:31:24.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.225+0000 7efd213d9700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efd040c3870 0x7efd040c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.225+0000 7efd2f026b80 1 -- start start 2023-09-23T09:31:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f39344400 0x561f393447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f3934fbd0 0x561f39351fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd2f026b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f392ad460 con 0x561f3934fbd0 2023-09-23T09:31:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd2f026b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f3929ec20 con 0x561f39344400 2023-09-23T09:31:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd2f026b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f3929c540 con 0x561f39344d20 2023-09-23T09:31:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.226+0000 7efd213d9700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efd040c3870 0x7efd040c5d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd10013fd0 tx=0x7efd100058e0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f39344400 0x561f393447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f39344400 0x561f393447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:54994/0 (socket says 172.21.15.45:54994) 2023-09-23T09:31:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 -- 172.21.15.45:0/599818046 learned_addr learned my addr 172.21.15.45:0/599818046 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd213d9700 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f3934fbd0 0x561f39351fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 -- 172.21.15.45:0/599818046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 msgr2=0x561f3934f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 -- 172.21.15.45:0/599818046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f3934fbd0 msgr2=0x561f39351fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f3934fbd0 0x561f39351fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 -- 172.21.15.45:0/599818046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f392fa8d0 con 0x561f39344400 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd213d9700 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f3934fbd0 0x561f39351fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.227+0000 7efd20bd8700 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f39344400 0x561f393447e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005e80 tx=0x7efd0c005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6450121ff9af22 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd02ffd700 1 -- 172.21.15.45:0/599818046 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00bee0 con 0x561f39344400 2023-09-23T09:31:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd02ffd700 1 -- 172.21.15.45:0/599818046 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd0c01e040 con 0x561f39344400 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd02ffd700 1 -- 172.21.15.45:0/599818046 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c004c50 con 0x561f39344400 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 -- 172.21.15.45:0/599818046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f39344400 msgr2=0x561f393447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f39344400 0x561f393447e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005e80 tx=0x7efd0c005eb0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 -- 172.21.15.45:0/599818046 shutdown_connections 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f3934fbd0 0x561f39351fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f39344400 0x561f393447e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 --2- 172.21.15.45:0/599818046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 -- 172.21.15.45:0/599818046 >> 172.21.15.45:0/599818046 conn(0x561f392b9260 msgr2=0x561f39340790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 -- 172.21.15.45:0/599818046 shutdown_connections 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.228+0000 7efd2f026b80 1 -- 172.21.15.45:0/599818046 wait complete. 2023-09-23T09:31:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.229+0000 7efd2f026b80 1 -- start start 2023-09-23T09:31:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.229+0000 7efd2f026b80 1 Processor -- start 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd2f026b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f39297e30 con 0x561f39344400 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd2f026b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f39234710 con 0x561f3934fbd0 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd2f026b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f392343b0 con 0x561f39344d20 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd2f026b80 10 client.0 _ll_register_callbacks cb 0x561f39091800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd213d9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33650/0 (socket says 172.21.15.45:33650) 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd20bd8700 1 -- 172.21.15.45:0/3520604666 learned_addr learned my addr 172.21.15.45:0/3520604666 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd1bfff700 1 -- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 msgr2=0x561f3934cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd1bfff700 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd1bfff700 1 -- 172.21.15.45:0/3520604666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 msgr2=0x561f3934f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd1bfff700 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd1bfff700 1 -- 172.21.15.45:0/3520604666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c1106e0 con 0x561f39344d20 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd213d9700 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd20bd8700 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efd1bfff700 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934c7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd140ca330 tx=0x7efd1409dca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731a10041bbc6b77 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.230+0000 7efcfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:31:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efcfbfff700 1 -- 172.21.15.45:0/3520604666 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd140b1070 con 0x561f39344d20 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efcfbfff700 1 -- 172.21.15.45:0/3520604666 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efd14099db0 con 0x561f39344d20 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f39338250 con 0x561f39344d20 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efcfbfff700 1 -- 172.21.15.45:0/3520604666 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd140a3020 con 0x561f39344d20 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 10 client.25516 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561f390c2de0 con 0x561f39344d20 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efcfbfff700 1 -- 172.21.15.45:0/3520604666 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f39338250 con 0x561f39344d20 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efcfbfff700 1 -- 172.21.15.45:0/3520604666 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efd140a1ae0 con 0x561f39344d20 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 10 client.25516 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 10 client.25516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 20 client.25516 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f3935f510 0x561f39361930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 4 client.25516 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f39290e10 con 0x561f3935f510 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.231+0000 7efd20bd8700 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f3935f510 0x561f39361930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.232+0000 7efcfbfff700 10 client.25516 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.232+0000 7efd20bd8700 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f3935f510 0x561f39361930 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=338992b34bd6fa8d server_cookie=eb8773226515b34 in_seq=0 out_seq=0 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.232+0000 7efcfbfff700 1 -- 172.21.15.45:0/3520604666 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f39290e10 con 0x561f3935f510 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.232+0000 7efcfbfff700 10 client.25516 handle_command_reply: tid=0 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 client.25516 shutdown 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 msgr2=0x561f3934c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934c7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd140ca330 tx=0x7efd1409dca0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 shutdown_connections 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f3935f510 0x561f39361930 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efcfbfff700 0 client.25516 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3520604666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934c7c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 >> 172.21.15.45:0/3520604666 conn(0x561f392b9260 msgr2=0x561f39340320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.233+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 shutdown_connections 2023-09-23T09:31:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.235+0000 7efd2f026b80 1 -- 172.21.15.45:0/3520604666 wait complete. 2023-09-23T09:31:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.235+0000 7efd2f026b80 20 client.25516 trim_cache size 0 max 16384 2023-09-23T09:31:24.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 Processor -- start 2023-09-23T09:31:24.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 -- start start 2023-09-23T09:31:24.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f39356710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f39118920 con 0x561f39344400 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f3935cbc0 con 0x561f3934fbd0 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd2f026b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f39337d90 con 0x561f39344d20 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f39356710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f39356710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33658/0 (socket says 172.21.15.45:33658) 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd213d9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd20bd8700 1 -- 172.21.15.45:0/4152598078 learned_addr learned my addr 172.21.15.45:0/4152598078 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.267+0000 7efd1bfff700 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd213d9700 1 -- 172.21.15.45:0/4152598078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 msgr2=0x561f3934f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd213d9700 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd213d9700 1 -- 172.21.15.45:0/4152598078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 msgr2=0x561f39356710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd213d9700 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f39356710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd213d9700 1 -- 172.21.15.45:0/4152598078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f390abc60 con 0x561f3934fbd0 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd213d9700 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934bf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd10005950 tx=0x7efd10015070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ab5ee8b51d71e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd037fe700 1 -- 172.21.15.45:0/4152598078 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10004a30 con 0x561f3934fbd0 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd037fe700 1 -- 172.21.15.45:0/4152598078 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd100054e0 con 0x561f3934fbd0 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.268+0000 7efd037fe700 1 -- 172.21.15.45:0/4152598078 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10010a60 con 0x561f3934fbd0 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.269+0000 7efd2f026b80 1 -- 172.21.15.45:0/4152598078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 msgr2=0x561f3934bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.269+0000 7efd2f026b80 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934bf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd10005950 tx=0x7efd10015070 comp rx=0 tx=0).stop 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.269+0000 7efd2f026b80 1 -- 172.21.15.45:0/4152598078 shutdown_connections 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.269+0000 7efd2f026b80 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f3934bf80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.269+0000 7efd2f026b80 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f39356710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.269+0000 7efd2f026b80 1 --2- 172.21.15.45:0/4152598078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f3934f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.269+0000 7efd2f026b80 1 -- 172.21.15.45:0/4152598078 >> 172.21.15.45:0/4152598078 conn(0x561f392b9260 msgr2=0x561f3935b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.270+0000 7efd2f026b80 1 -- 172.21.15.45:0/4152598078 shutdown_connections 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.270+0000 7efd2f026b80 1 -- 172.21.15.45:0/4152598078 wait complete. 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.270+0000 7efd2f026b80 1 -- start start 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.270+0000 7efd2f026b80 1 Processor -- start 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f39348a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd2f026b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f39348f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd2f026b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f393494c0 con 0x561f39344400 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd2f026b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f39349630 con 0x561f3934fbd0 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd2f026b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f393497f0 con 0x561f39344d20 2023-09-23T09:31:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd2f026b80 10 client.0 _ll_register_callbacks cb 0x7efd1409a250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f39348a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f39348a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35770/0 (socket says 172.21.15.45:35770) 2023-09-23T09:31:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd1bfff700 1 -- 172.21.15.45:0/3663471281 learned_addr learned my addr 172.21.15.45:0/3663471281 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd20bd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33664/0 (socket says 172.21.15.45:33664) 2023-09-23T09:31:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd213d9700 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f39348f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd20bd8700 1 -- 172.21.15.45:0/3663471281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 msgr2=0x561f39348a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd20bd8700 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f39348a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd20bd8700 1 -- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 msgr2=0x561f39348f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd20bd8700 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f39348f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.271+0000 7efd20bd8700 1 -- 172.21.15.45:0/3663471281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd10013110 con 0x561f39344400 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.272+0000 7efd20bd8700 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934a160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00bb30 tx=0x7efd0c005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef2d29d93f8d19b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.272+0000 7efcfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.273+0000 7efcfbfff700 1 -- 172.21.15.45:0/3663471281 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c032040 con 0x561f39344400 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.273+0000 7efcfbfff700 1 -- 172.21.15.45:0/3663471281 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efd0c0557b0 con 0x561f39344400 2023-09-23T09:31:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.273+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd0c01e040 con 0x561f39344400 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.273+0000 7efcfbfff700 1 -- 172.21.15.45:0/3663471281 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd0c055990 con 0x561f39344400 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.273+0000 7efd2f026b80 10 client.15852 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.273+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561f390e9250 con 0x561f39344400 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.273+0000 7efcfbfff700 1 -- 172.21.15.45:0/3663471281 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd0c01e040 con 0x561f39344400 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efcfbfff700 1 -- 172.21.15.45:0/3663471281 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efd0c00fb00 con 0x561f39344400 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd2f026b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd2f026b80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd2f026b80 20 client.15852 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f393b4330 0x561f393b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd2f026b80 4 client.15852 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f39290b30 con 0x561f393b4330 2023-09-23T09:31:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd1bfff700 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f393b4330 0x561f393b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efcfbfff700 10 client.15852 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.274+0000 7efd1bfff700 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f393b4330 0x561f393b6750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5ceb85e9640348 server_cookie=5963d82d6e7d27f1 in_seq=0 out_seq=0 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efcfbfff700 1 -- 172.21.15.45:0/3663471281 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561f39290b30 con 0x561f393b4330 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efcfbfff700 10 client.15852 handle_command_reply: tid=0 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 client.15852 shutdown 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 msgr2=0x561f3934a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934a160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00bb30 tx=0x7efd0c005cb0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 shutdown_connections 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561f393b4330 0x561f393b6750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f3934fbd0 0x561f39348f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f39344400 0x561f3934a160 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 --2- 172.21.15.45:0/3663471281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f39344d20 0x561f39348a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 >> 172.21.15.45:0/3663471281 conn(0x561f392b9260 msgr2=0x561f39346440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efcfbfff700 0 client.15852 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 shutdown_connections 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 1 -- 172.21.15.45:0/3663471281 wait complete. 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.275+0000 7efd2f026b80 20 client.15852 trim_cache size 0 max 16384 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:31:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:31:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:31:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:31:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:31:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:31:24.282 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:31:24.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efd040c3870 msgr2=0x7efd040c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efd040c3870 0x7efd040c5d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd10013fd0 tx=0x7efd100058e0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 msgr2=0x7efd1c1aaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 0x7efd1c1aaef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd14007f10 tx=0x7efd1400d3b0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 shutdown_connections 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1c10d6c0 0x7efd1c1ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd1c0ffb10 0x7efd1c1aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efd040c3870 0x7efd040c5d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 --2- 172.21.15.45:0/1410845230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd1c1004c0 0x7efd1c1aaef0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.276+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 >> 172.21.15.45:0/1410845230 conn(0x7efd1c0fb770 msgr2=0x7efd1c10f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.277+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 shutdown_connections 2023-09-23T09:31:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:24.277+0000 7efd22e3c700 1 -- 172.21.15.45:0/1410845230 wait complete. 2023-09-23T09:31:24.298 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:31:25.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:25 smithi098 ceph-mon[102386]: pgmap v63: 97 pgs: 97 active+clean; 3.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2023-09-23T09:31:25.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:25 smithi045 ceph-mon[98627]: pgmap v63: 97 pgs: 97 active+clean; 3.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2023-09-23T09:31:25.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:25 smithi123 ceph-mon[103577]: pgmap v63: 97 pgs: 97 active+clean; 3.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2023-09-23T09:31:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 100.0 s, 3326.4 tps, lat 30.173 ms stddev 55.738 2023-09-23T09:31:27.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:27 smithi098 ceph-mon[102386]: pgmap v64: 97 pgs: 97 active+clean; 3.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2023-09-23T09:31:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:27 smithi045 ceph-mon[98627]: pgmap v64: 97 pgs: 97 active+clean; 3.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2023-09-23T09:31:27.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:27 smithi123 ceph-mon[103577]: pgmap v64: 97 pgs: 97 active+clean; 3.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2023-09-23T09:31:29.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:29 smithi098 ceph-mon[102386]: pgmap v65: 97 pgs: 97 active+clean; 3.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 60 MiB/s wr, 2.62k op/s 2023-09-23T09:31:29.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:29 smithi045 ceph-mon[98627]: pgmap v65: 97 pgs: 97 active+clean; 3.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 60 MiB/s wr, 2.62k op/s 2023-09-23T09:31:29.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:29 smithi123 ceph-mon[103577]: pgmap v65: 97 pgs: 97 active+clean; 3.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 60 MiB/s wr, 2.62k op/s 2023-09-23T09:31:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.418+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3656343255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab34105610 msgr2=0x7fab341059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.418+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3656343255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab34105610 0x7fab341059f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b0a0 tx=0x7fab2c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.418+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3656343255 shutdown_connections 2023-09-23T09:31:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.418+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3656343255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab340ffff0 0x7fab3410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.418+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3656343255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab34105610 0x7fab341059f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.418+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3656343255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab340ff630 0x7fab340ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.418+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3656343255 >> 172.21.15.45:0/3656343255 conn(0x7fab34076140 msgr2=0x7fab34076550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.419+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3656343255 shutdown_connections 2023-09-23T09:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.419+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3656343255 wait complete. 2023-09-23T09:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 Processor -- start 2023-09-23T09:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 -- start start 2023-09-23T09:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab340ff630 0x7fab34106520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab340ffff0 0x7fab34106a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34105610 0x7fab34106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab341a9c00 con 0x7fab340ffff0 2023-09-23T09:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab341a9d70 con 0x7fab34105610 2023-09-23T09:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.420+0000 7fab3a8b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab341a9f30 con 0x7fab340ff630 2023-09-23T09:31:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.421+0000 7fab38e4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34105610 0x7fab34106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.421+0000 7fab33fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab340ff630 0x7fab34106520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.421+0000 7fab33fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab340ff630 0x7fab34106520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35782/0 (socket says 172.21.15.45:35782) 2023-09-23T09:31:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.421+0000 7fab33fff700 1 -- 172.21.15.45:0/3974106755 learned_addr learned my addr 172.21.15.45:0/3974106755 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab337fe700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab340ffff0 0x7fab34106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab33fff700 1 -- 172.21.15.45:0/3974106755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34105610 msgr2=0x7fab34106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab33fff700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34105610 0x7fab34106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab33fff700 1 -- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab340ffff0 msgr2=0x7fab34106a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab33fff700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab340ffff0 0x7fab34106a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab33fff700 1 -- 172.21.15.45:0/3974106755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009d00 con 0x7fab340ff630 2023-09-23T09:31:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab38e4d700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34105610 0x7fab34106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.422+0000 7fab33fff700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab340ff630 0x7fab34106520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b070 tx=0x7fab2c00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.423+0000 7fab317fa700 1 -- 172.21.15.45:0/3974106755 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00e040 con 0x7fab340ff630 2023-09-23T09:31:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.423+0000 7fab317fa700 1 -- 172.21.15.45:0/3974106755 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab2c009480 con 0x7fab340ff630 2023-09-23T09:31:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.423+0000 7fab317fa700 1 -- 172.21.15.45:0/3974106755 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c01b6c0 con 0x7fab340ff630 2023-09-23T09:31:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.423+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab341aa1b0 con 0x7fab340ff630 2023-09-23T09:31:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.423+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab341aa5d0 con 0x7fab340ff630 2023-09-23T09:31:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.425+0000 7fab317fa700 1 -- 172.21.15.45:0/3974106755 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fab2c01d470 con 0x7fab340ff630 2023-09-23T09:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.426+0000 7fab317fa700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fab1c0a1790 0x7fab1c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.426+0000 7fab317fa700 1 -- 172.21.15.45:0/3974106755 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fab2c0ce370 con 0x7fab340ff630 2023-09-23T09:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab337fe700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fab1c0a1790 0x7fab1c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 Processor -- start 2023-09-23T09:31:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 -- start start 2023-09-23T09:31:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab337fe700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fab1c0a1790 0x7fab1c0a3c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab34107f20 tx=0x7fab28009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0cd860 0x56086e0d81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e036400 con 0x56086e0cd860 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e027bc0 con 0x56086e0ccf40 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e0254e0 con 0x56086e0d8710 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0cd860 0x56086e0d81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab33fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0cd860 0x56086e0d81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33686/0 (socket says 172.21.15.45:33686) 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab33fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55034/0 (socket says 172.21.15.45:55034) 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab38e4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.428+0000 7fab337fe700 1 -- 172.21.15.45:0/2319113166 learned_addr learned my addr 172.21.15.45:0/2319113166 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab38e4d700 1 -- 172.21.15.45:0/2319113166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 msgr2=0x56086e0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab38e4d700 1 --2- 172.21.15.45:0/2319113166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab38e4d700 1 -- 172.21.15.45:0/2319113166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0cd860 msgr2=0x56086e0d81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab38e4d700 1 --2- 172.21.15.45:0/2319113166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0cd860 0x56086e0d81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab38e4d700 1 -- 172.21.15.45:0/2319113166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56086e083870 con 0x56086e0d8710 2023-09-23T09:31:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab38e4d700 1 --2- 172.21.15.45:0/2319113166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0dab00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab24009fc0 tx=0x7fab24011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f829343751a54f3b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab1a7fc700 1 -- 172.21.15.45:0/2319113166 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24004310 con 0x56086e0d8710 2023-09-23T09:31:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.429+0000 7fab1a7fc700 1 -- 172.21.15.45:0/2319113166 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab24004470 con 0x56086e0d8710 2023-09-23T09:31:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.430+0000 7fab1a7fc700 1 -- 172.21.15.45:0/2319113166 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240199d0 con 0x56086e0d8710 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.430+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2319113166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 msgr2=0x56086e0dab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.430+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2319113166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0dab00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab24009fc0 tx=0x7fab24011720 comp rx=0 tx=0).stop 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.431+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2319113166 shutdown_connections 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.431+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2319113166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0dab00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.431+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2319113166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.431+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2319113166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0cd860 0x56086e0d81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.431+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2319113166 >> 172.21.15.45:0/2319113166 conn(0x56086e042200 msgr2=0x56086e0c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.431+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2319113166 shutdown_connections 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.432+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2319113166 wait complete. 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.432+0000 7fab46a9ab80 1 -- start start 2023-09-23T09:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.432+0000 7fab46a9ab80 1 Processor -- start 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0ccf40 0x56086e0d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e020dd0 con 0x56086e0d8710 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086dfbd6b0 con 0x56086e0cd860 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086dfbd350 con 0x56086e0ccf40 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab46a9ab80 10 client.0 _ll_register_callbacks cb 0x56086de1a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0ccf40 0x56086e0d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab38e4d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0ccf40 0x56086e0d8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35796/0 (socket says 172.21.15.45:35796) 2023-09-23T09:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 -- 172.21.15.45:0/3201818518 learned_addr learned my addr 172.21.15.45:0/3201818518 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 -- 172.21.15.45:0/3201818518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 msgr2=0x56086e0d53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 -- 172.21.15.45:0/3201818518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 msgr2=0x56086e0d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 -- 172.21.15.45:0/3201818518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab24009c70 con 0x56086e0ccf40 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab38e4d700 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab33fff700 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0ccf40 0x56086e0d8140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab2c09b2f0 tx=0x7fab2c09af20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4403e930a97685e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab137fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.433+0000 7fab137fe700 1 -- 172.21.15.45:0/3201818518 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c09b7c0 con 0x56086e0ccf40 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.434+0000 7fab137fe700 1 -- 172.21.15.45:0/3201818518 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fab2c09ba60 con 0x56086e0ccf40 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.434+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56086e0c1090 con 0x56086e0ccf40 2023-09-23T09:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.434+0000 7fab137fe700 1 -- 172.21.15.45:0/3201818518 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab2c09f2e0 con 0x56086e0ccf40 2023-09-23T09:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.434+0000 7fab46a9ab80 10 client.25534 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.434+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56086de4bd80 con 0x56086e0ccf40 2023-09-23T09:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.434+0000 7fab137fe700 1 -- 172.21.15.45:0/3201818518 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56086e0c1090 con 0x56086e0ccf40 2023-09-23T09:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.435+0000 7fab137fe700 1 -- 172.21.15.45:0/3201818518 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fab2c0a31d0 con 0x56086e0ccf40 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.435+0000 7fab46a9ab80 10 client.25534 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.435+0000 7fab46a9ab80 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.435+0000 7fab46a9ab80 20 client.25534 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.435+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e0f9040 0x56086e0fb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.435+0000 7fab46a9ab80 4 client.25534 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.435+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56086e019db0 con 0x56086e0f9040 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.436+0000 7fab337fe700 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e0f9040 0x56086e0fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.436+0000 7fab137fe700 10 client.25534 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.436+0000 7fab337fe700 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e0f9040 0x56086e0fb460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b01db1d41aad2fa server_cookie=3861d1dadaaa7770 in_seq=0 out_seq=0 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.437+0000 7fab137fe700 1 -- 172.21.15.45:0/3201818518 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56086e019db0 con 0x56086e0f9040 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.437+0000 7fab137fe700 10 client.25534 handle_command_reply: tid=0 2023-09-23T09:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.437+0000 7fab46a9ab80 1 client.25534 shutdown 2023-09-23T09:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.437+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0ccf40 msgr2=0x56086e0d8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.437+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0ccf40 0x56086e0d8140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab2c09b2f0 tx=0x7fab2c09af20 comp rx=0 tx=0).stop 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 shutdown_connections 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e0f9040 0x56086e0fb460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0ccf40 0x56086e0d8140 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab137fe700 0 client.25534 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/3201818518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 >> 172.21.15.45:0/3201818518 conn(0x56086e042200 msgr2=0x56086e0c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 shutdown_connections 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 1 -- 172.21.15.45:0/3201818518 wait complete. 2023-09-23T09:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.439+0000 7fab46a9ab80 20 client.25534 trim_cache size 0 max 16384 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 Processor -- start 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 -- start start 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0ccf40 0x56086e0d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086dea18c0 con 0x56086e0ccf40 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e0c3540 con 0x56086e0cd860 2023-09-23T09:31:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e0d69a0 con 0x56086e0d8710 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55048/0 (socket says 172.21.15.45:55048) 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab337fe700 1 -- 172.21.15.45:0/842249857 learned_addr learned my addr 172.21.15.45:0/842249857 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.464+0000 7fab38e4d700 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab337fe700 1 -- 172.21.15.45:0/842249857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 msgr2=0x56086e0d7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab337fe700 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0d7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab337fe700 1 -- 172.21.15.45:0/842249857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0ccf40 msgr2=0x56086e0d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab337fe700 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0ccf40 0x56086e0d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab337fe700 1 -- 172.21.15.45:0/842249857 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56086de34c00 con 0x56086e0cd860 2023-09-23T09:31:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab337fe700 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d4b80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab280058e0 tx=0x7fab28014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b64b2dd92a9e3f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab1affd700 1 -- 172.21.15.45:0/842249857 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28025070 con 0x56086e0cd860 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab1affd700 1 -- 172.21.15.45:0/842249857 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab28016070 con 0x56086e0cd860 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.465+0000 7fab1affd700 1 -- 172.21.15.45:0/842249857 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab280257a0 con 0x56086e0cd860 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.466+0000 7fab46a9ab80 1 -- 172.21.15.45:0/842249857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 msgr2=0x56086e0d4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.466+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d4b80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab280058e0 tx=0x7fab28014000 comp rx=0 tx=0).stop 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.467+0000 7fab46a9ab80 1 -- 172.21.15.45:0/842249857 shutdown_connections 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.467+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0d8710 0x56086e0d7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.467+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0ccf40 0x56086e0d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.467+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/842249857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0cd860 0x56086e0d4b80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.467+0000 7fab46a9ab80 1 -- 172.21.15.45:0/842249857 >> 172.21.15.45:0/842249857 conn(0x56086e042200 msgr2=0x56086e0f54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.467+0000 7fab46a9ab80 1 -- 172.21.15.45:0/842249857 shutdown_connections 2023-09-23T09:31:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.467+0000 7fab46a9ab80 1 -- 172.21.15.45:0/842249857 wait complete. 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 -- start start 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 Processor -- start 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0cd860 0x56086e0edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0ee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e14bfa0 con 0x56086e0d8710 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e14c110 con 0x56086e0ccf40 2023-09-23T09:31:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e14c2d0 con 0x56086e0cd860 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab46a9ab80 10 client.0 _ll_register_callbacks cb 0x7fab2c097770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab33fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0cd860 0x56086e0edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.468+0000 7fab38e4d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0cd860 0x56086e0edfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35810/0 (socket says 172.21.15.45:35810) 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab33fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0eda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55052/0 (socket says 172.21.15.45:55052) 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 -- 172.21.15.45:0/2219582683 learned_addr learned my addr 172.21.15.45:0/2219582683 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 -- 172.21.15.45:0/2219582683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 msgr2=0x56086e0eda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0eda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 -- 172.21.15.45:0/2219582683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 msgr2=0x56086e0ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0ee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 -- 172.21.15.45:0/2219582683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab28009920 con 0x56086e0cd860 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab337fe700 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0cd860 0x56086e0edfd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fab28009ab0 tx=0x7fab28004380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98806d1eccf8cfef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab137fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab137fe700 1 -- 172.21.15.45:0/2219582683 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28012ca0 con 0x56086e0cd860 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab137fe700 1 -- 172.21.15.45:0/2219582683 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fab28018070 con 0x56086e0cd860 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab24004470 con 0x56086e0cd860 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.469+0000 7fab137fe700 1 -- 172.21.15.45:0/2219582683 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab28012e00 con 0x56086e0cd860 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 10 client.25540 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56086de721f0 con 0x56086e0cd860 2023-09-23T09:31:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab137fe700 1 -- 172.21.15.45:0/2219582683 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab2803c020 con 0x56086e0cd860 2023-09-23T09:31:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab137fe700 1 -- 172.21.15.45:0/2219582683 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fab28004ae0 con 0x56086e0cd860 2023-09-23T09:31:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 10 client.25540 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 10 client.25540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 20 client.25540 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e14da90 0x56086e14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 4 client.25540 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:31:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.470+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56086e019ad0 con 0x56086e14da90 2023-09-23T09:31:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.472+0000 7fab33fff700 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e14da90 0x56086e14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.472+0000 7fab33fff700 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e14da90 0x56086e14feb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6afe68ed21197 server_cookie=f2c8f8d70571be52 in_seq=0 out_seq=0 2023-09-23T09:31:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.473+0000 7fab137fe700 10 client.25540 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab137fe700 1 -- 172.21.15.45:0/2219582683 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56086e019ad0 con 0x56086e14da90 2023-09-23T09:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab137fe700 10 client.25540 handle_command_reply: tid=0 2023-09-23T09:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 client.25540 shutdown 2023-09-23T09:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0cd860 msgr2=0x56086e0edfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0cd860 0x56086e0edfd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fab28009ab0 tx=0x7fab28004380 comp rx=0 tx=0).stop 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 shutdown_connections 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56086e14da90 0x56086e14feb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56086e0d8710 0x56086e0ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56086e0ccf40 0x56086e0eda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab137fe700 0 client.25540 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 --2- 172.21.15.45:0/2219582683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56086e0cd860 0x56086e0edfd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 >> 172.21.15.45:0/2219582683 conn(0x56086e042200 msgr2=0x56086e0d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 shutdown_connections 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 1 -- 172.21.15.45:0/2219582683 wait complete. 2023-09-23T09:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.506+0000 7fab46a9ab80 20 client.25540 trim_cache size 0 max 16384 2023-09-23T09:31:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:31:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:31:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:31:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.508+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fab1c0a1790 msgr2=0x7fab1c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.508+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fab1c0a1790 0x7fab1c0a3c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab34107f20 tx=0x7fab28009260 comp rx=0 tx=0).stop 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.508+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab340ff630 msgr2=0x7fab34106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.508+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab340ff630 0x7fab34106520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b070 tx=0x7fab2c00bce0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.508+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 shutdown_connections 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.509+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34105610 0x7fab34106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.509+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab340ff630 0x7fab34106520 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.509+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fab1c0a1790 0x7fab1c0a3c50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.509+0000 7fab3a8b0700 1 --2- 172.21.15.45:0/3974106755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab340ffff0 0x7fab34106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.509+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 >> 172.21.15.45:0/3974106755 conn(0x7fab34076140 msgr2=0x7fab340fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.509+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 shutdown_connections 2023-09-23T09:31:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:30.509+0000 7fab3a8b0700 1 -- 172.21.15.45:0/3974106755 wait complete. 2023-09-23T09:31:31.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:31 smithi098 ceph-mon[102386]: pgmap v66: 97 pgs: 97 active+clean; 3.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 44 MiB/s wr, 1.90k op/s 2023-09-23T09:31:31.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:31 smithi045 ceph-mon[98627]: pgmap v66: 97 pgs: 97 active+clean; 3.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 44 MiB/s wr, 1.90k op/s 2023-09-23T09:31:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:31 smithi123 ceph-mon[103577]: pgmap v66: 97 pgs: 97 active+clean; 3.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 44 MiB/s wr, 1.90k op/s 2023-09-23T09:31:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 105.0 s, 3943.0 tps, lat 25.394 ms stddev 25.450 2023-09-23T09:31:33.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:33 smithi098 ceph-mon[102386]: pgmap v67: 97 pgs: 97 active+clean; 3.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 49 MiB/s wr, 2.10k op/s 2023-09-23T09:31:33.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:33 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:31:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:33 smithi045 ceph-mon[98627]: pgmap v67: 97 pgs: 97 active+clean; 3.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 49 MiB/s wr, 2.10k op/s 2023-09-23T09:31:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:33 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:31:33.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:33 smithi123 ceph-mon[103577]: pgmap v67: 97 pgs: 97 active+clean; 3.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 49 MiB/s wr, 2.10k op/s 2023-09-23T09:31:33.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:33 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:31:34.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:31:34.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:31:34.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:31:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:31:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:31:34.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:31:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:31:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:31:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:31:35.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:35 smithi098 ceph-mon[102386]: pgmap v68: 97 pgs: 97 active+clean; 3.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 49 MiB/s wr, 2.11k op/s 2023-09-23T09:31:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:35 smithi045 ceph-mon[98627]: pgmap v68: 97 pgs: 97 active+clean; 3.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 49 MiB/s wr, 2.11k op/s 2023-09-23T09:31:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:35 smithi123 ceph-mon[103577]: pgmap v68: 97 pgs: 97 active+clean; 3.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 49 MiB/s wr, 2.11k op/s 2023-09-23T09:31:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 110.0 s, 3967.0 tps, lat 25.221 ms stddev 26.052 2023-09-23T09:31:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:37 smithi045 ceph-mon[98627]: pgmap v69: 97 pgs: 97 active+clean; 3.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 39 MiB/s wr, 1.76k op/s 2023-09-23T09:31:37.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:37 smithi123 ceph-mon[103577]: pgmap v69: 97 pgs: 97 active+clean; 3.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 39 MiB/s wr, 1.76k op/s 2023-09-23T09:31:38.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:37 smithi098 ceph-mon[102386]: pgmap v69: 97 pgs: 97 active+clean; 3.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 39 MiB/s wr, 1.76k op/s 2023-09-23T09:31:39.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:38 smithi098 ceph-mon[102386]: pgmap v70: 97 pgs: 97 active+clean; 3.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 45 MiB/s wr, 2.10k op/s 2023-09-23T09:31:39.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:38 smithi045 ceph-mon[98627]: pgmap v70: 97 pgs: 97 active+clean; 3.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 45 MiB/s wr, 2.10k op/s 2023-09-23T09:31:39.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:38 smithi123 ceph-mon[103577]: pgmap v70: 97 pgs: 97 active+clean; 3.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 45 MiB/s wr, 2.10k op/s 2023-09-23T09:31:41.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:41 smithi123 ceph-mon[103577]: pgmap v71: 97 pgs: 97 active+clean; 3.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 687 op/s 2023-09-23T09:31:41.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:41 smithi098 ceph-mon[102386]: pgmap v71: 97 pgs: 97 active+clean; 3.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 687 op/s 2023-09-23T09:31:41.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:41 smithi045 ceph-mon[98627]: pgmap v71: 97 pgs: 97 active+clean; 3.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 687 op/s 2023-09-23T09:31:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 115.0 s, 3536.0 tps, lat 28.173 ms stddev 30.597 2023-09-23T09:31:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.673+0000 7f86c44ac700 1 -- 172.21.15.45:0/4257903652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc100b10 msgr2=0x7f86bc104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.673+0000 7f86c44ac700 1 --2- 172.21.15.45:0/4257903652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc100b10 0x7f86bc104920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009990 tx=0x7f86ac009ca0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.674+0000 7f86c44ac700 1 -- 172.21.15.45:0/4257903652 shutdown_connections 2023-09-23T09:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.674+0000 7f86c44ac700 1 --2- 172.21.15.45:0/4257903652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 0x7f86bc10cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.674+0000 7f86c44ac700 1 --2- 172.21.15.45:0/4257903652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86bc100160 0x7f86bc100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.674+0000 7f86c44ac700 1 --2- 172.21.15.45:0/4257903652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc100b10 0x7f86bc104920 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.674+0000 7f86c44ac700 1 -- 172.21.15.45:0/4257903652 >> 172.21.15.45:0/4257903652 conn(0x7f86bc076840 msgr2=0x7f86bc076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.674+0000 7f86c44ac700 1 -- 172.21.15.45:0/4257903652 shutdown_connections 2023-09-23T09:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.674+0000 7f86c44ac700 1 -- 172.21.15.45:0/4257903652 wait complete. 2023-09-23T09:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.675+0000 7f86c44ac700 1 Processor -- start 2023-09-23T09:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.675+0000 7f86c44ac700 1 -- start start 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.675+0000 7f86c44ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86bc100160 0x7f86bc06a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.675+0000 7f86c44ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 0x7f86bc06ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.675+0000 7f86c44ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc06b3b0 0x7f86bc1abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c44ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc072ea0 con 0x7f86bc105050 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c44ac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1ac320 con 0x7f86bc06b3b0 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c44ac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1ac4b0 con 0x7f86bc100160 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 0x7f86bc06ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c2248700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86bc100160 0x7f86bc06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c2a49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc06b3b0 0x7f86bc1abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 0x7f86bc06ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33718/0 (socket says 172.21.15.45:33718) 2023-09-23T09:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c2248700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86bc100160 0x7f86bc06a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35822/0 (socket says 172.21.15.45:35822) 2023-09-23T09:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 -- 172.21.15.45:0/525971668 learned_addr learned my addr 172.21.15.45:0/525971668 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 -- 172.21.15.45:0/525971668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86bc100160 msgr2=0x7f86bc06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86bc100160 0x7f86bc06a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 -- 172.21.15.45:0/525971668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc06b3b0 msgr2=0x7f86bc1abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc06b3b0 0x7f86bc1abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c1a47700 1 -- 172.21.15.45:0/525971668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac0095f0 con 0x7f86bc105050 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.676+0000 7f86c2a49700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc06b3b0 0x7f86bc1abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.677+0000 7f86c1a47700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 0x7f86bc06ae70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0098e0 tx=0x7f86ac003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.677+0000 7f86b37fe700 1 -- 172.21.15.45:0/525971668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac01d070 con 0x7f86bc105050 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.677+0000 7f86b37fe700 1 -- 172.21.15.45:0/525971668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86ac005580 con 0x7f86bc105050 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.677+0000 7f86b37fe700 1 -- 172.21.15.45:0/525971668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac022420 con 0x7f86bc105050 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.677+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc1ac730 con 0x7f86bc105050 2023-09-23T09:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.677+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc1acb00 con 0x7f86bc105050 2023-09-23T09:31:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.683+0000 7f86d0696b80 1 Processor -- start 2023-09-23T09:31:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.684+0000 7f86b37fe700 1 -- 172.21.15.45:0/525971668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f86ac005050 con 0x7f86bc105050 2023-09-23T09:31:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.684+0000 7f86d0696b80 1 -- start start 2023-09-23T09:31:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86d0696b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c831d400 con 0x55b9c83b4f50 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86d0696b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c830ebc0 con 0x55b9c83c37a0 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86d0696b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c830c4e0 con 0x55b9c83b5910 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86c1a47700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86b37fe700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86a80a1790 0x7f86a80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86c1a47700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35828/0 (socket says 172.21.15.45:35828) 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86c1a47700 1 -- 172.21.15.45:0/941648796 learned_addr learned my addr 172.21.15.45:0/941648796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86c2a49700 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86c2248700 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86b37fe700 1 -- 172.21.15.45:0/525971668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86ac0ce270 con 0x7f86bc105050 2023-09-23T09:31:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86c2a49700 1 -- 172.21.15.45:0/941648796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 msgr2=0x55b9c83ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.685+0000 7f86c2a49700 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86c2a49700 1 -- 172.21.15.45:0/941648796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 msgr2=0x55b9c83b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86c2a49700 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86c2a49700 1 -- 172.21.15.45:0/941648796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9c836a870 con 0x55b9c83c37a0 2023-09-23T09:31:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86c2248700 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:31:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86c2a49700 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83c3b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86b400a7f0 tx=0x7f86b400ab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f75ec196fe8b463 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86b0d86700 1 -- 172.21.15.45:0/941648796 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4016830 con 0x55b9c83c37a0 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86b0d86700 1 -- 172.21.15.45:0/941648796 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b4016e50 con 0x55b9c83c37a0 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.686+0000 7f86b0d86700 1 -- 172.21.15.45:0/941648796 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40142e0 con 0x55b9c83c37a0 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.687+0000 7f86d0696b80 1 -- 172.21.15.45:0/941648796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 msgr2=0x55b9c83c3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.687+0000 7f86d0696b80 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83c3b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86b400a7f0 tx=0x7f86b400ab00 comp rx=0 tx=0).stop 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.687+0000 7f86d0696b80 1 -- 172.21.15.45:0/941648796 shutdown_connections 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.687+0000 7f86d0696b80 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.687+0000 7f86d0696b80 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83c3b80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.687+0000 7f86d0696b80 1 --2- 172.21.15.45:0/941648796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.687+0000 7f86d0696b80 1 -- 172.21.15.45:0/941648796 >> 172.21.15.45:0/941648796 conn(0x55b9c8329200 msgr2=0x55b9c83b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.690+0000 7f86d0696b80 1 -- 172.21.15.45:0/941648796 shutdown_connections 2023-09-23T09:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.690+0000 7f86d0696b80 1 -- 172.21.15.45:0/941648796 wait complete. 2023-09-23T09:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.691+0000 7f86d0696b80 1 -- start start 2023-09-23T09:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.691+0000 7f86d0696b80 1 Processor -- start 2023-09-23T09:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.691+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b4f50 0x55b9c83ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83c37a0 0x55b9c83c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86d0696b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c8307dd0 con 0x55b9c83c37a0 2023-09-23T09:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86d0696b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c82a46b0 con 0x55b9c83b5910 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86d0696b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c82a4350 con 0x55b9c83b4f50 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86d0696b80 10 client.0 _ll_register_callbacks cb 0x55b9c81017a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86c2a49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83c37a0 0x55b9c83c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86c2a49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83c37a0 0x55b9c83c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33730/0 (socket says 172.21.15.45:33730) 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.692+0000 7f86c2a49700 1 -- 172.21.15.45:0/778491142 learned_addr learned my addr 172.21.15.45:0/778491142 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c2248700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b4f50 0x55b9c83ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c2a49700 1 -- 172.21.15.45:0/778491142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b4f50 msgr2=0x55b9c83ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c2a49700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b4f50 0x55b9c83ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c1a47700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c2a49700 1 -- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 msgr2=0x55b9c83c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c2a49700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c2a49700 1 -- 172.21.15.45:0/778491142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b400a420 con 0x55b9c83c37a0 2023-09-23T09:31:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c1a47700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:31:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86c2a49700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83c37a0 0x55b9c83c7c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86b40098d0 tx=0x7f86b40156f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb654c16013a0dc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.693+0000 7f86a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.694+0000 7f86a5ffb700 1 -- 172.21.15.45:0/778491142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4016630 con 0x55b9c83c37a0 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.694+0000 7f86a5ffb700 1 -- 172.21.15.45:0/778491142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86b4006e40 con 0x55b9c83c37a0 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.694+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9c83a8090 con 0x55b9c83c37a0 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.694+0000 7f86c2a49700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86a80a1790 0x7f86a80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.695+0000 7f86a5ffb700 1 -- 172.21.15.45:0/778491142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86b4016bd0 con 0x55b9c83c37a0 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.695+0000 7f86d0696b80 10 client.15876 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.695+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b9c8132d80 con 0x55b9c83c37a0 2023-09-23T09:31:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.695+0000 7f86c2a49700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86a80a1790 0x7f86a80a3c50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f86b4004b10 tx=0x7f86b4032000 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86a5ffb700 1 -- 172.21.15.45:0/778491142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9c83a8090 con 0x55b9c83c37a0 2023-09-23T09:31:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86a5ffb700 1 -- 172.21.15.45:0/778491142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f86b402a4e0 con 0x55b9c83c37a0 2023-09-23T09:31:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86d0696b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86d0696b80 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86d0696b80 20 client.15876 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86d0696b80 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c83ba130 0x55b9c83bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86d0696b80 4 client.15876 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:31:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9c8300db0 con 0x55b9c83ba130 2023-09-23T09:31:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.697+0000 7f86c2248700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c83ba130 0x55b9c83bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.698+0000 7f86a5ffb700 10 client.15876 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.698+0000 7f86c2248700 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c83ba130 0x55b9c83bc550 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1687620dca81dc5 server_cookie=20a1b6bbaf416978 in_seq=0 out_seq=0 2023-09-23T09:31:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.698+0000 7f86a5ffb700 1 -- 172.21.15.45:0/778491142 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b9c8300db0 con 0x55b9c83ba130 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.698+0000 7f86a5ffb700 10 client.15876 handle_command_reply: tid=0 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.698+0000 7f86d0696b80 1 client.15876 shutdown 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.698+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83c37a0 msgr2=0x55b9c83c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.698+0000 7f86d0696b80 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83c37a0 0x55b9c83c7c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86b40098d0 tx=0x7f86b40156f0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 shutdown_connections 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c83ba130 0x55b9c83bc550 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83c37a0 0x55b9c83c7c10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b4f50 0x55b9c83ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86a5ffb700 0 client.15876 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 --2- 172.21.15.45:0/778491142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 >> 172.21.15.45:0/778491142 conn(0x55b9c8329200 msgr2=0x55b9c83afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 shutdown_connections 2023-09-23T09:31:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 1 -- 172.21.15.45:0/778491142 wait complete. 2023-09-23T09:31:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.699+0000 7f86d0696b80 20 client.15876 trim_cache size 0 max 16384 2023-09-23T09:31:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.722+0000 7f86d0696b80 1 Processor -- start 2023-09-23T09:31:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.722+0000 7f86d0696b80 1 -- start start 2023-09-23T09:31:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.722+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.722+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.722+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.722+0000 7f86d0696b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c81888c0 con 0x55b9c83b4f50 2023-09-23T09:31:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.722+0000 7f86d0696b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c837ab90 con 0x55b9c83c37a0 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86d0696b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c83b18b0 con 0x55b9c83b5910 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2248700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2a49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2248700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33738/0 (socket says 172.21.15.45:33738) 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2248700 1 -- 172.21.15.45:0/1858069038 learned_addr learned my addr 172.21.15.45:0/1858069038 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c1a47700 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2a49700 1 -- 172.21.15.45:0/1858069038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 msgr2=0x55b9c83c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2a49700 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2a49700 1 -- 172.21.15.45:0/1858069038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 msgr2=0x55b9c83ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2a49700 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2a49700 1 -- 172.21.15.45:0/1858069038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9c811bc00 con 0x55b9c83c37a0 2023-09-23T09:31:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2248700 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.723+0000 7f86c2a49700 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83afe40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86b402a960 tx=0x7f86b40021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca7ccc3064c5bc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.724+0000 7f86b1587700 1 -- 172.21.15.45:0/1858069038 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4016bd0 con 0x55b9c83c37a0 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.724+0000 7f86b1587700 1 -- 172.21.15.45:0/1858069038 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b4016630 con 0x55b9c83c37a0 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.724+0000 7f86b1587700 1 -- 172.21.15.45:0/1858069038 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b402a3f0 con 0x55b9c83c37a0 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 -- 172.21.15.45:0/1858069038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 msgr2=0x55b9c83afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83afe40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86b402a960 tx=0x7f86b40021d0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 -- 172.21.15.45:0/1858069038 shutdown_connections 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83c37a0 0x55b9c83afe40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1858069038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83b5910 0x55b9c83c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 -- 172.21.15.45:0/1858069038 >> 172.21.15.45:0/1858069038 conn(0x55b9c8329200 msgr2=0x55b9c83d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 -- 172.21.15.45:0/1858069038 shutdown_connections 2023-09-23T09:31:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 -- 172.21.15.45:0/1858069038 wait complete. 2023-09-23T09:31:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 -- start start 2023-09-23T09:31:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.725+0000 7f86d0696b80 1 Processor -- start 2023-09-23T09:31:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86d0696b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83c37a0 0x55b9c83cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86d0696b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c8432d10 con 0x55b9c83b4f50 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86d0696b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c8432e50 con 0x55b9c83b5910 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86d0696b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9c8432f90 con 0x55b9c83c37a0 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86d0696b80 10 client.0 _ll_register_callbacks cb 0x7f86ac09b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86c1a47700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86c2248700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86c1a47700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55096/0 (socket says 172.21.15.45:55096) 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86c2248700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33744/0 (socket says 172.21.15.45:33744) 2023-09-23T09:31:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.727+0000 7f86c1a47700 1 -- 172.21.15.45:0/1304370398 learned_addr learned my addr 172.21.15.45:0/1304370398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86c2a49700 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83c37a0 0x55b9c83cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86c1a47700 1 -- 172.21.15.45:0/1304370398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83c37a0 msgr2=0x55b9c83cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86c1a47700 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83c37a0 0x55b9c83cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86c1a47700 1 -- 172.21.15.45:0/1304370398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 msgr2=0x55b9c83ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86c1a47700 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86c1a47700 1 -- 172.21.15.45:0/1304370398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b8009c10 con 0x55b9c83b5910 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86c1a47700 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83cd0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0cba60 tx=0x7f86ac0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8101d332399bf253 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86a5ffb700 1 -- 172.21.15.45:0/1304370398 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac09ada0 con 0x55b9c83b5910 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86a5ffb700 1 -- 172.21.15.45:0/1304370398 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86ac09b960 con 0x55b9c83b5910 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.728+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9c8300ad0 con 0x55b9c83b5910 2023-09-23T09:31:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86a5ffb700 1 -- 172.21.15.45:0/1304370398 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86ac09abc0 con 0x55b9c83b5910 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 10 client.25532 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f86b400ae70 con 0x55b9c83b5910 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86a5ffb700 1 -- 172.21.15.45:0/1304370398 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86ac0b7030 con 0x55b9c83b5910 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86a5ffb700 1 -- 172.21.15.45:0/1304370398 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f86ac09a750 con 0x55b9c83b5910 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 10 client.25532 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 10 client.25532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 20 client.25532 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c8434480 0x55b9c84368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 4 client.25532 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9c8153f10 con 0x55b9c8434480 2023-09-23T09:31:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.729+0000 7f86c2248700 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c8434480 0x55b9c84368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.730+0000 7f86a5ffb700 10 client.25532 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.730+0000 7f86c2248700 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c8434480 0x55b9c84368a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e01f1680b1a5d45 server_cookie=f942281267a4dea9 in_seq=0 out_seq=0 2023-09-23T09:31:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.747+0000 7f86a5ffb700 1 -- 172.21.15.45:0/1304370398 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9c8153f10 con 0x55b9c8434480 2023-09-23T09:31:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86a5ffb700 10 client.25532 handle_command_reply: tid=0 2023-09-23T09:31:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 client.25532 shutdown 2023-09-23T09:31:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 msgr2=0x55b9c83cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83cd0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0cba60 tx=0x7f86ac0057a0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 shutdown_connections 2023-09-23T09:31:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b9c8434480 0x55b9c84368a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c83c37a0 0x55b9c83cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9c83b4f50 0x55b9c83ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 --2- 172.21.15.45:0/1304370398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c83b5910 0x55b9c83cd0b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.748+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 >> 172.21.15.45:0/1304370398 conn(0x55b9c8329200 msgr2=0x55b9c83c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.749+0000 7f86a5ffb700 0 client.25532 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.749+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 shutdown_connections 2023-09-23T09:31:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.749+0000 7f86d0696b80 1 -- 172.21.15.45:0/1304370398 wait complete. 2023-09-23T09:31:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.749+0000 7f86d0696b80 20 client.25532 trim_cache size 0 max 16384 2023-09-23T09:31:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:31:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:31:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:31:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86a80a1790 msgr2=0x7f86a80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86a80a1790 0x7f86a80a3c50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f86b4004b10 tx=0x7f86b4032000 comp rx=0 tx=0).stop 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 msgr2=0x7f86bc06ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 0x7f86bc06ae70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0098e0 tx=0x7f86ac003bd0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 shutdown_connections 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86bc06b3b0 0x7f86bc1abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86bc100160 0x7f86bc06a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86a80a1790 0x7f86a80a3c50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 --2- 172.21.15.45:0/525971668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105050 0x7f86bc06ae70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 >> 172.21.15.45:0/525971668 conn(0x7f86bc076840 msgr2=0x7f86bc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 shutdown_connections 2023-09-23T09:31:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:43.750+0000 7f86c44ac700 1 -- 172.21.15.45:0/525971668 wait complete. 2023-09-23T09:31:43.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:43 smithi098 ceph-mon[102386]: pgmap v72: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 813 op/s 2023-09-23T09:31:43.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:43 smithi045 ceph-mon[98627]: pgmap v72: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 813 op/s 2023-09-23T09:31:43.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:43 smithi123 ceph-mon[103577]: pgmap v72: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 813 op/s 2023-09-23T09:31:45.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:45 smithi098 ceph-mon[102386]: pgmap v73: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 892 op/s 2023-09-23T09:31:45.835 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:45 smithi045 ceph-mon[98627]: pgmap v73: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 892 op/s 2023-09-23T09:31:45.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:45 smithi123 ceph-mon[103577]: pgmap v73: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 892 op/s 2023-09-23T09:31:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 120.0 s, 3210.0 tps, lat 31.230 ms stddev 34.942 2023-09-23T09:31:47.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:47 smithi098 ceph-mon[102386]: pgmap v74: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 739 op/s 2023-09-23T09:31:47.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:47 smithi045 ceph-mon[98627]: pgmap v74: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 739 op/s 2023-09-23T09:31:47.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:47 smithi123 ceph-mon[103577]: pgmap v74: 97 pgs: 97 active+clean; 3.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 739 op/s 2023-09-23T09:31:49.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:49 smithi098 ceph-mon[102386]: pgmap v75: 97 pgs: 97 active+clean; 3.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 1.16k op/s 2023-09-23T09:31:49.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:49 smithi045 ceph-mon[98627]: pgmap v75: 97 pgs: 97 active+clean; 3.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 1.16k op/s 2023-09-23T09:31:49.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:49 smithi123 ceph-mon[103577]: pgmap v75: 97 pgs: 97 active+clean; 3.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 1.16k op/s 2023-09-23T09:31:51.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:51 smithi098 ceph-mon[102386]: pgmap v76: 97 pgs: 97 active+clean; 3.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 MiB/s wr, 820 op/s 2023-09-23T09:31:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:51 smithi045 ceph-mon[98627]: pgmap v76: 97 pgs: 97 active+clean; 3.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 MiB/s wr, 820 op/s 2023-09-23T09:31:51.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:51 smithi123 ceph-mon[103577]: pgmap v76: 97 pgs: 97 active+clean; 3.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 MiB/s wr, 820 op/s 2023-09-23T09:31:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 125.0 s, 2178.4 tps, lat 45.696 ms stddev 93.092 2023-09-23T09:31:53.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:53 smithi098 ceph-mon[102386]: pgmap v77: 97 pgs: 97 active+clean; 3.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 959 op/s 2023-09-23T09:31:53.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:53 smithi045 ceph-mon[98627]: pgmap v77: 97 pgs: 97 active+clean; 3.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 959 op/s 2023-09-23T09:31:53.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:53 smithi123 ceph-mon[103577]: pgmap v77: 97 pgs: 97 active+clean; 3.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 959 op/s 2023-09-23T09:31:54.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:31:54.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.523+0000 7f09d1f1c700 1 -- 172.21.15.45:0/3312876209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09cc104ff0 msgr2=0x7f09cc10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.523+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/3312876209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09cc104ff0 0x7f09cc10cb70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b0a0 tx=0x7f09c400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.523+0000 7f09d1f1c700 1 -- 172.21.15.45:0/3312876209 shutdown_connections 2023-09-23T09:31:54.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.523+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/3312876209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09cc104ff0 0x7f09cc10cb70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.523+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/3312876209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc0ffed0 0x7f09cc1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.523+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/3312876209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09cc100880 0x7f09cc1048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.523+0000 7f09d1f1c700 1 -- 172.21.15.45:0/3312876209 >> 172.21.15.45:0/3312876209 conn(0x7f09cc0fb770 msgr2=0x7f09cc0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:54.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.524+0000 7f09d1f1c700 1 -- 172.21.15.45:0/3312876209 shutdown_connections 2023-09-23T09:31:54.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.525+0000 7f09d1f1c700 1 -- 172.21.15.45:0/3312876209 wait complete. 2023-09-23T09:31:54.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.526+0000 7f09d1f1c700 1 Processor -- start 2023-09-23T09:31:54.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.526+0000 7f09d1f1c700 1 -- start start 2023-09-23T09:31:54.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09d1f1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09cc0ffed0 0x7f09cc10c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09d1f1c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc104ff0 0x7f09cc109840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09d1f1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09cc109d80 0x7f09cc10a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09d1f1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1a9c50 con 0x7f09cc109d80 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09d1f1c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1a9d90 con 0x7f09cc104ff0 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09d1f1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1a9ed0 con 0x7f09cc0ffed0 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc104ff0 0x7f09cc109840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc104ff0 0x7f09cc109840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55106/0 (socket says 172.21.15.45:55106) 2023-09-23T09:31:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 -- 172.21.15.45:0/2108929219 learned_addr learned my addr 172.21.15.45:0/2108929219 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 -- 172.21.15.45:0/2108929219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09cc0ffed0 msgr2=0x7f09cc10c830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:31:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09cc0ffed0 0x7f09cc10c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 -- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09cc109d80 msgr2=0x7f09cc10a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09cc109d80 0x7f09cc10a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.527+0000 7f09caffd700 1 -- 172.21.15.45:0/2108929219 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4009d00 con 0x7f09cc104ff0 2023-09-23T09:31:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.528+0000 7f09caffd700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc104ff0 0x7f09cc109840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b550 tx=0x7f09c000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:54.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.528+0000 7f09c8ff9700 1 -- 172.21.15.45:0/2108929219 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0011830 con 0x7f09cc104ff0 2023-09-23T09:31:54.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.528+0000 7f09c8ff9700 1 -- 172.21.15.45:0/2108929219 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c0011e50 con 0x7f09cc104ff0 2023-09-23T09:31:54.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.528+0000 7f09c8ff9700 1 -- 172.21.15.45:0/2108929219 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000f2e0 con 0x7f09cc104ff0 2023-09-23T09:31:54.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.529+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc1aa160 con 0x7f09cc104ff0 2023-09-23T09:31:54.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.529+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc1aa5f0 con 0x7f09cc104ff0 2023-09-23T09:31:54.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.532+0000 7f09c8ff9700 1 -- 172.21.15.45:0/2108929219 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f09c00105a0 con 0x7f09cc104ff0 2023-09-23T09:31:54.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.533+0000 7f09c8ff9700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f09b40a1860 0x7f09b40a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.533+0000 7f09c8ff9700 1 -- 172.21.15.45:0/2108929219 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f09c00ccc80 con 0x7f09cc104ff0 2023-09-23T09:31:54.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.533+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f09b40a1860 0x7f09b40a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.533+0000 7f09de106b80 1 Processor -- start 2023-09-23T09:31:54.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09de106b80 1 -- start start 2023-09-23T09:31:54.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09de106b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09de106b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22807d20 0x559c22812690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09de106b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22812bd0 0x559c22814fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09de106b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22770460 con 0x559c22812bd0 2023-09-23T09:31:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09de106b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22761c20 con 0x559c22807400 2023-09-23T09:31:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09de106b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c2275f540 con 0x559c22807d20 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09cb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09cb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55116/0 (socket says 172.21.15.45:55116) 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09cb7fe700 1 -- 172.21.15.45:0/3300439279 learned_addr learned my addr 172.21.15.45:0/3300439279 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.534+0000 7f09caffd700 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22807d20 0x559c22812690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09cbfff700 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22812bd0 0x559c22814fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f09b40a1860 0x7f09b40a3d20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009af0 tx=0x7f09bc009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09cb7fe700 1 -- 172.21.15.45:0/3300439279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22807d20 msgr2=0x559c22812690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22807d20 0x559c22812690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09cb7fe700 1 -- 172.21.15.45:0/3300439279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22812bd0 msgr2=0x559c22814fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22812bd0 0x559c22814fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09cb7fe700 1 -- 172.21.15.45:0/3300439279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c227bd8d0 con 0x559c22807400 2023-09-23T09:31:54.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.535+0000 7f09caffd700 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22807d20 0x559c22812690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:31:54.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.536+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228077e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00ebb0 tx=0x7f09bc016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fb08f30427a8bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:54.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.536+0000 7f09b1ffb700 1 -- 172.21.15.45:0/3300439279 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc010a40 con 0x559c22807400 2023-09-23T09:31:54.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.536+0000 7f09b1ffb700 1 -- 172.21.15.45:0/3300439279 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09bc030070 con 0x559c22807400 2023-09-23T09:31:54.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.536+0000 7f09b1ffb700 1 -- 172.21.15.45:0/3300439279 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0343d0 con 0x559c22807400 2023-09-23T09:31:54.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 -- 172.21.15.45:0/3300439279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 msgr2=0x559c228077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228077e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00ebb0 tx=0x7f09bc016000 comp rx=0 tx=0).stop 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 -- 172.21.15.45:0/3300439279 shutdown_connections 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22812bd0 0x559c22814fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228077e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 --2- 172.21.15.45:0/3300439279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22807d20 0x559c22812690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 -- 172.21.15.45:0/3300439279 >> 172.21.15.45:0/3300439279 conn(0x559c2277c260 msgr2=0x559c22803790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 -- 172.21.15.45:0/3300439279 shutdown_connections 2023-09-23T09:31:54.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.537+0000 7f09de106b80 1 -- 172.21.15.45:0/3300439279 wait complete. 2023-09-23T09:31:54.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.538+0000 7f09de106b80 1 -- start start 2023-09-23T09:31:54.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.538+0000 7f09de106b80 1 Processor -- start 2023-09-23T09:31:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.544+0000 7f09de106b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c2280f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.544+0000 7f09de106b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 0x559c2280fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.544+0000 7f09de106b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c22810360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.544+0000 7f09de106b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c2275ae30 con 0x559c22807d20 2023-09-23T09:31:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.544+0000 7f09de106b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c226f7710 con 0x559c22807400 2023-09-23T09:31:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.544+0000 7f09de106b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c226f73b0 con 0x559c22812bd0 2023-09-23T09:31:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.544+0000 7f09de106b80 10 client.0 _ll_register_callbacks cb 0x559c22554800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:54.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.545+0000 7f09cbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c22810360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.545+0000 7f09cbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c22810360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35878/0 (socket says 172.21.15.45:35878) 2023-09-23T09:31:54.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.545+0000 7f09cbfff700 1 -- 172.21.15.45:0/126705836 learned_addr learned my addr 172.21.15.45:0/126705836 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:54.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.546+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c2280f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.546+0000 7f09cbfff700 1 -- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 msgr2=0x559c2280f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.546+0000 7f09cbfff700 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c2280f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.546+0000 7f09cbfff700 1 -- 172.21.15.45:0/126705836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 msgr2=0x559c2280fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.546+0000 7f09cbfff700 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 0x559c2280fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.546+0000 7f09cbfff700 1 -- 172.21.15.45:0/126705836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc00e860 con 0x559c22812bd0 2023-09-23T09:31:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.552+0000 7f09cbfff700 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c22810360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005bd0 tx=0x7f09c40077c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f190c3c5e80ab328 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:54.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.555+0000 7f09aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:31:54.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.555+0000 7f09aaffd700 1 -- 172.21.15.45:0/126705836 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c400e040 con 0x559c22812bd0 2023-09-23T09:31:54.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.555+0000 7f09aaffd700 1 -- 172.21.15.45:0/126705836 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f09c4007a10 con 0x559c22812bd0 2023-09-23T09:31:54.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.555+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c227fb250 con 0x559c22812bd0 2023-09-23T09:31:54.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09aaffd700 1 -- 172.21.15.45:0/126705836 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c4009450 con 0x559c22812bd0 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 10 client.25561 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559c22585de0 con 0x559c22812bd0 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09aaffd700 1 -- 172.21.15.45:0/126705836 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c227fb250 con 0x559c22812bd0 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09aaffd700 1 -- 172.21.15.45:0/126705836 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f09c400b0d0 con 0x559c22812bd0 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 10 client.25561 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 20 client.25561 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c228226b0 0x559c22824ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 4 client.25561 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:31:54.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.556+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c22753e10 con 0x559c228226b0 2023-09-23T09:31:54.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.557+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c228226b0 0x559c22824ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.561+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c228226b0 0x559c22824ad0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154edfd585574bb9 server_cookie=218367976dfedc52 in_seq=0 out_seq=0 2023-09-23T09:31:54.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.561+0000 7f09aaffd700 10 client.25561 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:54.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.562+0000 7f09aaffd700 1 -- 172.21.15.45:0/126705836 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c22753e10 con 0x559c228226b0 2023-09-23T09:31:54.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.562+0000 7f09aaffd700 10 client.25561 handle_command_reply: tid=0 2023-09-23T09:31:54.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.562+0000 7f09de106b80 1 client.25561 shutdown 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 msgr2=0x559c22810360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c22810360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005bd0 tx=0x7f09c40077c0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 shutdown_connections 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c228226b0 0x559c22824ad0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c22810360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c2280f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09aaffd700 0 client.25561 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:54.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 --2- 172.21.15.45:0/126705836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 0x559c2280fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 >> 172.21.15.45:0/126705836 conn(0x559c2277c260 msgr2=0x559c22803320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:54.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 shutdown_connections 2023-09-23T09:31:54.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 1 -- 172.21.15.45:0/126705836 wait complete. 2023-09-23T09:31:54.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.563+0000 7f09de106b80 20 client.25561 trim_cache size 0 max 16384 2023-09-23T09:31:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.591+0000 7f09de106b80 1 Processor -- start 2023-09-23T09:31:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09de106b80 1 -- start start 2023-09-23T09:31:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09de106b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807400 0x559c22819710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09de106b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807d20 0x559c2280cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09de106b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c2280d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09de106b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c225db920 con 0x559c22807400 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09de106b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22815460 con 0x559c22807d20 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09de106b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22753e10 con 0x559c22812bd0 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807400 0x559c22819710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807400 0x559c22819710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33778/0 (socket says 172.21.15.45:33778) 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cb7fe700 1 -- 172.21.15.45:0/455055766 learned_addr learned my addr 172.21.15.45:0/455055766 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c2280d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09caffd700 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807d20 0x559c2280cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cbfff700 1 -- 172.21.15.45:0/455055766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807d20 msgr2=0x559c2280cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cbfff700 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807d20 0x559c2280cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cbfff700 1 -- 172.21.15.45:0/455055766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807400 msgr2=0x559c22819710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.592+0000 7f09cbfff700 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807400 0x559c22819710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.593+0000 7f09cbfff700 1 -- 172.21.15.45:0/455055766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c227ff480 con 0x559c22812bd0 2023-09-23T09:31:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.593+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807400 0x559c22819710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:31:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.593+0000 7f09cbfff700 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c2280d0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009a20 tx=0x7f09c40095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8b6ecfa602e56b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.593+0000 7f09b27fc700 1 -- 172.21.15.45:0/455055766 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c400e070 con 0x559c22812bd0 2023-09-23T09:31:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.593+0000 7f09b27fc700 1 -- 172.21.15.45:0/455055766 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c4009450 con 0x559c22812bd0 2023-09-23T09:31:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.593+0000 7f09b27fc700 1 -- 172.21.15.45:0/455055766 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c400b0c0 con 0x559c22812bd0 2023-09-23T09:31:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.595+0000 7f09de106b80 1 -- 172.21.15.45:0/455055766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 msgr2=0x559c2280d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.595+0000 7f09de106b80 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c2280d0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009a20 tx=0x7f09c40095d0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.595+0000 7f09de106b80 1 -- 172.21.15.45:0/455055766 shutdown_connections 2023-09-23T09:31:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.595+0000 7f09de106b80 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c2280d0d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.595+0000 7f09de106b80 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807400 0x559c22819710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.595+0000 7f09de106b80 1 --2- 172.21.15.45:0/455055766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807d20 0x559c2280cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.595+0000 7f09de106b80 1 -- 172.21.15.45:0/455055766 >> 172.21.15.45:0/455055766 conn(0x559c2277c260 msgr2=0x559c2280ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.596+0000 7f09de106b80 1 -- 172.21.15.45:0/455055766 shutdown_connections 2023-09-23T09:31:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.596+0000 7f09de106b80 1 -- 172.21.15.45:0/455055766 wait complete. 2023-09-23T09:31:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.596+0000 7f09de106b80 1 -- start start 2023-09-23T09:31:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.596+0000 7f09de106b80 1 Processor -- start 2023-09-23T09:31:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.596+0000 7f09de106b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.596+0000 7f09de106b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 0x559c22817e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.596+0000 7f09de106b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c228754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09de106b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22875a20 con 0x559c22807d20 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09de106b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22875b60 con 0x559c22807400 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09de106b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22875ca0 con 0x559c22812bd0 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09de106b80 10 client.0 _ll_register_callbacks cb 0x7f09bc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c228754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c228754e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35892/0 (socket says 172.21.15.45:35892) 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09caffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 0x559c22817e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 -- 172.21.15.45:0/1948006122 learned_addr learned my addr 172.21.15.45:0/1948006122 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 -- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 msgr2=0x559c228178f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228178f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 -- 172.21.15.45:0/1948006122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 msgr2=0x559c22817e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 0x559c22817e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.597+0000 7f09cbfff700 1 -- 172.21.15.45:0/1948006122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc039040 con 0x559c22812bd0 2023-09-23T09:31:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.598+0000 7f09cbfff700 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c228754e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f09c401b6d0 tx=0x7f09c4006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a1969890c63b7b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.598+0000 7f09aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:31:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.598+0000 7f09aaffd700 1 -- 172.21.15.45:0/1948006122 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c400e070 con 0x559c22812bd0 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.598+0000 7f09aaffd700 1 -- 172.21.15.45:0/1948006122 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f09c4007980 con 0x559c22812bd0 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.598+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c22753b30 con 0x559c22812bd0 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.598+0000 7f09aaffd700 1 -- 172.21.15.45:0/1948006122 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c400b0c0 con 0x559c22812bd0 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 10 client.25567 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559c22585de0 con 0x559c22812bd0 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09aaffd700 1 -- 172.21.15.45:0/1948006122 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c403b080 con 0x559c22812bd0 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09aaffd700 1 -- 172.21.15.45:0/1948006122 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f09c4025530 con 0x559c22812bd0 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 10 client.25567 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 10 client.25567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 20 client.25567 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c22876f50 0x559c22879370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 4 client.25567 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:31:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c225a6f70 con 0x559c22876f50 2023-09-23T09:31:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.599+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c22876f50 0x559c22879370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.600+0000 7f09aaffd700 10 client.25567 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.600+0000 7f09cb7fe700 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c22876f50 0x559c22879370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe9803d4e225ed0 server_cookie=d8f794a2e8a3cf29 in_seq=0 out_seq=0 2023-09-23T09:31:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09aaffd700 1 -- 172.21.15.45:0/1948006122 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559c225a6f70 con 0x559c22876f50 2023-09-23T09:31:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09aaffd700 10 client.25567 handle_command_reply: tid=0 2023-09-23T09:31:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 client.25567 shutdown 2023-09-23T09:31:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 msgr2=0x559c228754e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c228754e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f09c401b6d0 tx=0x7f09c4006000 comp rx=0 tx=0).stop 2023-09-23T09:31:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 shutdown_connections 2023-09-23T09:31:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c22876f50 0x559c22879370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c22812bd0 0x559c228754e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c22807400 0x559c228178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 --2- 172.21.15.45:0/1948006122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c22807d20 0x559c22817e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 >> 172.21.15.45:0/1948006122 conn(0x559c2277c260 msgr2=0x559c2280e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.601+0000 7f09aaffd700 0 client.25567 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.602+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 shutdown_connections 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.602+0000 7f09de106b80 1 -- 172.21.15.45:0/1948006122 wait complete. 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.602+0000 7f09de106b80 20 client.25567 trim_cache size 0 max 16384 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:31:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.603+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f09b40a1860 msgr2=0x7f09b40a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.603+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f09b40a1860 0x7f09b40a3d20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009af0 tx=0x7f09bc009260 comp rx=0 tx=0).stop 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.603+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc104ff0 msgr2=0x7f09cc109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.603+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc104ff0 0x7f09cc109840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b550 tx=0x7f09c000b910 comp rx=0 tx=0).stop 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 shutdown_connections 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f09b40a1860 0x7f09b40a3d20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09cc109d80 0x7f09cc10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09cc0ffed0 0x7f09cc10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 --2- 172.21.15.45:0/2108929219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09cc104ff0 0x7f09cc109840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 >> 172.21.15.45:0/2108929219 conn(0x7f09cc0fb770 msgr2=0x7f09cc103940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 shutdown_connections 2023-09-23T09:31:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:31:54.604+0000 7f09d1f1c700 1 -- 172.21.15.45:0/2108929219 wait complete. 2023-09-23T09:31:54.625 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:31:55.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:55 smithi098 ceph-mon[102386]: pgmap v78: 97 pgs: 97 active+clean; 3.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 MiB/s wr, 2.29k op/s 2023-09-23T09:31:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:55 smithi045 ceph-mon[98627]: pgmap v78: 97 pgs: 97 active+clean; 3.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 MiB/s wr, 2.29k op/s 2023-09-23T09:31:55.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:55 smithi123 ceph-mon[103577]: pgmap v78: 97 pgs: 97 active+clean; 3.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 MiB/s wr, 2.29k op/s 2023-09-23T09:31:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 130.0 s, 3538.6 tps, lat 28.341 ms stddev 33.503 2023-09-23T09:31:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.920+0000 7f239fe9c700 1 -- 172.21.15.45:0/1394172762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981008b0 msgr2=0x7f23981048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.920+0000 7f239fe9c700 1 --2- 172.21.15.45:0/1394172762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981008b0 0x7f23981048f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2388009990 tx=0x7f2388009ca0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.921+0000 7f239fe9c700 1 -- 172.21.15.45:0/1394172762 shutdown_connections 2023-09-23T09:31:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.921+0000 7f239fe9c700 1 --2- 172.21.15.45:0/1394172762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2398105020 0x7f239810cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.921+0000 7f239fe9c700 1 --2- 172.21.15.45:0/1394172762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23980fff00 0x7f23981002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.921+0000 7f239fe9c700 1 --2- 172.21.15.45:0/1394172762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981008b0 0x7f23981048f0 secure :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2388009990 tx=0x7f2388009ca0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.921+0000 7f239fe9c700 1 -- 172.21.15.45:0/1394172762 >> 172.21.15.45:0/1394172762 conn(0x7f23980fb760 msgr2=0x7f23980fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.923+0000 7f239fe9c700 1 -- 172.21.15.45:0/1394172762 shutdown_connections 2023-09-23T09:31:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.923+0000 7f239fe9c700 1 -- 172.21.15.45:0/1394172762 wait complete. 2023-09-23T09:31:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 Processor -- start 2023-09-23T09:31:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 -- start start 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23980fff00 0x7f2398072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2398105020 0x7f239806b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239806b9a0 0x7f239806be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398072f10 con 0x7f239806b9a0 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398070220 con 0x7f23980fff00 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239fe9c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980703e0 con 0x7f2398105020 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239dc38700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23980fff00 0x7f2398072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239dc38700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23980fff00 0x7f2398072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55148/0 (socket says 172.21.15.45:55148) 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239dc38700 1 -- 172.21.15.45:0/3206015808 learned_addr learned my addr 172.21.15.45:0/3206015808 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.924+0000 7f239d437700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2398105020 0x7f239806b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239e439700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239806b9a0 0x7f239806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239dc38700 1 -- 172.21.15.45:0/3206015808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2398105020 msgr2=0x7f239806b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239dc38700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2398105020 0x7f239806b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239dc38700 1 -- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239806b9a0 msgr2=0x7f239806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239dc38700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239806b9a0 0x7f239806be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239dc38700 1 -- 172.21.15.45:0/3206015808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880095f0 con 0x7f23980fff00 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239dc38700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23980fff00 0x7f2398072660 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f239400b530 tx=0x7f239400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f238effd700 1 -- 172.21.15.45:0/3206015808 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394011830 con 0x7f23980fff00 2023-09-23T09:31:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23980706c0 con 0x7f23980fff00 2023-09-23T09:31:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f238effd700 1 -- 172.21.15.45:0/3206015808 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2394011e50 con 0x7f23980fff00 2023-09-23T09:31:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f238effd700 1 -- 172.21.15.45:0/3206015808 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394010e50 con 0x7f23980fff00 2023-09-23T09:31:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.925+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2398070ba0 con 0x7f23980fff00 2023-09-23T09:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.930+0000 7f23ac086b80 1 Processor -- start 2023-09-23T09:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.931+0000 7f238effd700 1 -- 172.21.15.45:0/3206015808 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f23940104f0 con 0x7f23980fff00 2023-09-23T09:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.932+0000 7f238effd700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f23840a1780 0x7f23840a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.932+0000 7f238effd700 1 -- 172.21.15.45:0/3206015808 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f23940ccc90 con 0x7f23980fff00 2023-09-23T09:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f23ac086b80 1 -- start start 2023-09-23T09:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f239e439700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f23840a1780 0x7f23840a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7bf40 0x5584b3a7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a871d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a87710 0x5584b3a89b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f23ac086b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b39e5400 con 0x5584b3a87710 2023-09-23T09:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f23ac086b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b39d6bc0 con 0x5584b3a7bf40 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f23ac086b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b39d44e0 con 0x5584b3a7c860 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f239dc38700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7bf40 0x5584b3a7c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f239dc38700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7bf40 0x5584b3a7c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55154/0 (socket says 172.21.15.45:55154) 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f239dc38700 1 -- 172.21.15.45:0/158911072 learned_addr learned my addr 172.21.15.45:0/158911072 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.933+0000 7f239d437700 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239e439700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f23840a1780 0x7f23840a3c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f239000f590 tx=0x7f2390009920 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239e439700 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a87710 0x5584b3a89b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239dc38700 1 -- 172.21.15.45:0/158911072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 msgr2=0x5584b3a871d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239dc38700 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a871d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239dc38700 1 -- 172.21.15.45:0/158911072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a87710 msgr2=0x5584b3a89b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239dc38700 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a87710 0x5584b3a89b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239dc38700 1 -- 172.21.15.45:0/158911072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584b3a32870 con 0x5584b3a7bf40 2023-09-23T09:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f239dc38700 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7bf40 0x5584b3a7c320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f23940961a0 tx=0x7f2394004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a980239791eb00 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f2383fff700 1 -- 172.21.15.45:0/158911072 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239409a3a0 con 0x5584b3a7bf40 2023-09-23T09:31:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f2383fff700 1 -- 172.21.15.45:0/158911072 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2394099ca0 con 0x5584b3a7bf40 2023-09-23T09:31:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.934+0000 7f2383fff700 1 -- 172.21.15.45:0/158911072 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239409a140 con 0x5584b3a7bf40 2023-09-23T09:31:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 -- 172.21.15.45:0/158911072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7bf40 msgr2=0x5584b3a7c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7bf40 0x5584b3a7c320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f23940961a0 tx=0x7f2394004860 comp rx=0 tx=0).stop 2023-09-23T09:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 -- 172.21.15.45:0/158911072 shutdown_connections 2023-09-23T09:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a87710 0x5584b3a89b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7bf40 0x5584b3a7c320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 --2- 172.21.15.45:0/158911072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a871d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 -- 172.21.15.45:0/158911072 >> 172.21.15.45:0/158911072 conn(0x5584b39f1200 msgr2=0x5584b3a783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 -- 172.21.15.45:0/158911072 shutdown_connections 2023-09-23T09:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.935+0000 7f23ac086b80 1 -- 172.21.15.45:0/158911072 wait complete. 2023-09-23T09:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.936+0000 7f23ac086b80 1 -- start start 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.936+0000 7f23ac086b80 1 Processor -- start 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.936+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7bf40 0x5584b3a87170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7c860 0x5584b3a84460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3a84ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f23ac086b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b39cfdd0 con 0x5584b3a7c860 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f23ac086b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b396c6b0 con 0x5584b3a87710 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f23ac086b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b396c350 con 0x5584b3a7bf40 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f23ac086b80 10 client.0 _ll_register_callbacks cb 0x5584b37c97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f239e439700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3a84ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f239e439700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3a84ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55160/0 (socket says 172.21.15.45:55160) 2023-09-23T09:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.937+0000 7f239e439700 1 -- 172.21.15.45:0/2583132708 learned_addr learned my addr 172.21.15.45:0/2583132708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.938+0000 7f239e439700 1 -- 172.21.15.45:0/2583132708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7bf40 msgr2=0x5584b3a87170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.938+0000 7f239e439700 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7bf40 0x5584b3a87170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.938+0000 7f239e439700 1 -- 172.21.15.45:0/2583132708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7c860 msgr2=0x5584b3a84460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.938+0000 7f239e439700 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7c860 0x5584b3a84460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.938+0000 7f239e439700 1 -- 172.21.15.45:0/2583132708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2398070ba0 con 0x5584b3a87710 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f239e439700 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3a84ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2390019e30 tx=0x7f2390004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ba2ce794ccc3a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f23817fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f23817fa700 1 -- 172.21.15.45:0/2583132708 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000e040 con 0x5584b3a87710 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f23817fa700 1 -- 172.21.15.45:0/2583132708 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2390017c50 con 0x5584b3a87710 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584b3a70090 con 0x5584b3a87710 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f23817fa700 1 -- 172.21.15.45:0/2583132708 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2390017e10 con 0x5584b3a87710 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f23ac086b80 10 client.25562 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.939+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5584b37fad80 con 0x5584b3a87710 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.940+0000 7f23817fa700 1 -- 172.21.15.45:0/2583132708 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584b3a70090 con 0x5584b3a87710 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23817fa700 1 -- 172.21.15.45:0/2583132708 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f239002b450 con 0x5584b3a87710 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23ac086b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23ac086b80 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23ac086b80 20 client.25562 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3ab9330 0x5584b3abb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23ac086b80 4 client.25562 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584b39c8db0 con 0x5584b3ab9330 2023-09-23T09:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f239dc38700 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3ab9330 0x5584b3abb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f23817fa700 10 client.25562 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.941+0000 7f239dc38700 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3ab9330 0x5584b3abb750 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a64086df13c6f2c server_cookie=30bd621dfff46bf7 in_seq=0 out_seq=0 2023-09-23T09:31:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23817fa700 1 -- 172.21.15.45:0/2583132708 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5584b39c8db0 con 0x5584b3ab9330 2023-09-23T09:31:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23817fa700 10 client.25562 handle_command_reply: tid=0 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 client.25562 shutdown 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 msgr2=0x5584b3a84ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3a84ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2390019e30 tx=0x7f2390004b40 comp rx=0 tx=0).stop 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 shutdown_connections 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3ab9330 0x5584b3abb750 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3a84ba0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7bf40 0x5584b3a87170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2583132708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7c860 0x5584b3a84460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 >> 172.21.15.45:0/2583132708 conn(0x5584b39f1200 msgr2=0x5584b3a77f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.942+0000 7f23817fa700 0 client.25562 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.943+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 shutdown_connections 2023-09-23T09:31:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.943+0000 7f23ac086b80 1 -- 172.21.15.45:0/2583132708 wait complete. 2023-09-23T09:31:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.943+0000 7f23ac086b80 20 client.25562 trim_cache size 0 max 16384 2023-09-23T09:31:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.970+0000 7f23ac086b80 1 Processor -- start 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f23ac086b80 1 -- start start 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 0x5584b3a7ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7c860 0x5584b3a8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a87710 0x5584b3a83c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f23ac086b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b38508c0 con 0x5584b3a7bf40 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f23ac086b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b3a79100 con 0x5584b3a7c860 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f23ac086b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b3a8e7a0 con 0x5584b3a87710 2023-09-23T09:31:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f239d437700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7c860 0x5584b3a8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.971+0000 7f239d437700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7c860 0x5584b3a8e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55166/0 (socket says 172.21.15.45:55166) 2023-09-23T09:31:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.972+0000 7f239d437700 1 -- 172.21.15.45:0/3938598231 learned_addr learned my addr 172.21.15.45:0/3938598231 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.972+0000 7f239d437700 1 -- 172.21.15.45:0/3938598231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a87710 msgr2=0x5584b3a83c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.972+0000 7f239d437700 1 --2- 172.21.15.45:0/3938598231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a87710 0x5584b3a83c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.972+0000 7f239d437700 1 -- 172.21.15.45:0/3938598231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 msgr2=0x5584b3a7ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.972+0000 7f239d437700 1 --2- 172.21.15.45:0/3938598231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 0x5584b3a7ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.973+0000 7f239d437700 1 -- 172.21.15.45:0/3938598231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584b37e3c00 con 0x5584b3a7c860 2023-09-23T09:31:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.982+0000 7f239d437700 1 --2- 172.21.15.45:0/3938598231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7c860 0x5584b3a8e260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2398068d80 tx=0x7f2388005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80e0e3225576035 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.983+0000 7f238cff9700 1 -- 172.21.15.45:0/3938598231 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388023070 con 0x5584b3a7c860 2023-09-23T09:31:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.983+0000 7f238cff9700 1 -- 172.21.15.45:0/3938598231 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f238801e040 con 0x5584b3a7c860 2023-09-23T09:31:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.983+0000 7f238cff9700 1 -- 172.21.15.45:0/3938598231 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388004760 con 0x5584b3a7c860 2023-09-23T09:31:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.986+0000 7f23ac086b80 1 -- 172.21.15.45:0/3938598231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7c860 msgr2=0x5584b3a8e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.986+0000 7f23ac086b80 1 --2- 172.21.15.45:0/3938598231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7c860 0x5584b3a8e260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2398068d80 tx=0x7f2388005960 comp rx=0 tx=0).stop 2023-09-23T09:31:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.989+0000 7f23ac086b80 1 -- 172.21.15.45:0/3938598231 shutdown_connections 2023-09-23T09:31:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.989+0000 7f23ac086b80 1 --2- 172.21.15.45:0/3938598231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a87710 0x5584b3a83c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.989+0000 7f23ac086b80 1 --2- 172.21.15.45:0/3938598231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 0x5584b3a7ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.989+0000 7f23ac086b80 1 --2- 172.21.15.45:0/3938598231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a7c860 0x5584b3a8e260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.989+0000 7f23ac086b80 1 -- 172.21.15.45:0/3938598231 >> 172.21.15.45:0/3938598231 conn(0x5584b39f1200 msgr2=0x5584b3a77590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.990+0000 7f23ac086b80 1 -- 172.21.15.45:0/3938598231 shutdown_connections 2023-09-23T09:31:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.990+0000 7f23ac086b80 1 -- 172.21.15.45:0/3938598231 wait complete. 2023-09-23T09:31:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 -- start start 2023-09-23T09:31:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 Processor -- start 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 0x5584b3a8d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a8dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3b06130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b3b06670 con 0x5584b3a7bf40 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b3b067e0 con 0x5584b3a87710 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b3b069a0 con 0x5584b3a7c860 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.991+0000 7f23ac086b80 10 client.0 _ll_register_callbacks cb 0x7f2390019800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.992+0000 7f239e439700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3b06130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.992+0000 7f239e439700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3b06130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55170/0 (socket says 172.21.15.45:55170) 2023-09-23T09:31:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.992+0000 7f239e439700 1 -- 172.21.15.45:0/2715018565 learned_addr learned my addr 172.21.15.45:0/2715018565 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:31:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.993+0000 7f239d437700 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a8dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.994+0000 7f239dc38700 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 0x5584b3a8d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.994+0000 7f239e439700 1 -- 172.21.15.45:0/2715018565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 msgr2=0x5584b3a8dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.994+0000 7f239e439700 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a8dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.994+0000 7f239e439700 1 -- 172.21.15.45:0/2715018565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 msgr2=0x5584b3a8d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:31:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.994+0000 7f239e439700 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 0x5584b3a8d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.994+0000 7f239e439700 1 -- 172.21.15.45:0/2715018565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239810d410 con 0x5584b3a87710 2023-09-23T09:31:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.994+0000 7f239e439700 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3b06130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f239002b830 tx=0x7f239002bad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4f99fd00cf4763 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:31:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.995+0000 7f23817fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.995+0000 7f23817fa700 1 -- 172.21.15.45:0/2715018565 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000e070 con 0x5584b3a87710 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.995+0000 7f23817fa700 1 -- 172.21.15.45:0/2715018565 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2390017de0 con 0x5584b3a87710 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.995+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2394099ca0 con 0x5584b3a87710 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.995+0000 7f23817fa700 1 -- 172.21.15.45:0/2715018565 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2390035440 con 0x5584b3a87710 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.995+0000 7f23ac086b80 10 client.25568 fetch_fsmap learned FSMap version 28 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.995+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5584b38211f0 con 0x5584b3a87710 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23817fa700 1 -- 172.21.15.45:0/2715018565 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2394099ca0 con 0x5584b3a87710 2023-09-23T09:31:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23817fa700 1 -- 172.21.15.45:0/2715018565 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f239003d070 con 0x5584b3a87710 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23ac086b80 10 client.25568 fetch_fsmap finished waiting for FSMap version 28 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23ac086b80 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23ac086b80 20 client.25568 populate_metadata read hostname 'smithi045' 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3b0e000 0x5584b3b10420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23ac086b80 4 client.25568 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.996+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584b39c8ad0 con 0x5584b3b0e000 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.997+0000 7f239dc38700 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3b0e000 0x5584b3b10420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:31:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.997+0000 7f239dc38700 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3b0e000 0x5584b3b10420 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10babef465b42cf server_cookie=3771fa705a1ab231 in_seq=0 out_seq=0 2023-09-23T09:31:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:56.997+0000 7f23817fa700 10 client.25568 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.029+0000 7f23817fa700 1 -- 172.21.15.45:0/2715018565 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584b39c8ad0 con 0x5584b3b0e000 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.029+0000 7f23817fa700 10 client.25568 handle_command_reply: tid=0 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.029+0000 7f23ac086b80 1 client.25568 shutdown 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 msgr2=0x5584b3b06130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3b06130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f239002b830 tx=0x7f239002bad0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 shutdown_connections 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5584b3b0e000 0x5584b3b10420 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584b3a87710 0x5584b3b06130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23817fa700 0 client.25568 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:31:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5584b3a7bf40 0x5584b3a8d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 --2- 172.21.15.45:0/2715018565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584b3a7c860 0x5584b3a8dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 >> 172.21.15.45:0/2715018565 conn(0x5584b39f1200 msgr2=0x5584b3a77e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 shutdown_connections 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 1 -- 172.21.15.45:0/2715018565 wait complete. 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.030+0000 7f23ac086b80 20 client.25568 trim_cache size 0 max 16384 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:31:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:31:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f23840a1780 msgr2=0x7f23840a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f23840a1780 0x7f23840a3c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f239000f590 tx=0x7f2390009920 comp rx=0 tx=0).stop 2023-09-23T09:31:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23980fff00 msgr2=0x7f2398072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:31:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23980fff00 0x7f2398072660 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f239400b530 tx=0x7f239400b840 comp rx=0 tx=0).stop 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 shutdown_connections 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f23840a1780 0x7f23840a3c40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239806b9a0 0x7f239806be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23980fff00 0x7f2398072660 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.031+0000 7f239fe9c700 1 --2- 172.21.15.45:0/3206015808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2398105020 0x7f239806b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.032+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 >> 172.21.15.45:0/3206015808 conn(0x7f23980fb760 msgr2=0x7f23980fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.032+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 shutdown_connections 2023-09-23T09:31:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:31:57.032+0000 7f239fe9c700 1 -- 172.21.15.45:0/3206015808 wait complete. 2023-09-23T09:31:57.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:57 smithi098 ceph-mon[102386]: pgmap v79: 97 pgs: 97 active+clean; 3.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 2.02k op/s 2023-09-23T09:31:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:57 smithi045 ceph-mon[98627]: pgmap v79: 97 pgs: 97 active+clean; 3.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 2.02k op/s 2023-09-23T09:31:57.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:57 smithi123 ceph-mon[103577]: pgmap v79: 97 pgs: 97 active+clean; 3.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 2.02k op/s 2023-09-23T09:31:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:58 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:31:58.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:58 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:31:58.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:58 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:31:59.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:31:59 smithi098 ceph-mon[102386]: pgmap v80: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 MiB/s wr, 2.45k op/s 2023-09-23T09:31:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:31:59 smithi045 ceph-mon[98627]: pgmap v80: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 MiB/s wr, 2.45k op/s 2023-09-23T09:31:59.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:31:59 smithi123 ceph-mon[103577]: pgmap v80: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 MiB/s wr, 2.45k op/s 2023-09-23T09:32:01.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:01 smithi098 ceph-mon[102386]: pgmap v81: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 MiB/s wr, 2.03k op/s 2023-09-23T09:32:01.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:01 smithi045 ceph-mon[98627]: pgmap v81: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 MiB/s wr, 2.03k op/s 2023-09-23T09:32:01.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:01 smithi123 ceph-mon[103577]: pgmap v81: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 MiB/s wr, 2.03k op/s 2023-09-23T09:32:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 135.0 s, 2834.5 tps, lat 34.921 ms stddev 69.140 2023-09-23T09:32:03.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:03 smithi098 ceph-mon[102386]: pgmap v82: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 48 MiB/s wr, 2.16k op/s 2023-09-23T09:32:03.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:03 smithi045 ceph-mon[98627]: pgmap v82: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 48 MiB/s wr, 2.16k op/s 2023-09-23T09:32:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:03 smithi123 ceph-mon[103577]: pgmap v82: 97 pgs: 97 active+clean; 3.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 48 MiB/s wr, 2.16k op/s 2023-09-23T09:32:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:05 smithi098 ceph-mon[102386]: pgmap v83: 97 pgs: 97 active+clean; 4.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 65 MiB/s wr, 3.30k op/s 2023-09-23T09:32:05.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:05 smithi045 ceph-mon[98627]: pgmap v83: 97 pgs: 97 active+clean; 4.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 65 MiB/s wr, 3.30k op/s 2023-09-23T09:32:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:05 smithi123 ceph-mon[103577]: pgmap v83: 97 pgs: 97 active+clean; 4.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 65 MiB/s wr, 3.30k op/s 2023-09-23T09:32:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 140.0 s, 2971.0 tps, lat 33.679 ms stddev 43.794 2023-09-23T09:32:07.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:07 smithi098 ceph-mon[102386]: pgmap v84: 97 pgs: 97 active+clean; 4.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 1.85k op/s 2023-09-23T09:32:07.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:07 smithi045 ceph-mon[98627]: pgmap v84: 97 pgs: 97 active+clean; 4.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 1.85k op/s 2023-09-23T09:32:07.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:07 smithi123 ceph-mon[103577]: pgmap v84: 97 pgs: 97 active+clean; 4.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 1.85k op/s 2023-09-23T09:32:09.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:09 smithi098 ceph-mon[102386]: pgmap v85: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 43 MiB/s wr, 2.24k op/s 2023-09-23T09:32:09.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:09 smithi045 ceph-mon[98627]: pgmap v85: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 43 MiB/s wr, 2.24k op/s 2023-09-23T09:32:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:09 smithi123 ceph-mon[103577]: pgmap v85: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 43 MiB/s wr, 2.24k op/s 2023-09-23T09:32:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.221+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2025086124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 msgr2=0x7fb9a0079080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.221+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2025086124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0079080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb998009f90 tx=0x7fb99800b330 comp rx=0 tx=0).stop 2023-09-23T09:32:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.223+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2025086124 shutdown_connections 2023-09-23T09:32:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.223+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2025086124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0079080 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.223+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2025086124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9a010d430 0x7fb9a010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.223+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2025086124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a010dd50 0x7fb9a0107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.223+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2025086124 >> 172.21.15.45:0/2025086124 conn(0x7fb9a006cee0 msgr2=0x7fb9a006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.223+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2025086124 shutdown_connections 2023-09-23T09:32:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.223+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2025086124 wait complete. 2023-09-23T09:32:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.224+0000 7fb9a7ff6700 1 Processor -- start 2023-09-23T09:32:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.224+0000 7fb9a7ff6700 1 -- start start 2023-09-23T09:32:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a7ff6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0083600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a7ff6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9a010d430 0x7fb9a007d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a7ff6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a010dd50 0x7fb9a007d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a7ff6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a007de50 con 0x7fb9a0108550 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a7ff6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a007dfc0 con 0x7fb9a010d430 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a7ff6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a007e180 con 0x7fb9a010dd50 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0083600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0083600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33834/0 (socket says 172.21.15.45:33834) 2023-09-23T09:32:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 -- 172.21.15.45:0/2169586455 learned_addr learned my addr 172.21.15.45:0/2169586455 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a6593700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a010dd50 0x7fb9a007d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 -- 172.21.15.45:0/2169586455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a010dd50 msgr2=0x7fb9a007d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a010dd50 0x7fb9a007d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 -- 172.21.15.45:0/2169586455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9a010d430 msgr2=0x7fb9a007d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9a010d430 0x7fb9a007d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.225+0000 7fb9a5d92700 1 -- 172.21.15.45:0/2169586455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb998009c40 con 0x7fb9a0108550 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.226+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0083600 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb99400e9a0 tx=0x7fb99400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.226+0000 7fb992ffd700 1 -- 172.21.15.45:0/2169586455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994009790 con 0x7fb9a0108550 2023-09-23T09:32:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.226+0000 7fb992ffd700 1 -- 172.21.15.45:0/2169586455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99400cbd0 con 0x7fb9a0108550 2023-09-23T09:32:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.226+0000 7fb992ffd700 1 -- 172.21.15.45:0/2169586455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994018680 con 0x7fb9a0108550 2023-09-23T09:32:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.226+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a01b6fe0 con 0x7fb9a0108550 2023-09-23T09:32:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.226+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a01b74f0 con 0x7fb9a0108550 2023-09-23T09:32:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.229+0000 7fb992ffd700 1 -- 172.21.15.45:0/2169586455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fb99400cd40 con 0x7fb9a0108550 2023-09-23T09:32:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.229+0000 7fb992ffd700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb98c0a17d0 0x7fb98c0a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.229+0000 7fb992ffd700 1 -- 172.21.15.45:0/2169586455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb994014070 con 0x7fb9a0108550 2023-09-23T09:32:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.229+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb98c0a17d0 0x7fb98c0a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.230+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb98c0a17d0 0x7fb98c0a3c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9a007edd0 tx=0x7fb99c00c040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.230+0000 7fb9b41e0b80 1 Processor -- start 2023-09-23T09:32:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.230+0000 7fb9b41e0b80 1 -- start start 2023-09-23T09:32:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.231+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164516f40 0x558164517320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.231+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x5581645221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.231+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164522710 0x558164524b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.231+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164480400 con 0x558164516f40 2023-09-23T09:32:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.231+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164471bc0 con 0x558164517860 2023-09-23T09:32:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.231+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816446f4e0 con 0x558164522710 2023-09-23T09:32:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x5581645221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x5581645221d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55196/0 (socket says 172.21.15.45:55196) 2023-09-23T09:32:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 -- 172.21.15.45:0/984895372 learned_addr learned my addr 172.21.15.45:0/984895372 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a6593700 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164522710 0x558164524b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164516f40 0x558164517320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 -- 172.21.15.45:0/984895372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164522710 msgr2=0x558164524b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164522710 0x558164524b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 -- 172.21.15.45:0/984895372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164516f40 msgr2=0x558164517320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164516f40 0x558164517320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.232+0000 7fb9a5591700 1 -- 172.21.15.45:0/984895372 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581644cd870 con 0x558164517860 2023-09-23T09:32:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.234+0000 7fb9a5591700 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x5581645221d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009a40 tx=0x7fb99c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81cf9d7fc39e160d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.234+0000 7fb98bfff700 1 -- 172.21.15.45:0/984895372 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01d070 con 0x558164517860 2023-09-23T09:32:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.234+0000 7fb98bfff700 1 -- 172.21.15.45:0/984895372 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99c004cf0 con 0x558164517860 2023-09-23T09:32:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.234+0000 7fb98bfff700 1 -- 172.21.15.45:0/984895372 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c011070 con 0x558164517860 2023-09-23T09:32:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/984895372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 msgr2=0x5581645221d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x5581645221d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009a40 tx=0x7fb99c011000 comp rx=0 tx=0).stop 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/984895372 shutdown_connections 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164522710 0x558164524b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164516f40 0x558164517320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/984895372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x5581645221d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/984895372 >> 172.21.15.45:0/984895372 conn(0x55816448c200 msgr2=0x5581645133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/984895372 shutdown_connections 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.235+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/984895372 wait complete. 2023-09-23T09:32:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 -- start start 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 Processor -- start 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164522190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x55816451f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x55816451fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816446add0 con 0x558164522710 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581644076b0 con 0x558164517860 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164407350 con 0x558164516f40 2023-09-23T09:32:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.236+0000 7fb9b41e0b80 10 client.0 _ll_register_callbacks cb 0x5581642647a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5d92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164522190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x55816451f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x55816451f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55200/0 (socket says 172.21.15.45:55200) 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5591700 1 -- 172.21.15.45:0/3337999789 learned_addr learned my addr 172.21.15.45:0/3337999789 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a6593700 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x55816451fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5d92700 1 -- 172.21.15.45:0/3337999789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 msgr2=0x55816451f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x55816451f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5d92700 1 -- 172.21.15.45:0/3337999789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 msgr2=0x55816451fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x55816451fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5d92700 1 -- 172.21.15.45:0/3337999789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c009720 con 0x558164516f40 2023-09-23T09:32:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a6593700 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x55816451fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:32:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164522190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb9940cac60 tx=0x7fb994098a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c935a3d89c7252 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9897fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:32:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9897fa700 1 -- 172.21.15.45:0/3337999789 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994099b70 con 0x558164516f40 2023-09-23T09:32:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55816450b090 con 0x558164516f40 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.237+0000 7fb9897fa700 1 -- 172.21.15.45:0/3337999789 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb99409a480 con 0x558164516f40 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9897fa700 1 -- 172.21.15.45:0/3337999789 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb99407a070 con 0x558164516f40 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 10 client.25588 fetch_fsmap learned FSMap version 29 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558164295d80 con 0x558164516f40 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9897fa700 1 -- 172.21.15.45:0/3337999789 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55816450b090 con 0x558164516f40 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9897fa700 1 -- 172.21.15.45:0/3337999789 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb994096270 con 0x558164516f40 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 10 client.25588 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T09:32:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 10 client.25588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 20 client.25588 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645322c0 0x5581645346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 4 client.25588 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558164463db0 con 0x5581645322c0 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.238+0000 7fb9a5591700 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645322c0 0x5581645346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.239+0000 7fb9897fa700 10 client.25588 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.239+0000 7fb9a5591700 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645322c0 0x5581645346e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d2fa67d91ff427 server_cookie=552be64c158ad3ee in_seq=0 out_seq=0 2023-09-23T09:32:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.240+0000 7fb9897fa700 1 -- 172.21.15.45:0/3337999789 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558164463db0 con 0x5581645322c0 2023-09-23T09:32:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.240+0000 7fb9897fa700 10 client.25588 handle_command_reply: tid=0 2023-09-23T09:32:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.240+0000 7fb9b41e0b80 1 client.25588 shutdown 2023-09-23T09:32:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 msgr2=0x558164522190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164522190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb9940cac60 tx=0x7fb994098a30 comp rx=0 tx=0).stop 2023-09-23T09:32:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 shutdown_connections 2023-09-23T09:32:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645322c0 0x5581645346e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x55816451fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164522190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9897fa700 0 client.25588 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/3337999789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x55816451f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 >> 172.21.15.45:0/3337999789 conn(0x55816448c200 msgr2=0x558164512fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 shutdown_connections 2023-09-23T09:32:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/3337999789 wait complete. 2023-09-23T09:32:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.241+0000 7fb9b41e0b80 20 client.25588 trim_cache size 0 max 16384 2023-09-23T09:32:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.264+0000 7fb9b41e0b80 1 Processor -- start 2023-09-23T09:32:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.264+0000 7fb9b41e0b80 1 -- start start 2023-09-23T09:32:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164516f40 0x558164519f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164517860 0x558164524b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164529260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581642eb8c0 con 0x558164522710 2023-09-23T09:32:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164517ea0 con 0x558164516f40 2023-09-23T09:32:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164534900 con 0x558164517860 2023-09-23T09:32:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164517860 0x558164524b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164517860 0x558164524b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35962/0 (socket says 172.21.15.45:35962) 2023-09-23T09:32:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9a5591700 1 -- 172.21.15.45:0/2119639514 learned_addr learned my addr 172.21.15.45:0/2119639514 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9a6593700 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164529260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164516f40 0x558164519f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.265+0000 7fb9a5591700 1 -- 172.21.15.45:0/2119639514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164516f40 msgr2=0x558164519f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164516f40 0x558164519f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb9a5591700 1 -- 172.21.15.45:0/2119639514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 msgr2=0x558164529260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164529260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb9a5591700 1 -- 172.21.15.45:0/2119639514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55816427ec00 con 0x558164517860 2023-09-23T09:32:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164517860 0x558164524b00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb99c013040 tx=0x7fb99c00ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7bb0c009339276b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb990ff9700 1 -- 172.21.15.45:0/2119639514 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01d040 con 0x558164517860 2023-09-23T09:32:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb990ff9700 1 -- 172.21.15.45:0/2119639514 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99c005770 con 0x558164517860 2023-09-23T09:32:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.266+0000 7fb990ff9700 1 -- 172.21.15.45:0/2119639514 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c005190 con 0x558164517860 2023-09-23T09:32:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.267+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2119639514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164517860 msgr2=0x558164524b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.267+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164517860 0x558164524b00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb99c013040 tx=0x7fb99c00ace0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.267+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2119639514 shutdown_connections 2023-09-23T09:32:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.267+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164529260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.267+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164516f40 0x558164519f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.267+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2119639514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164517860 0x558164524b00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.267+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2119639514 >> 172.21.15.45:0/2119639514 conn(0x55816448c200 msgr2=0x558164539000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.268+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2119639514 shutdown_connections 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.268+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2119639514 wait complete. 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.268+0000 7fb9b41e0b80 1 -- start start 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.268+0000 7fb9b41e0b80 1 Processor -- start 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164525230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x558164525770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9b41e0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164525cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581645851d0 con 0x558164522710 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164585310 con 0x558164517860 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9b41e0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581645854d0 con 0x558164516f40 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9b41e0b80 10 client.0 _ll_register_callbacks cb 0x7fb99409a090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5d92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164525230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x558164525770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a6593700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164525cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x558164525770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55214/0 (socket says 172.21.15.45:55214) 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5d92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164525230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35968/0 (socket says 172.21.15.45:35968) 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 -- 172.21.15.45:0/2437967389 learned_addr learned my addr 172.21.15.45:0/2437967389 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a6593700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164525cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33866/0 (socket says 172.21.15.45:33866) 2023-09-23T09:32:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 -- 172.21.15.45:0/2437967389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 msgr2=0x558164525230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164525230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 -- 172.21.15.45:0/2437967389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 msgr2=0x558164525cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164525cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.269+0000 7fb9a5591700 1 -- 172.21.15.45:0/2437967389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581644cd870 con 0x558164517860 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9a5591700 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x558164525770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x558164463ef0 tx=0x7fb99c004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb61507e16551f87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9897fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9897fa700 1 -- 172.21.15.45:0/2437967389 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01d050 con 0x558164517860 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9897fa700 1 -- 172.21.15.45:0/2437967389 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb99c031cc0 con 0x558164517860 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb99c004cf0 con 0x558164517860 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9897fa700 1 -- 172.21.15.45:0/2437967389 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb99c011070 con 0x558164517860 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9b41e0b80 10 client.25583 fetch_fsmap learned FSMap version 29 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.270+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5581642bc1f0 con 0x558164517860 2023-09-23T09:32:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9897fa700 1 -- 172.21.15.45:0/2437967389 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99c004cf0 con 0x558164517860 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9897fa700 1 -- 172.21.15.45:0/2437967389 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb99c0184d0 con 0x558164517860 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9b41e0b80 10 client.25583 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9b41e0b80 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9b41e0b80 20 client.25583 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645869a0 0x558164588dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9b41e0b80 4 client.25583 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.271+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558164463ad0 con 0x5581645869a0 2023-09-23T09:32:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.272+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645869a0 0x558164588dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.272+0000 7fb9897fa700 10 client.25583 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.273+0000 7fb9a5d92700 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645869a0 0x558164588dc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76e6daf08101186 server_cookie=530fc49d142c882e in_seq=0 out_seq=0 2023-09-23T09:32:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.289+0000 7fb9897fa700 1 -- 172.21.15.45:0/2437967389 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558164463ad0 con 0x5581645869a0 2023-09-23T09:32:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.289+0000 7fb9897fa700 10 client.25583 handle_command_reply: tid=0 2023-09-23T09:32:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.289+0000 7fb9b41e0b80 1 client.25583 shutdown 2023-09-23T09:32:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.289+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 msgr2=0x558164525770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.289+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x558164525770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x558164463ef0 tx=0x7fb99c004ba0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.290+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 shutdown_connections 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.290+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5581645869a0 0x558164588dc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.290+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558164522710 0x558164525cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.290+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558164516f40 0x558164525230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.290+0000 7fb9b41e0b80 1 --2- 172.21.15.45:0/2437967389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558164517860 0x558164525770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.290+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 >> 172.21.15.45:0/2437967389 conn(0x55816448c200 msgr2=0x55816451a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.290+0000 7fb9897fa700 0 client.25583 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.291+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 shutdown_connections 2023-09-23T09:32:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.291+0000 7fb9b41e0b80 1 -- 172.21.15.45:0/2437967389 wait complete. 2023-09-23T09:32:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9b41e0b80 20 client.25583 trim_cache size 0 max 16384 2023-09-23T09:32:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:32:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:32:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:32:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb98c0a17d0 msgr2=0x7fb98c0a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb98c0a17d0 0x7fb98c0a3c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9a007edd0 tx=0x7fb99c00c040 comp rx=0 tx=0).stop 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 msgr2=0x7fb9a0083600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0083600 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb99400e9a0 tx=0x7fb99400ecb0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 shutdown_connections 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a010dd50 0x7fb9a007d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb98c0a17d0 0x7fb98c0a3c90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a0108550 0x7fb9a0083600 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 --2- 172.21.15.45:0/2169586455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9a010d430 0x7fb9a007d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 >> 172.21.15.45:0/2169586455 conn(0x7fb9a006cee0 msgr2=0x7fb9a00714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 shutdown_connections 2023-09-23T09:32:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:10.293+0000 7fb9a7ff6700 1 -- 172.21.15.45:0/2169586455 wait complete. 2023-09-23T09:32:11.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:11 smithi098 ceph-mon[102386]: pgmap v86: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-09-23T09:32:11.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:11 smithi045 ceph-mon[98627]: pgmap v86: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-09-23T09:32:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:11 smithi123 ceph-mon[103577]: pgmap v86: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-09-23T09:32:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 145.0 s, 2292.6 tps, lat 43.969 ms stddev 45.432 2023-09-23T09:32:13.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:13 smithi098 ceph-mon[102386]: pgmap v87: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-09-23T09:32:13.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:13 smithi045 ceph-mon[98627]: pgmap v87: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-09-23T09:32:13.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:13 smithi123 ceph-mon[103577]: pgmap v87: 97 pgs: 97 active+clean; 4.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-09-23T09:32:15.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:15 smithi098 ceph-mon[102386]: pgmap v88: 97 pgs: 97 active+clean; 4.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 43 MiB/s wr, 2.20k op/s 2023-09-23T09:32:15.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:15 smithi045 ceph-mon[98627]: pgmap v88: 97 pgs: 97 active+clean; 4.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 43 MiB/s wr, 2.20k op/s 2023-09-23T09:32:15.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:15 smithi123 ceph-mon[103577]: pgmap v88: 97 pgs: 97 active+clean; 4.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 43 MiB/s wr, 2.20k op/s 2023-09-23T09:32:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 150.0 s, 2099.9 tps, lat 47.740 ms stddev 73.130 2023-09-23T09:32:17.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:17 smithi098 ceph-mon[102386]: pgmap v89: 97 pgs: 97 active+clean; 4.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 920 op/s 2023-09-23T09:32:17.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:17 smithi045 ceph-mon[98627]: pgmap v89: 97 pgs: 97 active+clean; 4.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 920 op/s 2023-09-23T09:32:17.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:17 smithi123 ceph-mon[103577]: pgmap v89: 97 pgs: 97 active+clean; 4.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 920 op/s 2023-09-23T09:32:19.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:19 smithi098 ceph-mon[102386]: pgmap v90: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 MiB/s wr, 2.54k op/s 2023-09-23T09:32:19.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:19 smithi045 ceph-mon[98627]: pgmap v90: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 MiB/s wr, 2.54k op/s 2023-09-23T09:32:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:19 smithi123 ceph-mon[103577]: pgmap v90: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 MiB/s wr, 2.54k op/s 2023-09-23T09:32:21.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:21 smithi098 ceph-mon[102386]: pgmap v91: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 2.14k op/s 2023-09-23T09:32:21.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:21 smithi045 ceph-mon[98627]: pgmap v91: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 2.14k op/s 2023-09-23T09:32:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:21 smithi123 ceph-mon[103577]: pgmap v91: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 2.14k op/s 2023-09-23T09:32:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 155.0 s, 2977.6 tps, lat 33.471 ms stddev 38.093 2023-09-23T09:32:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 -- 172.21.15.45:0/2511731004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c100690 msgr2=0x7f980c0690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 --2- 172.21.15.45:0/2511731004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c100690 0x7f980c0690e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009990 tx=0x7f97f4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 -- 172.21.15.45:0/2511731004 shutdown_connections 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 --2- 172.21.15.45:0/2511731004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c100690 0x7f980c0690e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 --2- 172.21.15.45:0/2511731004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c105d90 0x7f980c106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 --2- 172.21.15.45:0/2511731004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f980c0ffcd0 0x7f980c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 -- 172.21.15.45:0/2511731004 >> 172.21.15.45:0/2511731004 conn(0x7f980c076e20 msgr2=0x7f980c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 -- 172.21.15.45:0/2511731004 shutdown_connections 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.460+0000 7f9812342700 1 -- 172.21.15.45:0/2511731004 wait complete. 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 Processor -- start 2023-09-23T09:32:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 -- start start 2023-09-23T09:32:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c0ffcd0 0x7f980c198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c100690 0x7f980c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f980c105d90 0x7f980c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c19e1b0 con 0x7f980c105d90 2023-09-23T09:32:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c19e320 con 0x7f980c100690 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f9812342700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c19e4e0 con 0x7f980c0ffcd0 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f980bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c0ffcd0 0x7f980c198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c100690 0x7f980c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f980bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c0ffcd0 0x7f980c198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:35982/0 (socket says 172.21.15.45:35982) 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c100690 0x7f980c199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55226/0 (socket says 172.21.15.45:55226) 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f980bfff700 1 -- 172.21.15.45:0/2074898240 learned_addr learned my addr 172.21.15.45:0/2074898240 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.461+0000 7f98108df700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f980c105d90 0x7f980c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.462+0000 7f980b7fe700 1 -- 172.21.15.45:0/2074898240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c0ffcd0 msgr2=0x7f980c198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.462+0000 7f980b7fe700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c0ffcd0 0x7f980c198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.462+0000 7f980b7fe700 1 -- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f980c105d90 msgr2=0x7f980c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.462+0000 7f980b7fe700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f980c105d90 0x7f980c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.462+0000 7f980b7fe700 1 -- 172.21.15.45:0/2074898240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804009c40 con 0x7f980c100690 2023-09-23T09:32:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.462+0000 7f980b7fe700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c100690 0x7f980c199500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f980400efc0 tx=0x7f980400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.463+0000 7f98097fa700 1 -- 172.21.15.45:0/2074898240 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804018070 con 0x7f980c100690 2023-09-23T09:32:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.463+0000 7f98097fa700 1 -- 172.21.15.45:0/2074898240 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980400f040 con 0x7f980c100690 2023-09-23T09:32:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.463+0000 7f98097fa700 1 -- 172.21.15.45:0/2074898240 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804004310 con 0x7f980c100690 2023-09-23T09:32:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.464+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc009520 con 0x7f980c100690 2023-09-23T09:32:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.464+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f40095f0 con 0x7f980c100690 2023-09-23T09:32:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.467+0000 7f98097fa700 1 -- 172.21.15.45:0/2074898240 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f980400cad0 con 0x7f980c100690 2023-09-23T09:32:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.468+0000 7f98097fa700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f97f80a1860 0x7f97f80a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.468+0000 7f98097fa700 1 -- 172.21.15.45:0/2074898240 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f98040cca50 con 0x7f980c100690 2023-09-23T09:32:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.469+0000 7f981e52cb80 1 Processor -- start 2023-09-23T09:32:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.469+0000 7f981e52cb80 1 -- start start 2023-09-23T09:32:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.469+0000 7f980bfff700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f97f80a1860 0x7f97f80a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515d8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 0x5620515d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515ec990 0x5620515eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f981e52cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051539400 con 0x5620515d9070 2023-09-23T09:32:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f981e52cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205152abc0 con 0x5620515d8750 2023-09-23T09:32:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f981e52cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620515284e0 con 0x5620515ec990 2023-09-23T09:32:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f980bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515d8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f980bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515d8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55236/0 (socket says 172.21.15.45:55236) 2023-09-23T09:32:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.470+0000 7f980bfff700 1 -- 172.21.15.45:0/1699421863 learned_addr learned my addr 172.21.15.45:0/1699421863 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f980bfff700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f97f80a1860 0x7f97f80a3d20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97fc013730 tx=0x7f97fc013af0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f980bfff700 1 -- 172.21.15.45:0/1699421863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515ec990 msgr2=0x5620515eed80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:32:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f980bfff700 1 --2- 172.21.15.45:0/1699421863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515ec990 0x5620515eed80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f980bfff700 1 -- 172.21.15.45:0/1699421863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 msgr2=0x5620515d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f980bfff700 1 --2- 172.21.15.45:0/1699421863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 0x5620515d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f980bfff700 1 -- 172.21.15.45:0/1699421863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562051586870 con 0x5620515d8750 2023-09-23T09:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f980bfff700 1 --2- 172.21.15.45:0/1699421863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515d8b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d00 tx=0x7f97fc020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b475d6f0772855f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f98027fc700 1 -- 172.21.15.45:0/1699421863 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc004220 con 0x5620515d8750 2023-09-23T09:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f98027fc700 1 -- 172.21.15.45:0/1699421863 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97fc004380 con 0x5620515d8750 2023-09-23T09:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.471+0000 7f98027fc700 1 -- 172.21.15.45:0/1699421863 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0208d0 con 0x5620515d8750 2023-09-23T09:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 -- 172.21.15.45:0/1699421863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 msgr2=0x5620515d8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1699421863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515d8b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d00 tx=0x7f97fc020000 comp rx=0 tx=0).stop 2023-09-23T09:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 -- 172.21.15.45:0/1699421863 shutdown_connections 2023-09-23T09:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1699421863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515ec990 0x5620515eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1699421863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515d8b30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1699421863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 0x5620515d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 -- 172.21.15.45:0/1699421863 >> 172.21.15.45:0/1699421863 conn(0x562051545200 msgr2=0x5620515cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 -- 172.21.15.45:0/1699421863 shutdown_connections 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.472+0000 7f981e52cb80 1 -- 172.21.15.45:0/1699421863 wait complete. 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 -- start start 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 Processor -- start 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 0x5620515ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515ec990 0x5620515f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051523dd0 con 0x5620515d9070 2023-09-23T09:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620514c06b0 con 0x5620515ec990 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620514c0350 con 0x5620515d8750 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.473+0000 7f981e52cb80 10 client.0 _ll_register_callbacks cb 0x56205131d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515ec990 0x5620515f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 0x5620515ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515ec990 0x5620515f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55244/0 (socket says 172.21.15.45:55244) 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 -- 172.21.15.45:0/1677926458 learned_addr learned my addr 172.21.15.45:0/1677926458 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 -- 172.21.15.45:0/1677926458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 msgr2=0x5620515ecd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 -- 172.21.15.45:0/1677926458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 msgr2=0x5620515ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 0x5620515ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 -- 172.21.15.45:0/1677926458 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc00f4c0 con 0x5620515ec990 2023-09-23T09:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f98108df700 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515ec990 0x5620515f7df0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97f4015000 tx=0x7f97f400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9628b61a3f946b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.474+0000 7f97eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1677926458 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f401d070 con 0x5620515ec990 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1677926458 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f97f4004500 con 0x5620515ec990 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620515c4090 con 0x5620515ec990 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1677926458 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97f40046c0 con 0x5620515ec990 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 10 client.25595 fetch_fsmap learned FSMap version 29 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56205134ed80 con 0x5620515ec990 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1677926458 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620515c4090 con 0x5620515ec990 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1677926458 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f97f400f7c0 con 0x5620515ec990 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 10 client.25595 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 10 client.25595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 20 client.25595 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x5620515fe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 4 client.25595 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:32:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.475+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56205151cdb0 con 0x5620515fc180 2023-09-23T09:32:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.476+0000 7f980bfff700 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x5620515fe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.476+0000 7f97eb7fe700 10 client.25595 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.476+0000 7f980bfff700 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x5620515fe5a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf786e2a1932bb71 server_cookie=ace482ee6910a2fa in_seq=0 out_seq=0 2023-09-23T09:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1677926458 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56205151cdb0 con 0x5620515fc180 2023-09-23T09:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f97eb7fe700 10 client.25595 handle_command_reply: tid=0 2023-09-23T09:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 client.25595 shutdown 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515ec990 msgr2=0x5620515f7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515ec990 0x5620515f7df0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97f4015000 tx=0x7f97f400bf60 comp rx=0 tx=0).stop 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 shutdown_connections 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x5620515fe5a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515ec990 0x5620515f7df0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f97eb7fe700 0 client.25595 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515ecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1677926458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515d9070 0x5620515ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 >> 172.21.15.45:0/1677926458 conn(0x562051545200 msgr2=0x5620515cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 shutdown_connections 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 1 -- 172.21.15.45:0/1677926458 wait complete. 2023-09-23T09:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.477+0000 7f981e52cb80 20 client.25595 trim_cache size 0 max 16384 2023-09-23T09:32:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 Processor -- start 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 -- start start 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d9070 0x5620515ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620513a48c0 con 0x5620515ec990 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620515cb170 con 0x5620515d8750 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f981e52cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620515d1390 con 0x5620515d9070 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f98108df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d9070 0x5620515ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f980bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f98108df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ed2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33902/0 (socket says 172.21.15.45:33902) 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f980bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55248/0 (socket says 172.21.15.45:55248) 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.502+0000 7f98108df700 1 -- 172.21.15.45:0/345864207 learned_addr learned my addr 172.21.15.45:0/345864207 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f980b7fe700 1 -- 172.21.15.45:0/345864207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 msgr2=0x5620515f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f980b7fe700 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f980b7fe700 1 -- 172.21.15.45:0/345864207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 msgr2=0x5620515ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f980b7fe700 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f980b7fe700 1 -- 172.21.15.45:0/345864207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562051337c00 con 0x5620515d9070 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f98108df700 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f980b7fe700 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d9070 0x5620515ecd70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98040ca190 tx=0x7f98040092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f617c1e4cb1a09c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f9802ffd700 1 -- 172.21.15.45:0/345864207 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980409a0b0 con 0x5620515d9070 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f9802ffd700 1 -- 172.21.15.45:0/345864207 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98040999b0 con 0x5620515d9070 2023-09-23T09:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.503+0000 7f9802ffd700 1 -- 172.21.15.45:0/345864207 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980401bc10 con 0x5620515d9070 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 -- 172.21.15.45:0/345864207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d9070 msgr2=0x5620515ecd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d9070 0x5620515ecd70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98040ca190 tx=0x7f98040092e0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 -- 172.21.15.45:0/345864207 shutdown_connections 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d8750 0x5620515f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 --2- 172.21.15.45:0/345864207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d9070 0x5620515ecd70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 -- 172.21.15.45:0/345864207 >> 172.21.15.45:0/345864207 conn(0x562051545200 msgr2=0x5620515d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 -- 172.21.15.45:0/345864207 shutdown_connections 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.504+0000 7f981e52cb80 1 -- 172.21.15.45:0/345864207 wait complete. 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.505+0000 7f981e52cb80 1 -- start start 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.505+0000 7f981e52cb80 1 Processor -- start 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d9070 0x5620515edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f981e52cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f981e52cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620515ee730 con 0x5620515ec990 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f981e52cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620515ee8a0 con 0x5620515d9070 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f981e52cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620515eea60 con 0x5620515d8750 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f981e52cb80 10 client.0 _ll_register_callbacks cb 0x7f97f4009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d9070 0x5620515edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36010/0 (socket says 172.21.15.45:36010) 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d9070 0x5620515edcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55256/0 (socket says 172.21.15.45:55256) 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 -- 172.21.15.45:0/1595785398 learned_addr learned my addr 172.21.15.45:0/1595785398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f98108df700 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 -- 172.21.15.45:0/1595785398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d9070 msgr2=0x5620515edcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d9070 0x5620515edcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 -- 172.21.15.45:0/1595785398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 msgr2=0x5620515ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 -- 172.21.15.45:0/1595785398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56205134ed80 con 0x5620515d8750 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f98108df700 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f980bfff700 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515f18e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0204a0 tx=0x7f97fc009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=171526837ed2c5d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f97eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.506+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1595785398 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc02f030 con 0x5620515d8750 2023-09-23T09:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.507+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1595785398 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f97fc033570 con 0x5620515d8750 2023-09-23T09:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.507+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97fc004380 con 0x5620515d8750 2023-09-23T09:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.507+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1595785398 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97fc033730 con 0x5620515d8750 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.508+0000 7f981e52cb80 10 client.25606 fetch_fsmap learned FSMap version 29 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.508+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f97f40095f0 con 0x5620515d8750 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.508+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1595785398 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97fc004380 con 0x5620515d8750 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1595785398 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f97fc03b070 con 0x5620515d8750 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f981e52cb80 10 client.25606 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f981e52cb80 10 client.25606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f981e52cb80 20 client.25606 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x562051653370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f981e52cb80 4 client.25606 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56205151cad0 con 0x5620515fc180 2023-09-23T09:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.509+0000 7f980b7fe700 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x562051653370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.510+0000 7f97eb7fe700 10 client.25606 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:23.510+0000 7f980b7fe700 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x562051653370 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5129058b73db998 server_cookie=fc2ce5875e728479 in_seq=0 out_seq=0 2023-09-23T09:32:23.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:23 smithi098 ceph-mon[102386]: pgmap v92: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 2.19k op/s 2023-09-23T09:32:23.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:23 smithi045 ceph-mon[98627]: pgmap v92: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 2.19k op/s 2023-09-23T09:32:23.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:23 smithi123 ceph-mon[103577]: pgmap v92: 97 pgs: 97 active+clean; 4.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 2.19k op/s 2023-09-23T09:32:24.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:32:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.835+0000 7f6486f9a700 1 -- 172.21.15.45:0/4167623673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 msgr2=0x7f648010d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.835+0000 7f6486f9a700 1 --2- 172.21.15.45:0/4167623673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f648010d060 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f646c0098f0 tx=0x7f646c009c00 comp rx=0 tx=0).stop 2023-09-23T09:32:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.836+0000 7f6486f9a700 1 -- 172.21.15.45:0/4167623673 shutdown_connections 2023-09-23T09:32:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.836+0000 7f6486f9a700 1 --2- 172.21.15.45:0/4167623673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f648011a600 0x7f648011c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.836+0000 7f6486f9a700 1 --2- 172.21.15.45:0/4167623673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f648010d060 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.836+0000 7f6486f9a700 1 --2- 172.21.15.45:0/4167623673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f648010f940 0x7f648010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.836+0000 7f6486f9a700 1 -- 172.21.15.45:0/4167623673 >> 172.21.15.45:0/4167623673 conn(0x7f648006d0e0 msgr2=0x7f648006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.837+0000 7f6486f9a700 1 -- 172.21.15.45:0/4167623673 shutdown_connections 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.837+0000 7f6486f9a700 1 -- 172.21.15.45:0/4167623673 wait complete. 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 Processor -- start 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 -- start start 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f64801a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f648011a600 0x7f64801a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64801aa8a0 0x7f64801aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64801ab320 con 0x7f648010f4d0 2023-09-23T09:32:24.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64801ab490 con 0x7f64801aa8a0 2023-09-23T09:32:24.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.838+0000 7f6486f9a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64801ab650 con 0x7f648011a600 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f64801a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f64801a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33918/0 (socket says 172.21.15.45:33918) 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 -- 172.21.15.45:0/1236155591 learned_addr learned my addr 172.21.15.45:0/1236155591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f647ffff700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f648011a600 0x7f64801a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 -- 172.21.15.45:0/1236155591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f648011a600 msgr2=0x7f64801a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f648011a600 0x7f64801a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 -- 172.21.15.45:0/1236155591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64801aa8a0 msgr2=0x7f64801aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6485537700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64801aa8a0 0x7f64801aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64801aa8a0 0x7f64801aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 -- 172.21.15.45:0/1236155591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c009520 con 0x7f648010f4d0 2023-09-23T09:32:24.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6485537700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64801aa8a0 0x7f64801aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:32:24.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f6484d36700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f64801a5f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f646c010fe0 tx=0x7f646c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:24.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.839+0000 7f647dffb700 1 -- 172.21.15.45:0/1236155591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c01c070 con 0x7f648010f4d0 2023-09-23T09:32:24.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.840+0000 7f647dffb700 1 -- 172.21.15.45:0/1236155591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f646c022420 con 0x7f648010f4d0 2023-09-23T09:32:24.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.840+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6480079080 con 0x7f648010f4d0 2023-09-23T09:32:24.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.840+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6480079530 con 0x7f648010f4d0 2023-09-23T09:32:24.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.840+0000 7f647dffb700 1 -- 172.21.15.45:0/1236155591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c017660 con 0x7f648010f4d0 2023-09-23T09:32:24.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.843+0000 7f647dffb700 1 -- 172.21.15.45:0/1236155591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f646c020070 con 0x7f648010f4d0 2023-09-23T09:32:24.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.844+0000 7f647dffb700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f64700a1790 0x7f64700a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.844+0000 7f647dffb700 1 -- 172.21.15.45:0/1236155591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f646c0ce400 con 0x7f648010f4d0 2023-09-23T09:32:24.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 Processor -- start 2023-09-23T09:32:24.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 -- start start 2023-09-23T09:32:24.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 0x55ed028ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ead20 0x55ed028f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f5bd0 0x55ed028f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed02853460 con 0x55ed028ea400 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f647ffff700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f64700a1790 0x7f64700a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed02844c20 con 0x55ed028ead20 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.845+0000 7f6493184b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed02842540 con 0x55ed028f5bd0 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ead20 0x55ed028f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ead20 0x55ed028f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55278/0 (socket says 172.21.15.45:55278) 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 -- 172.21.15.45:0/910255373 learned_addr learned my addr 172.21.15.45:0/910255373 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f6484d36700 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 0x55ed028ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f6485537700 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f5bd0 0x55ed028f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 -- 172.21.15.45:0/910255373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f5bd0 msgr2=0x55ed028f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f5bd0 0x55ed028f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 -- 172.21.15.45:0/910255373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 msgr2=0x55ed028ea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 0x55ed028ea7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 -- 172.21.15.45:0/910255373 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed028a08d0 con 0x55ed028ead20 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f6485537700 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f5bd0 0x55ed028f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.846+0000 7f647ffff700 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ead20 0x55ed028f5690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f647400bec0 tx=0x7f6474003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635aca26adb1d0c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.847+0000 7f647ffff700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f64700a1790 0x7f64700a3c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6474003d60 tx=0x7f6474010000 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.847+0000 7f646affd700 1 -- 172.21.15.45:0/910255373 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400b7f0 con 0x55ed028ead20 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.847+0000 7f646affd700 1 -- 172.21.15.45:0/910255373 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6474010460 con 0x55ed028ead20 2023-09-23T09:32:24.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.847+0000 7f646affd700 1 -- 172.21.15.45:0/910255373 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647401e660 con 0x55ed028ead20 2023-09-23T09:32:24.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.847+0000 7f6493184b80 1 -- 172.21.15.45:0/910255373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ead20 msgr2=0x55ed028f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:24.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.847+0000 7f6493184b80 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ead20 0x55ed028f5690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f647400bec0 tx=0x7f6474003960 comp rx=0 tx=0).stop 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.848+0000 7f6493184b80 1 -- 172.21.15.45:0/910255373 shutdown_connections 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.848+0000 7f6493184b80 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f5bd0 0x55ed028f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.848+0000 7f6493184b80 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 0x55ed028ea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.848+0000 7f6493184b80 1 --2- 172.21.15.45:0/910255373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ead20 0x55ed028f5690 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f647400bec0 tx=0x7f6474003960 comp rx=0 tx=0).stop 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.848+0000 7f6493184b80 1 -- 172.21.15.45:0/910255373 >> 172.21.15.45:0/910255373 conn(0x55ed0285f260 msgr2=0x55ed028e6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.849+0000 7f6493184b80 1 -- 172.21.15.45:0/910255373 shutdown_connections 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.849+0000 7f6493184b80 1 -- 172.21.15.45:0/910255373 wait complete. 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.850+0000 7f6493184b80 1 -- start start 2023-09-23T09:32:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.850+0000 7f6493184b80 1 Processor -- start 2023-09-23T09:32:24.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.850+0000 7f6493184b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ea400 0x55ed028f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.850+0000 7f6493184b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f5bd0 0x55ed028f3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6493184b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 0x55ed02901730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6493184b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0283de30 con 0x55ed028f5bd0 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6493184b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed027da710 con 0x55ed028ea400 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6493184b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed027da3b0 con 0x55ed028f36b0 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6493184b80 10 client.0 _ll_register_callbacks cb 0x55ed02637800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ea400 0x55ed028f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ea400 0x55ed028f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55282/0 (socket says 172.21.15.45:55282) 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 -- 172.21.15.45:0/179887415 learned_addr learned my addr 172.21.15.45:0/179887415 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6485537700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 0x55ed02901730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f647ffff700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f5bd0 0x55ed028f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 -- 172.21.15.45:0/179887415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 msgr2=0x55ed02901730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 0x55ed02901730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 -- 172.21.15.45:0/179887415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f5bd0 msgr2=0x55ed028f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f5bd0 0x55ed028f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f6484d36700 1 -- 172.21.15.45:0/179887415 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647401a040 con 0x55ed028ea400 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.851+0000 7f647ffff700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f5bd0 0x55ed028f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f6484d36700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ea400 0x55ed028f2c30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f646c000c00 tx=0x7f646c09acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8233aec4156121c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f645ffff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:24.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f645ffff700 1 -- 172.21.15.45:0/179887415 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c0cbf60 con 0x55ed028ea400 2023-09-23T09:32:24.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed028de250 con 0x55ed028ea400 2023-09-23T09:32:24.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f645ffff700 1 -- 172.21.15.45:0/179887415 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f646c09b390 con 0x55ed028ea400 2023-09-23T09:32:24.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f645ffff700 1 -- 172.21.15.45:0/179887415 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f646c09f4e0 con 0x55ed028ea400 2023-09-23T09:32:24.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f6493184b80 10 client.25610 fetch_fsmap learned FSMap version 29 2023-09-23T09:32:24.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.852+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ed02668de0 con 0x55ed028ea400 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f645ffff700 1 -- 172.21.15.45:0/179887415 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed028de250 con 0x55ed028ea400 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f645ffff700 1 -- 172.21.15.45:0/179887415 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f646c09a880 con 0x55ed028ea400 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f6493184b80 10 client.25610 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f6493184b80 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f6493184b80 20 client.25610 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f6493184b80 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed02905920 0x55ed02907d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f6493184b80 4 client.25610 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed02836e10 con 0x55ed02905920 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.853+0000 7f647ffff700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed02905920 0x55ed02907d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.854+0000 7f645ffff700 10 client.25610 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:24.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:24.854+0000 7f647ffff700 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed02905920 0x55ed02907d10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4932958e0d8984 server_cookie=11645799dfd57402 in_seq=0 out_seq=0 2023-09-23T09:32:25.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:25 smithi098 ceph-mon[102386]: pgmap v93: 97 pgs: 97 active+clean; 4.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 MiB/s wr, 2.49k op/s 2023-09-23T09:32:25.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:25 smithi045 ceph-mon[98627]: pgmap v93: 97 pgs: 97 active+clean; 4.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 MiB/s wr, 2.49k op/s 2023-09-23T09:32:25.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:25 smithi123 ceph-mon[103577]: pgmap v93: 97 pgs: 97 active+clean; 4.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 MiB/s wr, 2.49k op/s 2023-09-23T09:32:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 160.0 s, 2947.7 tps, lat 33.757 ms stddev 36.824 2023-09-23T09:32:27.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:27 smithi098 ceph-mon[102386]: pgmap v94: 97 pgs: 97 active+clean; 4.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2023-09-23T09:32:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:27 smithi045 ceph-mon[98627]: pgmap v94: 97 pgs: 97 active+clean; 4.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2023-09-23T09:32:27.849 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:27 smithi123 ceph-mon[103577]: pgmap v94: 97 pgs: 97 active+clean; 4.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2023-09-23T09:32:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.962+0000 7f97eb7fe700 1 -- 172.21.15.45:0/1595785398 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56205151cad0 con 0x5620515fc180 2023-09-23T09:32:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.962+0000 7f97eb7fe700 10 client.25606 handle_command_reply: tid=0 2023-09-23T09:32:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.962+0000 7f645ffff700 1 -- 172.21.15.45:0/179887415 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed02836e10 con 0x55ed02905920 2023-09-23T09:32:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.962+0000 7f645ffff700 10 client.25610 handle_command_reply: tid=0 2023-09-23T09:32:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.962+0000 7f981e52cb80 1 client.25606 shutdown 2023-09-23T09:32:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.962+0000 7f6493184b80 1 client.25610 shutdown 2023-09-23T09:32:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 msgr2=0x5620515f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515f18e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0204a0 tx=0x7f97fc009790 comp rx=0 tx=0).stop 2023-09-23T09:32:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 shutdown_connections 2023-09-23T09:32:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620515fc180 0x562051653370 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ea400 msgr2=0x55ed028f2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ea400 0x55ed028f2c30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f646c000c00 tx=0x7f646c09acf0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620515ec990 0x5620515ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620515d8750 0x5620515f18e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 --2- 172.21.15.45:0/1595785398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620515d9070 0x5620515edcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f97eb7fe700 0 client.25606 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 >> 172.21.15.45:0/1595785398 conn(0x562051545200 msgr2=0x5620515ecd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 shutdown_connections 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed02905920 0x55ed02907d10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 0x55ed02901730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028ea400 0x55ed028f2c30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 --2- 172.21.15.45:0/179887415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f5bd0 0x55ed028f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 >> 172.21.15.45:0/179887415 conn(0x55ed0285f260 msgr2=0x55ed028e6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f645ffff700 0 client.25610 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 shutdown_connections 2023-09-23T09:32:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 shutdown_connections 2023-09-23T09:32:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 1 -- 172.21.15.45:0/179887415 wait complete. 2023-09-23T09:32:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 1 -- 172.21.15.45:0/1595785398 wait complete. 2023-09-23T09:32:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f6493184b80 20 client.25610 trim_cache size 0 max 16384 2023-09-23T09:32:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.963+0000 7f981e52cb80 20 client.25606 trim_cache size 0 max 16384 2023-09-23T09:32:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:32:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:32:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:32:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:32:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f97f80a1860 msgr2=0x7f97f80a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f97f80a1860 0x7f97f80a3d20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97fc013730 tx=0x7f97fc013af0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c100690 msgr2=0x7f980c199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c100690 0x7f980c199500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f980400efc0 tx=0x7f980400c3a0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 shutdown_connections 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f97f80a1860 0x7f97f80a3d20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f980c105d90 0x7f980c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f980c0ffcd0 0x7f980c198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 --2- 172.21.15.45:0/2074898240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f980c100690 0x7f980c199500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 >> 172.21.15.45:0/2074898240 conn(0x7f980c076e20 msgr2=0x7f980c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 shutdown_connections 2023-09-23T09:32:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:28.964+0000 7f9812342700 1 -- 172.21.15.45:0/2074898240 wait complete. 2023-09-23T09:32:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.988+0000 7f6493184b80 1 Processor -- start 2023-09-23T09:32:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.988+0000 7f6493184b80 1 -- start start 2023-09-23T09:32:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6493184b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 0x55ed029016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6493184b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 0x55ed028fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6493184b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed028efea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6493184b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed026be920 con 0x55ed028ea400 2023-09-23T09:32:28.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6493184b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed028f4170 con 0x55ed028f5bd0 2023-09-23T09:32:28.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6493184b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed028fcc00 con 0x55ed028f36b0 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6485537700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed028efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6485537700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed028efea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55292/0 (socket says 172.21.15.45:55292) 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.989+0000 7f6485537700 1 -- 172.21.15.45:0/3037188991 learned_addr learned my addr 172.21.15.45:0/3037188991 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.990+0000 7f6485537700 1 -- 172.21.15.45:0/3037188991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 msgr2=0x55ed028fc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.990+0000 7f6485537700 1 --2- 172.21.15.45:0/3037188991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 0x55ed028fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.990+0000 7f6485537700 1 -- 172.21.15.45:0/3037188991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 msgr2=0x55ed029016a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.990+0000 7f6485537700 1 --2- 172.21.15.45:0/3037188991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 0x55ed029016a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.990+0000 7f6485537700 1 -- 172.21.15.45:0/3037188991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed02651c60 con 0x55ed028f5bd0 2023-09-23T09:32:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.990+0000 7f6485537700 1 --2- 172.21.15.45:0/3037188991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed028efea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6478009da0 tx=0x7f6478009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed8cfb58cf62f888 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f646b7fe700 1 -- 172.21.15.45:0/3037188991 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478007e70 con 0x55ed028f5bd0 2023-09-23T09:32:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f646b7fe700 1 -- 172.21.15.45:0/3037188991 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f647800f030 con 0x55ed028f5bd0 2023-09-23T09:32:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f646b7fe700 1 -- 172.21.15.45:0/3037188991 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478014640 con 0x55ed028f5bd0 2023-09-23T09:32:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f6493184b80 1 -- 172.21.15.45:0/3037188991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 msgr2=0x55ed028efea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f6493184b80 1 --2- 172.21.15.45:0/3037188991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed028efea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6478009da0 tx=0x7f6478009040 comp rx=0 tx=0).stop 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f6493184b80 1 -- 172.21.15.45:0/3037188991 shutdown_connections 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f6493184b80 1 --2- 172.21.15.45:0/3037188991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed028efea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f6493184b80 1 --2- 172.21.15.45:0/3037188991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028ea400 0x55ed029016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f6493184b80 1 --2- 172.21.15.45:0/3037188991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028f36b0 0x55ed028fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.991+0000 7f6493184b80 1 -- 172.21.15.45:0/3037188991 >> 172.21.15.45:0/3037188991 conn(0x55ed0285f260 msgr2=0x55ed02902080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.992+0000 7f6493184b80 1 -- 172.21.15.45:0/3037188991 shutdown_connections 2023-09-23T09:32:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.992+0000 7f6493184b80 1 -- 172.21.15.45:0/3037188991 wait complete. 2023-09-23T09:32:28.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.992+0000 7f6493184b80 1 -- start start 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 1 Processor -- start 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028ea400 0x55ed0294d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f36b0 0x55ed0294de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed02951be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed02952190 con 0x55ed028f36b0 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed02952300 con 0x55ed028f5bd0 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed029524c0 con 0x55ed028ea400 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f6493184b80 10 client.0 _ll_register_callbacks cb 0x7f646c097780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f647ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f36b0 0x55ed0294de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.993+0000 7f647ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f36b0 0x55ed0294de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33946/0 (socket says 172.21.15.45:33946) 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f647ffff700 1 -- 172.21.15.45:0/3865694619 learned_addr learned my addr 172.21.15.45:0/3865694619 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f6485537700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed02951be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f6484d36700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028ea400 0x55ed0294d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f647ffff700 1 -- 172.21.15.45:0/3865694619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028ea400 msgr2=0x55ed0294d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f647ffff700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028ea400 0x55ed0294d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f647ffff700 1 -- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 msgr2=0x55ed02951be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f647ffff700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed02951be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f647ffff700 1 -- 172.21.15.45:0/3865694619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64780099d0 con 0x55ed028f36b0 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f6485537700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed02951be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f647ffff700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f36b0 0x55ed0294de00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55ed028e72a0 tx=0x7f6474005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba417ba95950211 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.994+0000 7f645ffff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:32:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.995+0000 7f645ffff700 1 -- 172.21.15.45:0/3865694619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647402e070 con 0x55ed028f36b0 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.995+0000 7f645ffff700 1 -- 172.21.15.45:0/3865694619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6474010c50 con 0x55ed028f36b0 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.995+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6474010460 con 0x55ed028f36b0 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.996+0000 7f645ffff700 1 -- 172.21.15.45:0/3865694619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f647400b690 con 0x55ed028f36b0 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.996+0000 7f6493184b80 10 client.15948 fetch_fsmap learned FSMap version 29 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.996+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ed0268f250 con 0x55ed028f36b0 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f645ffff700 1 -- 172.21.15.45:0/3865694619 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f647403b030 con 0x55ed028f36b0 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f645ffff700 1 -- 172.21.15.45:0/3865694619 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6474025070 con 0x55ed028f36b0 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f6493184b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f6493184b80 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f6493184b80 20 client.15948 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f6493184b80 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed0295afa0 0x55ed0295d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f6493184b80 4 client.15948 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:32:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ed02836b30 con 0x55ed0295afa0 2023-09-23T09:32:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.997+0000 7f6484d36700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed0295afa0 0x55ed0295d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.998+0000 7f645ffff700 10 client.15948 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.998+0000 7f6484d36700 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed0295afa0 0x55ed0295d3c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa3b49e02bf50f35 server_cookie=e29fd9614e7533a2 in_seq=0 out_seq=0 2023-09-23T09:32:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f645ffff700 1 -- 172.21.15.45:0/3865694619 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ed02836b30 con 0x55ed0295afa0 2023-09-23T09:32:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f645ffff700 10 client.15948 handle_command_reply: tid=0 2023-09-23T09:32:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 client.15948 shutdown 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f36b0 msgr2=0x55ed0294de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f36b0 0x55ed0294de00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55ed028e72a0 tx=0x7f6474005860 comp rx=0 tx=0).stop 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 shutdown_connections 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ed0295afa0 0x55ed0295d3c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed028f5bd0 0x55ed02951be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f645ffff700 0 client.15948 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed028ea400 0x55ed0294d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 --2- 172.21.15.45:0/3865694619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed028f36b0 0x55ed0294de00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 >> 172.21.15.45:0/3865694619 conn(0x55ed0285f260 msgr2=0x55ed028e59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 shutdown_connections 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 1 -- 172.21.15.45:0/3865694619 wait complete. 2023-09-23T09:32:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:28.999+0000 7f6493184b80 20 client.15948 trim_cache size 0 max 16384 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:32:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:32:29.004 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:32:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f64700a1790 msgr2=0x7f64700a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f64700a1790 0x7f64700a3c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6474003d60 tx=0x7f6474010000 comp rx=0 tx=0).stop 2023-09-23T09:32:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 msgr2=0x7f64801a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f64801a5f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f646c010fe0 tx=0x7f646c005e40 comp rx=0 tx=0).stop 2023-09-23T09:32:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 shutdown_connections 2023-09-23T09:32:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64801aa8a0 0x7f64801aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f64700a1790 0x7f64700a3c50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648010f4d0 0x7f64801a5f00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 --2- 172.21.15.45:0/1236155591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f648011a600 0x7f64801a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.000+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 >> 172.21.15.45:0/1236155591 conn(0x7f648006d0e0 msgr2=0x7f64801185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.002+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 shutdown_connections 2023-09-23T09:32:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:29.002+0000 7f6486f9a700 1 -- 172.21.15.45:0/1236155591 wait complete. 2023-09-23T09:32:29.021 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:32:29.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:29 smithi098 ceph-mon[102386]: pgmap v95: 97 pgs: 97 active+clean; 4.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 46 MiB/s wr, 2.47k op/s 2023-09-23T09:32:29.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:29 smithi045 ceph-mon[98627]: pgmap v95: 97 pgs: 97 active+clean; 4.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 46 MiB/s wr, 2.47k op/s 2023-09-23T09:32:29.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:29 smithi123 ceph-mon[103577]: pgmap v95: 97 pgs: 97 active+clean; 4.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 46 MiB/s wr, 2.47k op/s 2023-09-23T09:32:31.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:31 smithi098 ceph-mon[102386]: pgmap v96: 97 pgs: 97 active+clean; 4.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 852 op/s 2023-09-23T09:32:31.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:31 smithi045 ceph-mon[98627]: pgmap v96: 97 pgs: 97 active+clean; 4.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 852 op/s 2023-09-23T09:32:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:31 smithi123 ceph-mon[103577]: pgmap v96: 97 pgs: 97 active+clean; 4.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 852 op/s 2023-09-23T09:32:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 165.0 s, 2959.4 tps, lat 33.399 ms stddev 35.813 2023-09-23T09:32:33.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:33 smithi098 ceph-mon[102386]: pgmap v97: 97 pgs: 97 active+clean; 4.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 883 op/s 2023-09-23T09:32:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:33 smithi045 ceph-mon[98627]: pgmap v97: 97 pgs: 97 active+clean; 4.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 883 op/s 2023-09-23T09:32:33.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:33 smithi123 ceph-mon[103577]: pgmap v97: 97 pgs: 97 active+clean; 4.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 883 op/s 2023-09-23T09:32:34.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:32:34.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:34 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:32:34.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:34 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:32:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:34.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:34 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:35 smithi045 ceph-mon[98627]: pgmap v98: 97 pgs: 97 active+clean; 4.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 25 MiB/s wr, 1.13k op/s 2023-09-23T09:32:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:35 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:35 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:35.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:35 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:32:35.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:35 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:32:35.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:35 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:35 smithi123 ceph-mon[103577]: pgmap v98: 97 pgs: 97 active+clean; 4.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 25 MiB/s wr, 1.13k op/s 2023-09-23T09:32:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:35 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:35 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:35.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:35 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:32:35.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:35 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:32:35.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:35 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:36.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:35 smithi098 ceph-mon[102386]: pgmap v98: 97 pgs: 97 active+clean; 4.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 25 MiB/s wr, 1.13k op/s 2023-09-23T09:32:36.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:35 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:36.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:35 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:36.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:35 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:32:36.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:35 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:32:36.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:35 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:32:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 170.0 s, 1692.2 tps, lat 60.244 ms stddev 67.435 2023-09-23T09:32:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:37 smithi045 ceph-mon[98627]: pgmap v99: 97 pgs: 97 active+clean; 4.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 781 op/s 2023-09-23T09:32:37.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:37 smithi123 ceph-mon[103577]: pgmap v99: 97 pgs: 97 active+clean; 4.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 781 op/s 2023-09-23T09:32:38.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:37 smithi098 ceph-mon[102386]: pgmap v99: 97 pgs: 97 active+clean; 4.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 781 op/s 2023-09-23T09:32:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:39 smithi045 ceph-mon[98627]: pgmap v100: 97 pgs: 97 active+clean; 4.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.30k op/s 2023-09-23T09:32:39.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:39 smithi123 ceph-mon[103577]: pgmap v100: 97 pgs: 97 active+clean; 4.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.30k op/s 2023-09-23T09:32:40.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:39 smithi098 ceph-mon[102386]: pgmap v100: 97 pgs: 97 active+clean; 4.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.30k op/s 2023-09-23T09:32:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:41 smithi123 ceph-mon[103577]: pgmap v101: 97 pgs: 97 active+clean; 4.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 849 op/s 2023-09-23T09:32:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:41 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:32:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 175.0 s, 1603.4 tps, lat 60.124 ms stddev 81.987 2023-09-23T09:32:42.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:41 smithi098 ceph-mon[102386]: pgmap v101: 97 pgs: 97 active+clean; 4.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 849 op/s 2023-09-23T09:32:42.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:41 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:32:42.087 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:41 smithi045 ceph-mon[98627]: pgmap v101: 97 pgs: 97 active+clean; 4.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 849 op/s 2023-09-23T09:32:42.087 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:41 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:32:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.227+0000 7f86931c2700 1 -- 172.21.15.45:0/3448687530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c0ff220 msgr2=0x7f868c0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.227+0000 7f86931c2700 1 --2- 172.21.15.45:0/3448687530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c0ff220 0x7f868c0ff6a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f867c009990 tx=0x7f867c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.227+0000 7f86931c2700 1 -- 172.21.15.45:0/3448687530 shutdown_connections 2023-09-23T09:32:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.227+0000 7f86931c2700 1 --2- 172.21.15.45:0/3448687530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f868c0ffbe0 0x7f868c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.227+0000 7f86931c2700 1 --2- 172.21.15.45:0/3448687530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c105cf0 0x7f868c1060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.227+0000 7f86931c2700 1 --2- 172.21.15.45:0/3448687530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c0ff220 0x7f868c0ff6a0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f867c009990 tx=0x7f867c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.227+0000 7f86931c2700 1 -- 172.21.15.45:0/3448687530 >> 172.21.15.45:0/3448687530 conn(0x7f868c0771f0 msgr2=0x7f868c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.232+0000 7f86931c2700 1 -- 172.21.15.45:0/3448687530 shutdown_connections 2023-09-23T09:32:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.237+0000 7f86931c2700 1 -- 172.21.15.45:0/3448687530 wait complete. 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 Processor -- start 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 -- start start 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c0ffbe0 0x7f868c10acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f868c105cf0 0x7f868c10b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c071460 0x7f868c0718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c071f40 con 0x7f868c0ffbe0 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0720b0 con 0x7f868c105cf0 2023-09-23T09:32:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.238+0000 7f86931c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c072270 con 0x7f868c071460 2023-09-23T09:32:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c0ffbe0 0x7f868c10acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c0ffbe0 0x7f868c10acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33958/0 (socket says 172.21.15.45:33958) 2023-09-23T09:32:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 -- 172.21.15.45:0/1967475518 learned_addr learned my addr 172.21.15.45:0/1967475518 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f869175f700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c071460 0x7f868c0718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 -- 172.21.15.45:0/1967475518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c071460 msgr2=0x7f868c0718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c071460 0x7f868c0718e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 -- 172.21.15.45:0/1967475518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f868c105cf0 msgr2=0x7f868c10b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f868c105cf0 0x7f868c10b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 -- 172.21.15.45:0/1967475518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c0095f0 con 0x7f868c0ffbe0 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.240+0000 7f8690f5e700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c0ffbe0 0x7f868c10acc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f868000b530 tx=0x7f868000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.241+0000 7f8689ffb700 1 -- 172.21.15.45:0/1967475518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680011830 con 0x7f868c0ffbe0 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.241+0000 7f8689ffb700 1 -- 172.21.15.45:0/1967475518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8680011e50 con 0x7f868c0ffbe0 2023-09-23T09:32:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.241+0000 7f8689ffb700 1 -- 172.21.15.45:0/1967475518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000f2e0 con 0x7f868c0ffbe0 2023-09-23T09:32:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.242+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c112d00 con 0x7f868c0ffbe0 2023-09-23T09:32:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.242+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c113190 con 0x7f868c0ffbe0 2023-09-23T09:32:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.247+0000 7f869f3acb80 1 Processor -- start 2023-09-23T09:32:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.247+0000 7f869f3acb80 1 -- start start 2023-09-23T09:32:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.248+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c935fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.248+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c936061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.248+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93606710 0x556c93608b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.248+0000 7f869f3acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c93564400 con 0x556c93606710 2023-09-23T09:32:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.248+0000 7f869f3acb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c93555bc0 con 0x556c935fb860 2023-09-23T09:32:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.248+0000 7f869f3acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c935534e0 con 0x556c935faf40 2023-09-23T09:32:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.253+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c936061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.253+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c936061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55314/0 (socket says 172.21.15.45:55314) 2023-09-23T09:32:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.253+0000 7f8690f5e700 1 -- 172.21.15.45:0/2185933632 learned_addr learned my addr 172.21.15.45:0/2185933632 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.253+0000 7f869175f700 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c935fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.254+0000 7f868bfff700 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93606710 0x556c93608b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.254+0000 7f868bfff700 1 -- 172.21.15.45:0/2185933632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 msgr2=0x556c935fb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.254+0000 7f868bfff700 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c935fb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.254+0000 7f868bfff700 1 -- 172.21.15.45:0/2185933632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 msgr2=0x556c936061d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.254+0000 7f868bfff700 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c936061d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.254+0000 7f868bfff700 1 -- 172.21.15.45:0/2185933632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c935b1870 con 0x556c93606710 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.255+0000 7f868bfff700 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93606710 0x556c93608b00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f867c008000 tx=0x7f867c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3282fe55a4d5dd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.255+0000 7f8689ffb700 1 -- 172.21.15.45:0/1967475518 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f8680010580 con 0x7f868c0ffbe0 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.255+0000 7f8672ffd700 1 -- 172.21.15.45:0/2185933632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c01d070 con 0x556c93606710 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.255+0000 7f8672ffd700 1 -- 172.21.15.45:0/2185933632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f867c00be00 con 0x556c93606710 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.255+0000 7f8672ffd700 1 -- 172.21.15.45:0/2185933632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c005320 con 0x556c93606710 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.256+0000 7f8689ffb700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86740a1810 0x7f86740a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.256+0000 7f8689ffb700 1 -- 172.21.15.45:0/1967475518 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86800ceed0 con 0x7f868c0ffbe0 2023-09-23T09:32:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 -- 172.21.15.45:0/2185933632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93606710 msgr2=0x556c93608b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93606710 0x556c93608b00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f867c008000 tx=0x7f867c003680 comp rx=0 tx=0).stop 2023-09-23T09:32:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 -- 172.21.15.45:0/2185933632 shutdown_connections 2023-09-23T09:32:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93606710 0x556c93608b00 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f867c008000 tx=0x7f867c003680 comp rx=0 tx=0).stop 2023-09-23T09:32:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c935fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2185933632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c936061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 -- 172.21.15.45:0/2185933632 >> 172.21.15.45:0/2185933632 conn(0x556c93570200 msgr2=0x556c935f73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.258+0000 7f869f3acb80 1 -- 172.21.15.45:0/2185933632 shutdown_connections 2023-09-23T09:32:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.261+0000 7f869175f700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86740a1810 0x7f86740a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.261+0000 7f869f3acb80 1 -- 172.21.15.45:0/2185933632 wait complete. 2023-09-23T09:32:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.262+0000 7f869175f700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86740a1810 0x7f86740a3cd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8684009c90 tx=0x7f868400b040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.262+0000 7f869f3acb80 1 -- start start 2023-09-23T09:32:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.262+0000 7f869f3acb80 1 Processor -- start 2023-09-23T09:32:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.263+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935faf40 0x556c93603540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.263+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c935fb860 0x556c93603a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.263+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c93603fc0 0x556c936122b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.263+0000 7f869f3acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9354edd0 con 0x556c935fb860 2023-09-23T09:32:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.263+0000 7f869f3acb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c934eb6b0 con 0x556c935faf40 2023-09-23T09:32:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.263+0000 7f869f3acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c934eb350 con 0x556c93603fc0 2023-09-23T09:32:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.263+0000 7f869f3acb80 10 client.0 _ll_register_callbacks cb 0x556c933487a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935faf40 0x556c93603540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935faf40 0x556c93603540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55326/0 (socket says 172.21.15.45:55326) 2023-09-23T09:32:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 -- 172.21.15.45:0/475570723 learned_addr learned my addr 172.21.15.45:0/475570723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 -- 172.21.15.45:0/475570723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c93603fc0 msgr2=0x556c936122b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c93603fc0 0x556c936122b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 -- 172.21.15.45:0/475570723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c935fb860 msgr2=0x556c93603a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c935fb860 0x556c93603a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.266+0000 7f8690f5e700 1 -- 172.21.15.45:0/475570723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c105410 con 0x556c935faf40 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.267+0000 7f8690f5e700 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935faf40 0x556c93603540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f868000bcf0 tx=0x7f868009ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2bb12fb64cdfa51 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.267+0000 7f866bfff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.268+0000 7f866bfff700 1 -- 172.21.15.45:0/475570723 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680010380 con 0x556c935faf40 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.268+0000 7f866bfff700 1 -- 172.21.15.45:0/475570723 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f868009c5e0 con 0x556c935faf40 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.268+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c935ef090 con 0x556c935faf40 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.268+0000 7f866bfff700 1 -- 172.21.15.45:0/475570723 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f868009bee0 con 0x556c935faf40 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.268+0000 7f869f3acb80 10 client.25625 fetch_fsmap learned FSMap version 30 2023-09-23T09:32:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.268+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556c93379d80 con 0x556c935faf40 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.275+0000 7f866bfff700 1 -- 172.21.15.45:0/475570723 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86800ba030 con 0x556c935faf40 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.275+0000 7f866bfff700 1 -- 172.21.15.45:0/475570723 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f868009f020 con 0x556c935faf40 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.276+0000 7f869f3acb80 10 client.25625 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.276+0000 7f869f3acb80 10 client.25625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.276+0000 7f869f3acb80 20 client.25625 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.276+0000 7f869f3acb80 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c93616700 0x556c93618bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.276+0000 7f869f3acb80 4 client.25625 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.276+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c93547db0 con 0x556c93616700 2023-09-23T09:32:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.276+0000 7f868bfff700 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c93616700 0x556c93618bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.290+0000 7f866bfff700 10 client.25625 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.290+0000 7f868bfff700 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c93616700 0x556c93618bc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fbc6c1d17f2fe1 server_cookie=4312bb7df95cf7d2 in_seq=0 out_seq=0 2023-09-23T09:32:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.292+0000 7f866bfff700 1 -- 172.21.15.45:0/475570723 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c93547db0 con 0x556c93616700 2023-09-23T09:32:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.292+0000 7f866bfff700 10 client.25625 handle_command_reply: tid=0 2023-09-23T09:32:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.292+0000 7f869f3acb80 1 client.25625 shutdown 2023-09-23T09:32:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.293+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935faf40 msgr2=0x556c93603540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.293+0000 7f869f3acb80 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935faf40 0x556c93603540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f868000bcf0 tx=0x7f868009ba20 comp rx=0 tx=0).stop 2023-09-23T09:32:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.304+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 shutdown_connections 2023-09-23T09:32:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.304+0000 7f869f3acb80 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c93616700 0x556c93618bc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.304+0000 7f869f3acb80 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c93603fc0 0x556c936122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.304+0000 7f869f3acb80 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935faf40 0x556c93603540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.304+0000 7f869f3acb80 1 --2- 172.21.15.45:0/475570723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c935fb860 0x556c93603a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.304+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 >> 172.21.15.45:0/475570723 conn(0x556c93570200 msgr2=0x556c935f6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.305+0000 7f866bfff700 0 client.25625 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.310+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 shutdown_connections 2023-09-23T09:32:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.316+0000 7f869f3acb80 1 -- 172.21.15.45:0/475570723 wait complete. 2023-09-23T09:32:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.317+0000 7f869f3acb80 20 client.25625 trim_cache size 0 max 16384 2023-09-23T09:32:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.345+0000 7f869f3acb80 1 Processor -- start 2023-09-23T09:32:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.357+0000 7f869f3acb80 1 -- start start 2023-09-23T09:32:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.357+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c93602d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.357+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c93611950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.357+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c93611e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.357+0000 7f869f3acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c933cf8c0 con 0x556c93603fc0 2023-09-23T09:32:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.357+0000 7f869f3acb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c93604560 con 0x556c935fb860 2023-09-23T09:32:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.357+0000 7f869f3acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c935f8150 con 0x556c935faf40 2023-09-23T09:32:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.359+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c93602d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.359+0000 7f8690f5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c93602d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36086/0 (socket says 172.21.15.45:36086) 2023-09-23T09:32:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.359+0000 7f8690f5e700 1 -- 172.21.15.45:0/2059749474 learned_addr learned my addr 172.21.15.45:0/2059749474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.363+0000 7f868bfff700 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c93611950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.363+0000 7f869175f700 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c93611e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.370+0000 7f869175f700 1 -- 172.21.15.45:0/2059749474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 msgr2=0x556c93602d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.370+0000 7f869175f700 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c93602d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.370+0000 7f869175f700 1 -- 172.21.15.45:0/2059749474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 msgr2=0x556c93611950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.370+0000 7f869175f700 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c93611950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.370+0000 7f869175f700 1 -- 172.21.15.45:0/2059749474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c93362c00 con 0x556c93603fc0 2023-09-23T09:32:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.385+0000 7f869175f700 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c93611e90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8684009220 tx=0x7f8684004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f2a761d9cbefe1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.390+0000 7f86737fe700 1 -- 172.21.15.45:0/2059749474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684017af0 con 0x556c93603fc0 2023-09-23T09:32:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.390+0000 7f86737fe700 1 -- 172.21.15.45:0/2059749474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8684017c50 con 0x556c93603fc0 2023-09-23T09:32:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.390+0000 7f86737fe700 1 -- 172.21.15.45:0/2059749474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684022ea0 con 0x556c93603fc0 2023-09-23T09:32:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.391+0000 7f869f3acb80 1 -- 172.21.15.45:0/2059749474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 msgr2=0x556c93611e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.391+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c93611e90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8684009220 tx=0x7f8684004ad0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.391+0000 7f869f3acb80 1 -- 172.21.15.45:0/2059749474 shutdown_connections 2023-09-23T09:32:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.391+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c93611e90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.391+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c93602d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.391+0000 7f869f3acb80 1 --2- 172.21.15.45:0/2059749474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c93611950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.391+0000 7f869f3acb80 1 -- 172.21.15.45:0/2059749474 >> 172.21.15.45:0/2059749474 conn(0x556c93570200 msgr2=0x556c936129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.397+0000 7f869f3acb80 1 -- 172.21.15.45:0/2059749474 shutdown_connections 2023-09-23T09:32:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.397+0000 7f869f3acb80 1 -- 172.21.15.45:0/2059749474 wait complete. 2023-09-23T09:32:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.398+0000 7f869f3acb80 1 -- start start 2023-09-23T09:32:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.398+0000 7f869f3acb80 1 Processor -- start 2023-09-23T09:32:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.401+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c9360d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.401+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c9360b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.401+0000 7f869f3acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c9360bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.401+0000 7f869f3acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9360c160 con 0x556c93603fc0 2023-09-23T09:32:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.401+0000 7f869f3acb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9360c2d0 con 0x556c935fb860 2023-09-23T09:32:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.401+0000 7f869f3acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9360c490 con 0x556c935faf40 2023-09-23T09:32:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.401+0000 7f869f3acb80 10 client.0 _ll_register_callbacks cb 0x7f8680098220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.402+0000 7f868bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c9360b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.402+0000 7f868bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c9360b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55340/0 (socket says 172.21.15.45:55340) 2023-09-23T09:32:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.402+0000 7f868bfff700 1 -- 172.21.15.45:0/926874508 learned_addr learned my addr 172.21.15.45:0/926874508 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.403+0000 7f8690f5e700 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c9360d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.403+0000 7f868bfff700 1 -- 172.21.15.45:0/926874508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 msgr2=0x556c9360d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.403+0000 7f868bfff700 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c9360d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.403+0000 7f868bfff700 1 -- 172.21.15.45:0/926874508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 msgr2=0x556c9360bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.403+0000 7f868bfff700 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c9360bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.403+0000 7f868bfff700 1 -- 172.21.15.45:0/926874508 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8684011040 con 0x556c935fb860 2023-09-23T09:32:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.403+0000 7f868bfff700 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c9360b650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f867c003ca0 tx=0x7f867c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19c4afdb19018bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.404+0000 7f866bfff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.404+0000 7f866bfff700 1 -- 172.21.15.45:0/926874508 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c030070 con 0x556c935fb860 2023-09-23T09:32:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.404+0000 7f866bfff700 1 -- 172.21.15.45:0/926874508 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f867c005da0 con 0x556c935fb860 2023-09-23T09:32:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.405+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f867c00be00 con 0x556c935fb860 2023-09-23T09:32:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.407+0000 7f866bfff700 1 -- 172.21.15.45:0/926874508 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f867c01d040 con 0x556c935fb860 2023-09-23T09:32:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.407+0000 7f869f3acb80 10 client.25631 fetch_fsmap learned FSMap version 30 2023-09-23T09:32:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.407+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556c933a01f0 con 0x556c935fb860 2023-09-23T09:32:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f866bfff700 1 -- 172.21.15.45:0/926874508 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f867c00be00 con 0x556c935fb860 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f866bfff700 1 -- 172.21.15.45:0/926874508 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f867c033030 con 0x556c935fb860 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f869f3acb80 10 client.25631 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f869f3acb80 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f869f3acb80 20 client.25631 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f869f3acb80 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c9366b740 0x556c9366db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f869f3acb80 4 client.25631 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556c93547ad0 con 0x556c9366b740 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.412+0000 7f8690f5e700 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c9366b740 0x556c9366db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.413+0000 7f8690f5e700 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c9366b740 0x556c9366db30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f4ea65b8be5a05 server_cookie=ec093b73f41b4fc8 in_seq=0 out_seq=0 2023-09-23T09:32:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.413+0000 7f866bfff700 10 client.25631 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.589+0000 7f866bfff700 1 -- 172.21.15.45:0/926874508 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556c93547ad0 con 0x556c9366b740 2023-09-23T09:32:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.589+0000 7f866bfff700 10 client.25631 handle_command_reply: tid=0 2023-09-23T09:32:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.590+0000 7f869f3acb80 1 client.25631 shutdown 2023-09-23T09:32:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.590+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 msgr2=0x556c9360b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.590+0000 7f869f3acb80 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c9360b650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f867c003ca0 tx=0x7f867c003b90 comp rx=0 tx=0).stop 2023-09-23T09:32:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 shutdown_connections 2023-09-23T09:32:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556c9366b740 0x556c9366db30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c93603fc0 0x556c9360bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c935faf40 0x556c9360d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 --2- 172.21.15.45:0/926874508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556c935fb860 0x556c9360b650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 >> 172.21.15.45:0/926874508 conn(0x556c93570200 msgr2=0x556c93611950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f866bfff700 0 client.25631 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 shutdown_connections 2023-09-23T09:32:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 1 -- 172.21.15.45:0/926874508 wait complete. 2023-09-23T09:32:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:32:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:32:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:32:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:32:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.594+0000 7f869f3acb80 20 client.25631 trim_cache size 0 max 16384 2023-09-23T09:32:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.601+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86740a1810 msgr2=0x7f86740a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.601+0000 7f86931c2700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86740a1810 0x7f86740a3cd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8684009c90 tx=0x7f868400b040 comp rx=0 tx=0).stop 2023-09-23T09:32:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.601+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c0ffbe0 msgr2=0x7f868c10acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.601+0000 7f86931c2700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c0ffbe0 0x7f868c10acc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f868000b530 tx=0x7f868000b8f0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.603+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 shutdown_connections 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.603+0000 7f86931c2700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f868c071460 0x7f868c0718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.603+0000 7f86931c2700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f86740a1810 0x7f86740a3cd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.603+0000 7f86931c2700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c0ffbe0 0x7f868c10acc0 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f868000b530 tx=0x7f868000b8f0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.603+0000 7f86931c2700 1 --2- 172.21.15.45:0/1967475518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f868c105cf0 0x7f868c10b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.603+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 >> 172.21.15.45:0/1967475518 conn(0x7f868c0771f0 msgr2=0x7f868c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.604+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 shutdown_connections 2023-09-23T09:32:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:42.604+0000 7f86931c2700 1 -- 172.21.15.45:0/1967475518 wait complete. 2023-09-23T09:32:43.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:42 smithi098 ceph-mon[102386]: pgmap v102: 97 pgs: 97 active+clean; 4.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 895 op/s 2023-09-23T09:32:43.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:42 smithi045 ceph-mon[98627]: pgmap v102: 97 pgs: 97 active+clean; 4.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 895 op/s 2023-09-23T09:32:43.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:42 smithi123 ceph-mon[103577]: pgmap v102: 97 pgs: 97 active+clean; 4.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 895 op/s 2023-09-23T09:32:45.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:44 smithi098 ceph-mon[102386]: pgmap v103: 97 pgs: 97 active+clean; 4.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 MiB/s wr, 1.39k op/s 2023-09-23T09:32:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:44 smithi045 ceph-mon[98627]: pgmap v103: 97 pgs: 97 active+clean; 4.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 MiB/s wr, 1.39k op/s 2023-09-23T09:32:45.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:44 smithi123 ceph-mon[103577]: pgmap v103: 97 pgs: 97 active+clean; 4.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 MiB/s wr, 1.39k op/s 2023-09-23T09:32:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 180.0 s, 1999.0 tps, lat 51.880 ms stddev 110.287 2023-09-23T09:32:47.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:47 smithi098 ceph-mon[102386]: pgmap v104: 97 pgs: 97 active+clean; 4.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 1.09k op/s 2023-09-23T09:32:47.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:47 smithi045 ceph-mon[98627]: pgmap v104: 97 pgs: 97 active+clean; 4.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 1.09k op/s 2023-09-23T09:32:47.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:47 smithi123 ceph-mon[103577]: pgmap v104: 97 pgs: 97 active+clean; 4.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 1.09k op/s 2023-09-23T09:32:49.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:49 smithi098 ceph-mon[102386]: pgmap v105: 97 pgs: 97 active+clean; 5.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 56 MiB/s wr, 2.81k op/s 2023-09-23T09:32:49.829 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:49 smithi045 ceph-mon[98627]: pgmap v105: 97 pgs: 97 active+clean; 5.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 56 MiB/s wr, 2.81k op/s 2023-09-23T09:32:49.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:49 smithi123 ceph-mon[103577]: pgmap v105: 97 pgs: 97 active+clean; 5.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 56 MiB/s wr, 2.81k op/s 2023-09-23T09:32:51.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:51 smithi098 ceph-mon[102386]: pgmap v106: 97 pgs: 97 active+clean; 5.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 43 MiB/s wr, 2.29k op/s 2023-09-23T09:32:51.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:51 smithi045 ceph-mon[98627]: pgmap v106: 97 pgs: 97 active+clean; 5.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 43 MiB/s wr, 2.29k op/s 2023-09-23T09:32:51.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:51 smithi123 ceph-mon[103577]: pgmap v106: 97 pgs: 97 active+clean; 5.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 43 MiB/s wr, 2.29k op/s 2023-09-23T09:32:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 185.0 s, 2736.6 tps, lat 36.507 ms stddev 44.417 2023-09-23T09:32:53.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:53 smithi098 ceph-mon[102386]: pgmap v107: 97 pgs: 97 active+clean; 5.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 44 MiB/s wr, 2.35k op/s 2023-09-23T09:32:53.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:53 smithi045 ceph-mon[98627]: pgmap v107: 97 pgs: 97 active+clean; 5.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 44 MiB/s wr, 2.35k op/s 2023-09-23T09:32:53.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:53 smithi123 ceph-mon[103577]: pgmap v107: 97 pgs: 97 active+clean; 5.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 44 MiB/s wr, 2.35k op/s 2023-09-23T09:32:55.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:55 smithi098 ceph-mon[102386]: pgmap v108: 97 pgs: 97 active+clean; 5.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 53 MiB/s wr, 2.80k op/s 2023-09-23T09:32:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.813+0000 7f30a4d89700 1 -- 172.21.15.45:0/574710396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f570 msgr2=0x7f30a0107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.813+0000 7f30a4d89700 1 --2- 172.21.15.45:0/574710396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f570 0x7f30a0107e20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3094009990 tx=0x7f3094009ca0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.816+0000 7f30a4d89700 1 -- 172.21.15.45:0/574710396 shutdown_connections 2023-09-23T09:32:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.816+0000 7f30a4d89700 1 --2- 172.21.15.45:0/574710396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30a011a600 0x7f30a011c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.816+0000 7f30a4d89700 1 --2- 172.21.15.45:0/574710396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f570 0x7f30a0107e20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.816+0000 7f30a4d89700 1 --2- 172.21.15.45:0/574710396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30a010f9e0 0x7f30a0108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.816+0000 7f30a4d89700 1 -- 172.21.15.45:0/574710396 >> 172.21.15.45:0/574710396 conn(0x7f30a006cee0 msgr2=0x7f30a006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.816+0000 7f30a4d89700 1 -- 172.21.15.45:0/574710396 shutdown_connections 2023-09-23T09:32:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.816+0000 7f30a4d89700 1 -- 172.21.15.45:0/574710396 wait complete. 2023-09-23T09:32:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 Processor -- start 2023-09-23T09:32:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 -- start start 2023-09-23T09:32:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30a010f570 0x7f30a0119990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f9e0 0x7f30a0119ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30a011a600 0x7f30a01137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a0113f70 con 0x7f30a011a600 2023-09-23T09:32:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a01140e0 con 0x7f30a010f570 2023-09-23T09:32:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.817+0000 7f30a4d89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a01142a0 con 0x7f30a010f9e0 2023-09-23T09:32:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30a010f570 0x7f30a0119990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30a010f570 0x7f30a0119990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55350/0 (socket says 172.21.15.45:55350) 2023-09-23T09:32:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309e59c700 1 -- 172.21.15.45:0/1394098886 learned_addr learned my addr 172.21.15.45:0/1394098886 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309dd9b700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f9e0 0x7f30a0119ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309ed9d700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30a011a600 0x7f30a01137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309dd9b700 1 -- 172.21.15.45:0/1394098886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30a010f570 msgr2=0x7f30a0119990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309dd9b700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30a010f570 0x7f30a0119990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309dd9b700 1 -- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30a011a600 msgr2=0x7f30a01137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309dd9b700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30a011a600 0x7f30a01137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309dd9b700 1 -- 172.21.15.45:0/1394098886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30940095f0 con 0x7f30a010f9e0 2023-09-23T09:32:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f309dd9b700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f9e0 0x7f30a0119ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f309800bd40 tx=0x7f309800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.818+0000 7f308f7fe700 1 -- 172.21.15.45:0/1394098886 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800cd10 con 0x7f30a010f9e0 2023-09-23T09:32:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.819+0000 7f308f7fe700 1 -- 172.21.15.45:0/1394098886 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309800ce70 con 0x7f30a010f9e0 2023-09-23T09:32:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.819+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a0114580 con 0x7f30a010f9e0 2023-09-23T09:32:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.819+0000 7f308f7fe700 1 -- 172.21.15.45:0/1394098886 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980127d0 con 0x7f30a010f9e0 2023-09-23T09:32:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.819+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a01bb550 con 0x7f30a010f9e0 2023-09-23T09:32:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.823+0000 7f30b0f73b80 1 Processor -- start 2023-09-23T09:32:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f30b0f73b80 1 -- start start 2023-09-23T09:32:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f308f7fe700 1 -- 172.21.15.45:0/1394098886 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f3098014ae0 con 0x7f30a010f9e0 2023-09-23T09:32:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7248320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7255b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc71b1400 con 0x561cc7248860 2023-09-23T09:32:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc71a2bc0 con 0x561cc7247f40 2023-09-23T09:32:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc71a04e0 con 0x561cc7253710 2023-09-23T09:32:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f309ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7248320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f309e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f309ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7248320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55356/0 (socket says 172.21.15.45:55356) 2023-09-23T09:32:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f309e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34008/0 (socket says 172.21.15.45:34008) 2023-09-23T09:32:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f309ed9d700 1 -- 172.21.15.45:0/4258641588 learned_addr learned my addr 172.21.15.45:0/4258641588 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.824+0000 7f309dd9b700 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7255b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.825+0000 7f308f7fe700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f30880a17d0 0x7f30880a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.825+0000 7f308f7fe700 1 -- 172.21.15.45:0/1394098886 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3098006e40 con 0x7f30a010f9e0 2023-09-23T09:32:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.825+0000 7f309ed9d700 1 -- 172.21.15.45:0/4258641588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 msgr2=0x561cc7255b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.825+0000 7f309ed9d700 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7255b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.825+0000 7f309ed9d700 1 -- 172.21.15.45:0/4258641588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 msgr2=0x561cc72531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.825+0000 7f309ed9d700 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.826+0000 7f309ed9d700 1 -- 172.21.15.45:0/4258641588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cc71fe870 con 0x561cc7247f40 2023-09-23T09:32:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.826+0000 7f309ed9d700 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7248320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f309000bc80 tx=0x7f309000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda447e2e249c6ed server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.826+0000 7f309ed9d700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f30880a17d0 0x7f30880a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:55 smithi045 ceph-mon[98627]: pgmap v108: 97 pgs: 97 active+clean; 5.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 53 MiB/s wr, 2.80k op/s 2023-09-23T09:32:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.828+0000 7f308cff9700 1 -- 172.21.15.45:0/4258641588 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309000fdb0 con 0x561cc7247f40 2023-09-23T09:32:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.828+0000 7f308cff9700 1 -- 172.21.15.45:0/4258641588 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3090011900 con 0x561cc7247f40 2023-09-23T09:32:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.828+0000 7f308cff9700 1 -- 172.21.15.45:0/4258641588 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30900109e0 con 0x561cc7247f40 2023-09-23T09:32:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.828+0000 7f309ed9d700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f30880a17d0 0x7f30880a3c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f309001e000 tx=0x7f3090010c20 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4258641588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 msgr2=0x561cc7248320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7248320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f309000bc80 tx=0x7f309000bf90 comp rx=0 tx=0).stop 2023-09-23T09:32:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4258641588 shutdown_connections 2023-09-23T09:32:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7255b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7248320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4258641588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4258641588 >> 172.21.15.45:0/4258641588 conn(0x561cc71bd200 msgr2=0x561cc72443c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4258641588 shutdown_connections 2023-09-23T09:32:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.829+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4258641588 wait complete. 2023-09-23T09:32:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 -- start start 2023-09-23T09:32:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 Processor -- start 2023-09-23T09:32:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7250460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7250ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc719bdd0 con 0x561cc7248860 2023-09-23T09:32:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc71386b0 con 0x561cc7247f40 2023-09-23T09:32:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc7138350 con 0x561cc7253710 2023-09-23T09:32:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.830+0000 7f30b0f73b80 10 client.0 _ll_register_callbacks cb 0x561cc6f957a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7250ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7250ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36124/0 (socket says 172.21.15.45:36124) 2023-09-23T09:32:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309ed9d700 1 -- 172.21.15.45:0/4181348205 learned_addr learned my addr 172.21.15.45:0/4181348205 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309e59c700 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7250460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309e59c700 1 -- 172.21.15.45:0/4181348205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 msgr2=0x561cc7250ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309e59c700 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7250ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309e59c700 1 -- 172.21.15.45:0/4181348205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 msgr2=0x561cc72509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309e59c700 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309e59c700 1 -- 172.21.15.45:0/4181348205 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309000b8e0 con 0x561cc7247f40 2023-09-23T09:32:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309ed9d700 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7250ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:32:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f309e59c700 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7250460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3094005910 tx=0x7f3094009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61bcfc4075b756ff server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f3085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f3085ffb700 1 -- 172.21.15.45:0/4181348205 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094003ca0 con 0x561cc7247f40 2023-09-23T09:32:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.831+0000 7f3085ffb700 1 -- 172.21.15.45:0/4181348205 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3094003e00 con 0x561cc7247f40 2023-09-23T09:32:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.832+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cc723c090 con 0x561cc7247f40 2023-09-23T09:32:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.832+0000 7f3085ffb700 1 -- 172.21.15.45:0/4181348205 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3094005070 con 0x561cc7247f40 2023-09-23T09:32:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.832+0000 7f30b0f73b80 10 client.25643 fetch_fsmap learned FSMap version 30 2023-09-23T09:32:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.832+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561cc6fc6d80 con 0x561cc7247f40 2023-09-23T09:32:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.832+0000 7f3085ffb700 1 -- 172.21.15.45:0/4181348205 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3094031020 con 0x561cc7247f40 2023-09-23T09:32:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f3085ffb700 1 -- 172.21.15.45:0/4181348205 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f309400fb60 con 0x561cc7247f40 2023-09-23T09:32:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f30b0f73b80 10 client.25643 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:32:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f30b0f73b80 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f30b0f73b80 20 client.25643 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72632a0 0x561cc72656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f30b0f73b80 4 client.25643 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:32:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cc7194db0 con 0x561cc72632a0 2023-09-23T09:32:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f309dd9b700 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72632a0 0x561cc72656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f3085ffb700 10 client.25643 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.833+0000 7f309dd9b700 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72632a0 0x561cc72656c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f703bcae50c885b server_cookie=dc3623b35bee9ec2 in_seq=0 out_seq=0 2023-09-23T09:32:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f3085ffb700 1 -- 172.21.15.45:0/4181348205 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561cc7194db0 con 0x561cc72632a0 2023-09-23T09:32:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f3085ffb700 10 client.25643 handle_command_reply: tid=0 2023-09-23T09:32:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 client.25643 shutdown 2023-09-23T09:32:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 msgr2=0x561cc7250460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7250460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3094005910 tx=0x7f3094009fb0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 shutdown_connections 2023-09-23T09:32:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72632a0 0x561cc72656c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc7250ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc7250460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f3085ffb700 0 client.25643 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/4181348205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc72509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 >> 172.21.15.45:0/4181348205 conn(0x561cc71bd200 msgr2=0x561cc7243f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 shutdown_connections 2023-09-23T09:32:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.834+0000 7f30b0f73b80 1 -- 172.21.15.45:0/4181348205 wait complete. 2023-09-23T09:32:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.835+0000 7f30b0f73b80 20 client.25643 trim_cache size 0 max 16384 2023-09-23T09:32:55.849 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:55 smithi123 ceph-mon[103577]: pgmap v108: 97 pgs: 97 active+clean; 5.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 53 MiB/s wr, 2.80k op/s 2023-09-23T09:32:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.857+0000 7f30b0f73b80 1 Processor -- start 2023-09-23T09:32:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.857+0000 7f30b0f73b80 1 -- start start 2023-09-23T09:32:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.857+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc724ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.857+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc7250d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.858+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc724f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.858+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc701c8c0 con 0x561cc7248860 2023-09-23T09:32:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.858+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc7253510 con 0x561cc7247f40 2023-09-23T09:32:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.858+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc724c2f0 con 0x561cc7253710 2023-09-23T09:32:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.858+0000 7f309dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc7250d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.859+0000 7f309dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc7250d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34022/0 (socket says 172.21.15.45:34022) 2023-09-23T09:32:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.859+0000 7f309dd9b700 1 -- 172.21.15.45:0/3989350783 learned_addr learned my addr 172.21.15.45:0/3989350783 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.859+0000 7f309dd9b700 1 -- 172.21.15.45:0/3989350783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 msgr2=0x561cc724f0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:32:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.859+0000 7f309dd9b700 1 --2- 172.21.15.45:0/3989350783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc724f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.859+0000 7f309dd9b700 1 -- 172.21.15.45:0/3989350783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 msgr2=0x561cc724ade0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:32:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.859+0000 7f309dd9b700 1 --2- 172.21.15.45:0/3989350783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc724ade0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.859+0000 7f309dd9b700 1 -- 172.21.15.45:0/3989350783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cc6fafc00 con 0x561cc7248860 2023-09-23T09:32:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.860+0000 7f309dd9b700 1 --2- 172.21.15.45:0/3989350783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc7250d10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f309800bf40 tx=0x7f30980cefa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedbcdcfa5dec5af server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.860+0000 7f308d7fa700 1 -- 172.21.15.45:0/3989350783 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980995f0 con 0x561cc7248860 2023-09-23T09:32:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.860+0000 7f308d7fa700 1 -- 172.21.15.45:0/3989350783 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309809a390 con 0x561cc7248860 2023-09-23T09:32:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.860+0000 7f308d7fa700 1 -- 172.21.15.45:0/3989350783 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098099ef0 con 0x561cc7248860 2023-09-23T09:32:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3989350783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 msgr2=0x561cc7250d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3989350783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc7250d10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f309800bf40 tx=0x7f30980cefa0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3989350783 shutdown_connections 2023-09-23T09:32:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3989350783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7253710 0x561cc724f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3989350783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7247f40 0x561cc724ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3989350783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7248860 0x561cc7250d10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3989350783 >> 172.21.15.45:0/3989350783 conn(0x561cc71bd200 msgr2=0x561cc724e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3989350783 shutdown_connections 2023-09-23T09:32:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.862+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3989350783 wait complete. 2023-09-23T09:32:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 -- start start 2023-09-23T09:32:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 Processor -- start 2023-09-23T09:32:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7247f40 0x561cc7257c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7248860 0x561cc7258190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7253710 0x561cc72b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc7258aa0 con 0x561cc7247f40 2023-09-23T09:32:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc72b6650 con 0x561cc7253710 2023-09-23T09:32:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc72b6790 con 0x561cc7248860 2023-09-23T09:32:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f30b0f73b80 10 client.0 _ll_register_callbacks cb 0x7f309400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f309e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7247f40 0x561cc7257c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f309e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7247f40 0x561cc7257c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34024/0 (socket says 172.21.15.45:34024) 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.863+0000 7f309e59c700 1 -- 172.21.15.45:0/3539134511 learned_addr learned my addr 172.21.15.45:0/3539134511 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309ed9d700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7253710 0x561cc72b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309dd9b700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7248860 0x561cc7258190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309e59c700 1 -- 172.21.15.45:0/3539134511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7248860 msgr2=0x561cc7258190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309e59c700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7248860 0x561cc7258190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309e59c700 1 -- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7253710 msgr2=0x561cc72b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309e59c700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7253710 0x561cc72b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309e59c700 1 -- 172.21.15.45:0/3539134511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a01bb550 con 0x561cc7247f40 2023-09-23T09:32:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309dd9b700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7248860 0x561cc7258190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:32:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f309e59c700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7247f40 0x561cc7257c50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3094001eb0 tx=0x7f3094009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6b48085ae18a9d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.864+0000 7f3085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:32:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.865+0000 7f3085ffb700 1 -- 172.21.15.45:0/3539134511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309402ce80 con 0x561cc7247f40 2023-09-23T09:32:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.865+0000 7f3085ffb700 1 -- 172.21.15.45:0/3539134511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3094000e40 con 0x561cc7247f40 2023-09-23T09:32:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.865+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3090011900 con 0x561cc7247f40 2023-09-23T09:32:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.865+0000 7f3085ffb700 1 -- 172.21.15.45:0/3539134511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f309402c3c0 con 0x561cc7247f40 2023-09-23T09:32:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.865+0000 7f30b0f73b80 10 client.15972 fetch_fsmap learned FSMap version 30 2023-09-23T09:32:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.865+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561cc6fed1f0 con 0x561cc7247f40 2023-09-23T09:32:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f3085ffb700 1 -- 172.21.15.45:0/3539134511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3090011900 con 0x561cc7247f40 2023-09-23T09:32:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f3085ffb700 1 -- 172.21.15.45:0/3539134511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3094005110 con 0x561cc7247f40 2023-09-23T09:32:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f30b0f73b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:32:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f30b0f73b80 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f30b0f73b80 20 client.15972 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72b7a90 0x561cc72b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f30b0f73b80 4 client.15972 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:32:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561cc7194ad0 con 0x561cc72b7a90 2023-09-23T09:32:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.866+0000 7f309dd9b700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72b7a90 0x561cc72b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.867+0000 7f3085ffb700 10 client.15972 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.867+0000 7f309dd9b700 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72b7a90 0x561cc72b9eb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2acdfcc1c7e2c3a3 server_cookie=692c56f21750ac14 in_seq=0 out_seq=0 2023-09-23T09:32:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f3085ffb700 1 -- 172.21.15.45:0/3539134511 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561cc7194ad0 con 0x561cc72b7a90 2023-09-23T09:32:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f3085ffb700 10 client.15972 handle_command_reply: tid=0 2023-09-23T09:32:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 client.15972 shutdown 2023-09-23T09:32:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7247f40 msgr2=0x561cc7257c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7247f40 0x561cc7257c50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3094001eb0 tx=0x7f3094009310 comp rx=0 tx=0).stop 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 shutdown_connections 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561cc72b7a90 0x561cc72b9eb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cc7253710 0x561cc72b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cc7247f40 0x561cc7257c50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f3085ffb700 0 client.15972 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 --2- 172.21.15.45:0/3539134511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561cc7248860 0x561cc7258190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 >> 172.21.15.45:0/3539134511 conn(0x561cc71bd200 msgr2=0x561cc7252600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 shutdown_connections 2023-09-23T09:32:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.887+0000 7f30b0f73b80 1 -- 172.21.15.45:0/3539134511 wait complete. 2023-09-23T09:32:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.888+0000 7f30b0f73b80 20 client.15972 trim_cache size 0 max 16384 2023-09-23T09:32:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:32:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:32:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:32:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:32:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.888+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f30880a17d0 msgr2=0x7f30880a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.888+0000 7f30a4d89700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f30880a17d0 0x7f30880a3c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f309001e000 tx=0x7f3090010c20 comp rx=0 tx=0).stop 2023-09-23T09:32:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.888+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f9e0 msgr2=0x7f30a0119ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.888+0000 7f30a4d89700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f9e0 0x7f30a0119ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f309800bd40 tx=0x7f309800be20 comp rx=0 tx=0).stop 2023-09-23T09:32:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 shutdown_connections 2023-09-23T09:32:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f30880a17d0 0x7f30880a3c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30a011a600 0x7f30a01137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30a010f570 0x7f30a0119990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 --2- 172.21.15.45:0/1394098886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a010f9e0 0x7f30a0119ed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 >> 172.21.15.45:0/1394098886 conn(0x7f30a006cee0 msgr2=0x7f30a011bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 shutdown_connections 2023-09-23T09:32:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:32:55.889+0000 7f30a4d89700 1 -- 172.21.15.45:0/1394098886 wait complete. 2023-09-23T09:32:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 190.0 s, 2783.2 tps, lat 35.911 ms stddev 39.980 2023-09-23T09:32:57.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:57 smithi098 ceph-mon[102386]: pgmap v109: 97 pgs: 97 active+clean; 5.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 41 MiB/s wr, 2.28k op/s 2023-09-23T09:32:57.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:57 smithi045 ceph-mon[98627]: pgmap v109: 97 pgs: 97 active+clean; 5.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 41 MiB/s wr, 2.28k op/s 2023-09-23T09:32:57.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:57 smithi123 ceph-mon[103577]: pgmap v109: 97 pgs: 97 active+clean; 5.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 41 MiB/s wr, 2.28k op/s 2023-09-23T09:32:59.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:32:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.238+0000 7f269056e700 1 -- 172.21.15.45:0/320134217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880ff600 msgr2=0x7f26880ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.238+0000 7f269056e700 1 --2- 172.21.15.45:0/320134217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880ff600 0x7f26880ffa80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f267801a9a0 tx=0x7f267801acb0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.239+0000 7f269056e700 1 -- 172.21.15.45:0/320134217 shutdown_connections 2023-09-23T09:32:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.239+0000 7f269056e700 1 --2- 172.21.15.45:0/320134217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880fffc0 0x7f268810c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.239+0000 7f269056e700 1 --2- 172.21.15.45:0/320134217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 0x7f26881058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.239+0000 7f269056e700 1 --2- 172.21.15.45:0/320134217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880ff600 0x7f26880ffa80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.239+0000 7f269056e700 1 -- 172.21.15.45:0/320134217 >> 172.21.15.45:0/320134217 conn(0x7f2688074fc0 msgr2=0x7f26880753d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.241+0000 7f269056e700 1 -- 172.21.15.45:0/320134217 shutdown_connections 2023-09-23T09:32:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.241+0000 7f269056e700 1 -- 172.21.15.45:0/320134217 wait complete. 2023-09-23T09:32:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.241+0000 7f269056e700 1 Processor -- start 2023-09-23T09:32:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f269056e700 1 -- start start 2023-09-23T09:32:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f269056e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880ff600 0x7f268810bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f269056e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880fffc0 0x7f2688106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f269056e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 0x7f26881074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f269056e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2688107ae0 con 0x7f26880fffc0 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f269056e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2688107c50 con 0x7f26880ff600 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f269056e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2688107e10 con 0x7f26881054c0 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 0x7f26881074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268db09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880fffc0 0x7f2688106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268e30a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880ff600 0x7f268810bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 0x7f26881074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36144/0 (socket says 172.21.15.45:36144) 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 -- 172.21.15.45:0/3443415249 learned_addr learned my addr 172.21.15.45:0/3443415249 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268db09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880fffc0 0x7f2688106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34038/0 (socket says 172.21.15.45:34038) 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 -- 172.21.15.45:0/3443415249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880ff600 msgr2=0x7f268810bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880ff600 0x7f268810bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 -- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880fffc0 msgr2=0x7f2688106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880fffc0 0x7f2688106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268eb0b700 1 -- 172.21.15.45:0/3443415249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267801a600 con 0x7f26881054c0 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.242+0000 7f268db09700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880fffc0 0x7f2688106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.243+0000 7f268eb0b700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 0x7f26881074a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2680007d70 tx=0x7f268000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:59.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.243+0000 7f267f7fe700 1 -- 172.21.15.45:0/3443415249 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268000ac90 con 0x7f26881054c0 2023-09-23T09:32:59.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.243+0000 7f267f7fe700 1 -- 172.21.15.45:0/3443415249 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f268000adf0 con 0x7f26881054c0 2023-09-23T09:32:59.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.243+0000 7f267f7fe700 1 -- 172.21.15.45:0/3443415249 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800115e0 con 0x7f26881054c0 2023-09-23T09:32:59.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.244+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268819d0e0 con 0x7f26881054c0 2023-09-23T09:32:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.244+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268819d570 con 0x7f26881054c0 2023-09-23T09:32:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.247+0000 7f267f7fe700 1 -- 172.21.15.45:0/3443415249 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f2680012420 con 0x7f26881054c0 2023-09-23T09:32:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.248+0000 7f267f7fe700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f26740a17f0 0x7f26740a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.249+0000 7f267f7fe700 1 -- 172.21.15.45:0/3443415249 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f26800cd190 con 0x7f26881054c0 2023-09-23T09:32:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.249+0000 7f268e30a700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f26740a17f0 0x7f26740a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.250+0000 7f268e30a700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f26740a17f0 0x7f26740a3cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2684005950 tx=0x7f268400b410 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.250+0000 7f269c758b80 1 Processor -- start 2023-09-23T09:32:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.251+0000 7f269c758b80 1 -- start start 2023-09-23T09:32:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.251+0000 7f269c758b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343abf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.251+0000 7f269c758b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 0x565343ad2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.251+0000 7f269c758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ad57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.251+0000 7f269c758b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343a28460 con 0x565343abf400 2023-09-23T09:32:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.251+0000 7f269c758b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343a19c20 con 0x565343abfd20 2023-09-23T09:32:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.251+0000 7f269c758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343a17540 con 0x565343ad33e0 2023-09-23T09:32:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268eb0b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ad57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268eb0b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ad57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36154/0 (socket says 172.21.15.45:36154) 2023-09-23T09:32:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268eb0b700 1 -- 172.21.15.45:0/2255723341 learned_addr learned my addr 172.21.15.45:0/2255723341 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268e30a700 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343abf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268e30a700 1 -- 172.21.15.45:0/2255723341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 msgr2=0x565343ad57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268e30a700 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ad57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268e30a700 1 -- 172.21.15.45:0/2255723341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 msgr2=0x565343ad2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268e30a700 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 0x565343ad2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268e30a700 1 -- 172.21.15.45:0/2255723341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565343a758d0 con 0x565343abf400 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.252+0000 7f268eb0b700 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ad57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.253+0000 7f268e30a700 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343abf7e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2684010760 tx=0x7f2684013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babadbc5187c6452 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.253+0000 7f267cff9700 1 -- 172.21.15.45:0/2255723341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684011bb0 con 0x565343abf400 2023-09-23T09:32:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.253+0000 7f267cff9700 1 -- 172.21.15.45:0/2255723341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2684015070 con 0x565343abf400 2023-09-23T09:32:59.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.253+0000 7f267cff9700 1 -- 172.21.15.45:0/2255723341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268401c610 con 0x565343abf400 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 -- 172.21.15.45:0/2255723341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 msgr2=0x565343abf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343abf7e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2684010760 tx=0x7f2684013000 comp rx=0 tx=0).stop 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 -- 172.21.15.45:0/2255723341 shutdown_connections 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ad57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343abf7e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 --2- 172.21.15.45:0/2255723341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 0x565343ad2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 -- 172.21.15.45:0/2255723341 >> 172.21.15.45:0/2255723341 conn(0x565343a34260 msgr2=0x565343abb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:59.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 -- 172.21.15.45:0/2255723341 shutdown_connections 2023-09-23T09:32:59.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.254+0000 7f269c758b80 1 -- 172.21.15.45:0/2255723341 wait complete. 2023-09-23T09:32:59.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.255+0000 7f269c758b80 1 -- start start 2023-09-23T09:32:59.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.255+0000 7f269c758b80 1 Processor -- start 2023-09-23T09:32:59.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f269c758b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343acc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f269c758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abfd20 0x565343accd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f269c758b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343ad33e0 0x565343ad25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f269c758b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343a12e30 con 0x565343abf400 2023-09-23T09:32:59.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f269c758b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653439af710 con 0x565343ad33e0 2023-09-23T09:32:59.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f269c758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653439af3b0 con 0x565343abfd20 2023-09-23T09:32:59.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f269c758b80 10 client.0 _ll_register_callbacks cb 0x56534380c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268e30a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343acc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268eb0b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343ad33e0 0x565343ad25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268eb0b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343ad33e0 0x565343ad25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55406/0 (socket says 172.21.15.45:55406) 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268e30a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343acc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34052/0 (socket says 172.21.15.45:34052) 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268eb0b700 1 -- 172.21.15.45:0/2894703077 learned_addr learned my addr 172.21.15.45:0/2894703077 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268db09700 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abfd20 0x565343accd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268eb0b700 1 -- 172.21.15.45:0/2894703077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abfd20 msgr2=0x565343accd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.258+0000 7f268eb0b700 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abfd20 0x565343accd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f268eb0b700 1 -- 172.21.15.45:0/2894703077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 msgr2=0x565343acc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f268eb0b700 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343acc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f268eb0b700 1 -- 172.21.15.45:0/2894703077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268819d570 con 0x565343ad33e0 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f268e30a700 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343acc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f268eb0b700 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343ad33e0 0x565343ad25c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26800966a0 tx=0x7f2680099e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c758b8db86411eeb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f2671ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f2671ffb700 1 -- 172.21.15.45:0/2894703077 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268009a600 con 0x565343ad33e0 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f2671ffb700 1 -- 172.21.15.45:0/2894703077 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f268009a1a0 con 0x565343ad33e0 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565343ab3250 con 0x565343ad33e0 2023-09-23T09:32:59.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.259+0000 7f2671ffb700 1 -- 172.21.15.45:0/2894703077 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f268009e2f0 con 0x565343ad33e0 2023-09-23T09:32:59.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.260+0000 7f269c758b80 10 client.25652 fetch_fsmap learned FSMap version 30 2023-09-23T09:32:59.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.260+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2684010a50 con 0x565343ad33e0 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f2671ffb700 1 -- 172.21.15.45:0/2894703077 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565343ab3250 con 0x565343ad33e0 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f2671ffb700 1 -- 172.21.15.45:0/2894703077 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f26800a2130 con 0x565343ad33e0 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f269c758b80 10 client.25652 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f269c758b80 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f269c758b80 20 client.25652 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f269c758b80 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343aeb650 0x565343aeda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f269c758b80 4 client.25652 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:32:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565343a0be10 con 0x565343aeb650 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.261+0000 7f268e30a700 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343aeb650 0x565343aeda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.262+0000 7f2671ffb700 10 client.25652 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.262+0000 7f268e30a700 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343aeb650 0x565343aeda70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac12c6795927859 server_cookie=5941a7dfe5e388e7 in_seq=0 out_seq=0 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.262+0000 7f2671ffb700 1 -- 172.21.15.45:0/2894703077 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565343a0be10 con 0x565343aeb650 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.262+0000 7f2671ffb700 10 client.25652 handle_command_reply: tid=0 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.262+0000 7f269c758b80 1 client.25652 shutdown 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343ad33e0 msgr2=0x565343ad25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343ad33e0 0x565343ad25c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26800966a0 tx=0x7f2680099e00 comp rx=0 tx=0).stop 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 shutdown_connections 2023-09-23T09:32:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343aeb650 0x565343aeda70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343ad33e0 0x565343ad25c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abf400 0x565343acc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 --2- 172.21.15.45:0/2894703077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abfd20 0x565343accd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 >> 172.21.15.45:0/2894703077 conn(0x565343a34260 msgr2=0x565343abb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f2671ffb700 0 client.25652 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.263+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 shutdown_connections 2023-09-23T09:32:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.264+0000 7f269c758b80 1 -- 172.21.15.45:0/2894703077 wait complete. 2023-09-23T09:32:59.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.266+0000 7f269c758b80 20 client.25652 trim_cache size 0 max 16384 2023-09-23T09:32:59.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.295+0000 7f269c758b80 1 Processor -- start 2023-09-23T09:32:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f269c758b80 1 -- start start 2023-09-23T09:32:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f269c758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abf400 0x565343acc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f269c758b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 0x565343acc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f269c758b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343ad33e0 0x565343ad47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f269c758b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343893920 con 0x565343ad33e0 2023-09-23T09:32:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f269c758b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343ac0f10 con 0x565343abfd20 2023-09-23T09:32:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f269c758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343ac8660 con 0x565343abf400 2023-09-23T09:32:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f268e30a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abf400 0x565343acc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f268e30a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abf400 0x565343acc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36166/0 (socket says 172.21.15.45:36166) 2023-09-23T09:32:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.299+0000 7f268e30a700 1 -- 172.21.15.45:0/4056002008 learned_addr learned my addr 172.21.15.45:0/4056002008 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.300+0000 7f268db09700 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 0x565343acc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.300+0000 7f268e30a700 1 -- 172.21.15.45:0/4056002008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 msgr2=0x565343acc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.300+0000 7f268e30a700 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 0x565343acc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.300+0000 7f268e30a700 1 -- 172.21.15.45:0/4056002008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343ad33e0 msgr2=0x565343ad47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.300+0000 7f268e30a700 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343ad33e0 0x565343ad47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.300+0000 7f268e30a700 1 -- 172.21.15.45:0/4056002008 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56534383dde0 con 0x565343abf400 2023-09-23T09:32:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.301+0000 7f268e30a700 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abf400 0x565343acc450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2684011a00 tx=0x7f268400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5525ca41f36d9b04 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.301+0000 7f267d7fa700 1 -- 172.21.15.45:0/4056002008 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268402b030 con 0x565343abf400 2023-09-23T09:32:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.301+0000 7f267d7fa700 1 -- 172.21.15.45:0/4056002008 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f268402c040 con 0x565343abf400 2023-09-23T09:32:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.301+0000 7f267d7fa700 1 -- 172.21.15.45:0/4056002008 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268401e7a0 con 0x565343abf400 2023-09-23T09:32:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.302+0000 7f269c758b80 1 -- 172.21.15.45:0/4056002008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abf400 msgr2=0x565343acc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.302+0000 7f269c758b80 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abf400 0x565343acc450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2684011a00 tx=0x7f268400bf90 comp rx=0 tx=0).stop 2023-09-23T09:32:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.302+0000 7f269c758b80 1 -- 172.21.15.45:0/4056002008 shutdown_connections 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.302+0000 7f269c758b80 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343ad33e0 0x565343ad47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.302+0000 7f269c758b80 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343abf400 0x565343acc450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.302+0000 7f269c758b80 1 --2- 172.21.15.45:0/4056002008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abfd20 0x565343acc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.302+0000 7f269c758b80 1 -- 172.21.15.45:0/4056002008 >> 172.21.15.45:0/4056002008 conn(0x565343a34260 msgr2=0x565343ab9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.303+0000 7f269c758b80 1 -- 172.21.15.45:0/4056002008 shutdown_connections 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.303+0000 7f269c758b80 1 -- 172.21.15.45:0/4056002008 wait complete. 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 -- start start 2023-09-23T09:32:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 Processor -- start 2023-09-23T09:32:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abf400 0x565343acda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abfd20 0x565343acdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ace510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343b3e430 con 0x565343abfd20 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343b3e570 con 0x565343abf400 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343b3e700 con 0x565343ad33e0 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.304+0000 7f269c758b80 10 client.0 _ll_register_callbacks cb 0x7f2680099ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abf400 0x565343acda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268db09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abfd20 0x565343acdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abf400 0x565343acda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55416/0 (socket says 172.21.15.45:55416) 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 -- 172.21.15.45:0/1598719771 learned_addr learned my addr 172.21.15.45:0/1598719771 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:32:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268eb0b700 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ace510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 -- 172.21.15.45:0/1598719771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 msgr2=0x565343ace510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ace510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 -- 172.21.15.45:0/1598719771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abfd20 msgr2=0x565343acdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abfd20 0x565343acdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 -- 172.21.15.45:0/1598719771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565343a758d0 con 0x565343abf400 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f268e30a700 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abf400 0x565343acda90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2684010f80 tx=0x7f2684032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903cd39ad1dfdb4d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f2671ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:32:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f2671ffb700 1 -- 172.21.15.45:0/1598719771 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268402b040 con 0x565343abf400 2023-09-23T09:32:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f2671ffb700 1 -- 172.21.15.45:0/1598719771 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2684017070 con 0x565343abf400 2023-09-23T09:32:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.305+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2684015070 con 0x565343abf400 2023-09-23T09:32:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f2671ffb700 1 -- 172.21.15.45:0/1598719771 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2684011d40 con 0x565343abf400 2023-09-23T09:32:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 10 client.25658 fetch_fsmap learned FSMap version 30 2023-09-23T09:32:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x565343826c60 con 0x565343abf400 2023-09-23T09:32:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f2671ffb700 1 -- 172.21.15.45:0/1598719771 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2684015070 con 0x565343abf400 2023-09-23T09:32:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f2671ffb700 1 -- 172.21.15.45:0/1598719771 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f268403d580 con 0x565343abf400 2023-09-23T09:32:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 20 client.25658 populate_metadata read hostname 'smithi045' 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343b3fa60 0x565343b41e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 4 client.25658 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.306+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565343a0bb30 con 0x565343b3fa60 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.307+0000 7f268db09700 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343b3fa60 0x565343b41e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.307+0000 7f2671ffb700 10 client.25658 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.307+0000 7f268db09700 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343b3fa60 0x565343b41e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bd070b97404c6b server_cookie=cb7278dae2f2617b in_seq=0 out_seq=0 2023-09-23T09:32:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f2671ffb700 1 -- 172.21.15.45:0/1598719771 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x565343a0bb30 con 0x565343b3fa60 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f2671ffb700 10 client.25658 handle_command_reply: tid=0 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 client.25658 shutdown 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abf400 msgr2=0x565343acda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abf400 0x565343acda90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2684010f80 tx=0x7f2684032430 comp rx=0 tx=0).stop 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 shutdown_connections 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565343b3fa60 0x565343b41e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565343ad33e0 0x565343ace510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565343abf400 0x565343acda90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f2671ffb700 0 client.25658 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 --2- 172.21.15.45:0/1598719771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565343abfd20 0x565343acdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 >> 172.21.15.45:0/1598719771 conn(0x565343a34260 msgr2=0x565343ac62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 shutdown_connections 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.308+0000 7f269c758b80 1 -- 172.21.15.45:0/1598719771 wait complete. 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269c758b80 20 client.25658 trim_cache size 0 max 16384 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:32:59.315 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:32:59.316 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:32:59.316 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:32:59.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:32:59.316 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:32:59.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:32:59.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:32:59.316 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:32:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f26740a17f0 msgr2=0x7f26740a3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f26740a17f0 0x7f26740a3cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2684005950 tx=0x7f268400b410 comp rx=0 tx=0).stop 2023-09-23T09:32:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 msgr2=0x7f26881074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:32:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 0x7f26881074a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2680007d70 tx=0x7f268000a340 comp rx=0 tx=0).stop 2023-09-23T09:32:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 shutdown_connections 2023-09-23T09:32:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26881054c0 0x7f26881074a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880ff600 0x7f268810bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f26740a17f0 0x7f26740a3cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 --2- 172.21.15.45:0/3443415249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26880fffc0 0x7f2688106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:32:59.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 >> 172.21.15.45:0/3443415249 conn(0x7f2688074fc0 msgr2=0x7f2688102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:32:59.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 shutdown_connections 2023-09-23T09:32:59.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:32:59.309+0000 7f269056e700 1 -- 172.21.15.45:0/3443415249 wait complete. 2023-09-23T09:32:59.331 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:32:59.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:32:59 smithi045 ceph-mon[98627]: pgmap v110: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 50 MiB/s wr, 2.67k op/s 2023-09-23T09:32:59.799 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:32:59 smithi098 ceph-mon[102386]: pgmap v110: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 50 MiB/s wr, 2.67k op/s 2023-09-23T09:32:59.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:32:59 smithi123 ceph-mon[103577]: pgmap v110: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 50 MiB/s wr, 2.67k op/s 2023-09-23T09:33:01.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:01 smithi098 ceph-mon[102386]: pgmap v111: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 943 op/s 2023-09-23T09:33:01.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:01 smithi045 ceph-mon[98627]: pgmap v111: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 943 op/s 2023-09-23T09:33:01.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:01 smithi123 ceph-mon[103577]: pgmap v111: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 943 op/s 2023-09-23T09:33:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 195.0 s, 2031.4 tps, lat 47.845 ms stddev 59.117 2023-09-23T09:33:03.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:03 smithi098 ceph-mon[102386]: pgmap v112: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 960 op/s 2023-09-23T09:33:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:03 smithi045 ceph-mon[98627]: pgmap v112: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 960 op/s 2023-09-23T09:33:03.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:03 smithi123 ceph-mon[103577]: pgmap v112: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 960 op/s 2023-09-23T09:33:05.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:05 smithi098 ceph-mon[102386]: pgmap v113: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.19k op/s 2023-09-23T09:33:05.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:05 smithi045 ceph-mon[98627]: pgmap v113: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.19k op/s 2023-09-23T09:33:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:05 smithi123 ceph-mon[103577]: pgmap v113: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.19k op/s 2023-09-23T09:33:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 200.0 s, 1134.6 tps, lat 89.698 ms stddev 147.542 2023-09-23T09:33:07.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:07 smithi098 ceph-mon[102386]: pgmap v114: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 692 op/s 2023-09-23T09:33:07.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:07 smithi045 ceph-mon[98627]: pgmap v114: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 692 op/s 2023-09-23T09:33:07.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:07 smithi123 ceph-mon[103577]: pgmap v114: 97 pgs: 97 active+clean; 5.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 692 op/s 2023-09-23T09:33:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.061+0000 7fbc20380700 1 -- 172.21.15.45:0/1111321214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f570 msgr2=0x7fbc18107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.061+0000 7fbc20380700 1 --2- 172.21.15.45:0/1111321214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f570 0x7fbc18107e20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00b0a0 tx=0x7fbc0c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 -- 172.21.15.45:0/1111321214 shutdown_connections 2023-09-23T09:33:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 --2- 172.21.15.45:0/1111321214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc1811a600 0x7fbc1811c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 --2- 172.21.15.45:0/1111321214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f570 0x7fbc18107e20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 --2- 172.21.15.45:0/1111321214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810f9e0 0x7fbc18108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 -- 172.21.15.45:0/1111321214 >> 172.21.15.45:0/1111321214 conn(0x7fbc1806cee0 msgr2=0x7fbc1806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 -- 172.21.15.45:0/1111321214 shutdown_connections 2023-09-23T09:33:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 -- 172.21.15.45:0/1111321214 wait complete. 2023-09-23T09:33:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.062+0000 7fbc20380700 1 Processor -- start 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc20380700 1 -- start start 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc20380700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810f570 0x7fbc181a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc20380700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f9e0 0x7fbc181a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc20380700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc1811a600 0x7fbc181aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc20380700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181ab020 con 0x7fbc1810f570 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc20380700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181ab190 con 0x7fbc1810f9e0 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc20380700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181ab350 con 0x7fbc1811a600 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc1e11c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810f570 0x7fbc181a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc1e91d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc1811a600 0x7fbc181aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc1e11c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810f570 0x7fbc181a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34078/0 (socket says 172.21.15.45:34078) 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc1e11c700 1 -- 172.21.15.45:0/1573344752 learned_addr learned my addr 172.21.15.45:0/1573344752 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.063+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f9e0 0x7fbc181a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc1e91d700 1 -- 172.21.15.45:0/1573344752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f9e0 msgr2=0x7fbc181a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f9e0 0x7fbc181a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc1e91d700 1 -- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810f570 msgr2=0x7fbc181a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810f570 0x7fbc181a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc1e91d700 1 -- 172.21.15.45:0/1573344752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c009d00 con 0x7fbc1811a600 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f9e0 0x7fbc181a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc1811a600 0x7fbc181aa9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b3c0 tx=0x7fbc1000b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc177fe700 1 -- 172.21.15.45:0/1573344752 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10011830 con 0x7fbc1811a600 2023-09-23T09:33:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc181ab630 con 0x7fbc1811a600 2023-09-23T09:33:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc177fe700 1 -- 172.21.15.45:0/1573344752 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc1000be80 con 0x7fbc1811a600 2023-09-23T09:33:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc177fe700 1 -- 172.21.15.45:0/1573344752 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10010d50 con 0x7fbc1811a600 2023-09-23T09:33:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.064+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc181b2e70 con 0x7fbc1811a600 2023-09-23T09:33:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.068+0000 7fbc177fe700 1 -- 172.21.15.45:0/1573344752 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fbc10010380 con 0x7fbc1811a600 2023-09-23T09:33:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.069+0000 7fbc2c56ab80 1 Processor -- start 2023-09-23T09:33:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.069+0000 7fbc177fe700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbc080a1710 0x7fbc080a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.070+0000 7fbc177fe700 1 -- 172.21.15.45:0/1573344752 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbc100cca80 con 0x7fbc1811a600 2023-09-23T09:33:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.071+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbc080a1710 0x7fbc080a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.072+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbc080a1710 0x7fbc080a3bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc04006fd0 tx=0x7fbc04008040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.088+0000 7fbc2c56ab80 1 -- start start 2023-09-23T09:33:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.088+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0850f40 0x5580d0851320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.088+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d085c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.088+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d085eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.088+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d07ba400 con 0x5580d085c710 2023-09-23T09:33:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.088+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d07abbc0 con 0x5580d0851860 2023-09-23T09:33:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.088+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d07a94e0 con 0x5580d0850f40 2023-09-23T09:33:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d085eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d085eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34088/0 (socket says 172.21.15.45:34088) 2023-09-23T09:33:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 -- 172.21.15.45:0/2640876660 learned_addr learned my addr 172.21.15.45:0/2640876660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d085c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 -- 172.21.15.45:0/2640876660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0850f40 msgr2=0x5580d0851320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:33:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0850f40 0x5580d0851320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 -- 172.21.15.45:0/2640876660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 msgr2=0x5580d085c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d085c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.089+0000 7fbc1e91d700 1 -- 172.21.15.45:0/2640876660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580d0807870 con 0x5580d085c710 2023-09-23T09:33:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.090+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d085eb00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbc10095ff0 tx=0x7fbc10099540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72cd21fc27e9ade server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.090+0000 7fbc14d86700 1 -- 172.21.15.45:0/2640876660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1009dc40 con 0x5580d085c710 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.090+0000 7fbc14d86700 1 -- 172.21.15.45:0/2640876660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc100a1e80 con 0x5580d085c710 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.090+0000 7fbc14d86700 1 -- 172.21.15.45:0/2640876660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10099240 con 0x5580d085c710 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.091+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2640876660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 msgr2=0x5580d085eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.091+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d085eb00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbc10095ff0 tx=0x7fbc10099540 comp rx=0 tx=0).stop 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.091+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2640876660 shutdown_connections 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.091+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d085eb00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.091+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0850f40 0x5580d0851320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.091+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2640876660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d085c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.091+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2640876660 >> 172.21.15.45:0/2640876660 conn(0x5580d07c6200 msgr2=0x5580d084d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.092+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2640876660 shutdown_connections 2023-09-23T09:33:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.092+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2640876660 wait complete. 2023-09-23T09:33:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.093+0000 7fbc2c56ab80 1 -- start start 2023-09-23T09:33:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.093+0000 7fbc2c56ab80 1 Processor -- start 2023-09-23T09:33:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.093+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d0859550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.093+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d0859a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d07a4dd0 con 0x5580d085c710 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d07416b0 con 0x5580d0851860 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d0741350 con 0x5580d0859fd0 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc2c56ab80 10 client.0 _ll_register_callbacks cb 0x5580d059e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc1d91b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d0859a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc1d91b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d0859a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34094/0 (socket says 172.21.15.45:34094) 2023-09-23T09:33:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.094+0000 7fbc1d91b700 1 -- 172.21.15.45:0/2769882939 learned_addr learned my addr 172.21.15.45:0/2769882939 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.098+0000 7fbc1d91b700 1 -- 172.21.15.45:0/2769882939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 msgr2=0x5580d08682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.098+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.098+0000 7fbc1d91b700 1 -- 172.21.15.45:0/2769882939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 msgr2=0x5580d0859550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:33:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.098+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d0859550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.098+0000 7fbc1d91b700 1 -- 172.21.15.45:0/2769882939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc181b2e70 con 0x5580d085c710 2023-09-23T09:33:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.098+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d0859a90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400cbd0 tx=0x7fbc04004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8fc4bfc93fed15 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.098+0000 7fbc01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:33:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.099+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2769882939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04013d60 con 0x5580d085c710 2023-09-23T09:33:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.099+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2769882939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbc0401f440 con 0x5580d085c710 2023-09-23T09:33:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.099+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580d0845090 con 0x5580d085c710 2023-09-23T09:33:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.099+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2769882939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc0401f600 con 0x5580d085c710 2023-09-23T09:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.103+0000 7fbc2c56ab80 10 client.15996 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.104+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580d05cfd80 con 0x5580d085c710 2023-09-23T09:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.104+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2769882939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580d0845090 con 0x5580d085c710 2023-09-23T09:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.104+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2769882939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbc04028880 con 0x5580d085c710 2023-09-23T09:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.105+0000 7fbc2c56ab80 10 client.15996 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.105+0000 7fbc2c56ab80 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.105+0000 7fbc2c56ab80 20 client.15996 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.105+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d086c8b0 0x5580d086ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.105+0000 7fbc2c56ab80 4 client.15996 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:33:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.105+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580d079ddb0 con 0x5580d086c8b0 2023-09-23T09:33:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.106+0000 7fbc1e11c700 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d086c8b0 0x5580d086ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.107+0000 7fbc1e11c700 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d086c8b0 0x5580d086ecd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277090760076cfdd server_cookie=9f18c0e7a7e0e9cd in_seq=0 out_seq=0 2023-09-23T09:33:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.107+0000 7fbc01ffb700 10 client.15996 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.112+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2769882939 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5580d079ddb0 con 0x5580d086c8b0 2023-09-23T09:33:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.112+0000 7fbc01ffb700 10 client.15996 handle_command_reply: tid=0 2023-09-23T09:33:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.112+0000 7fbc2c56ab80 1 client.15996 shutdown 2023-09-23T09:33:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.113+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 msgr2=0x5580d0859a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.113+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d0859a90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400cbd0 tx=0x7fbc04004800 comp rx=0 tx=0).stop 2023-09-23T09:33:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 shutdown_connections 2023-09-23T09:33:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d086c8b0 0x5580d086ecd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d0859550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2769882939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d0859a90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 >> 172.21.15.45:0/2769882939 conn(0x5580d07c6200 msgr2=0x5580d084cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc01ffb700 0 client.15996 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 shutdown_connections 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2769882939 wait complete. 2023-09-23T09:33:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.114+0000 7fbc2c56ab80 20 client.15996 trim_cache size 0 max 16384 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.138+0000 7fbc2c56ab80 1 Processor -- start 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc2c56ab80 1 -- start start 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d0853fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d0859fd0 0x5580d0863230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d085c710 0x5580d0867980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d06258c0 con 0x5580d0859fd0 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d0863770 con 0x5580d0851860 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d0855380 con 0x5580d085c710 2023-09-23T09:33:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc1e91d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d085c710 0x5580d0867980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc1e91d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d085c710 0x5580d0867980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36204/0 (socket says 172.21.15.45:36204) 2023-09-23T09:33:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.139+0000 7fbc1e91d700 1 -- 172.21.15.45:0/20545717 learned_addr learned my addr 172.21.15.45:0/20545717 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.140+0000 7fbc1e91d700 1 -- 172.21.15.45:0/20545717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 msgr2=0x5580d0853fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:33:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.140+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/20545717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d0853fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.140+0000 7fbc1e91d700 1 -- 172.21.15.45:0/20545717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d0859fd0 msgr2=0x5580d0863230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.140+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/20545717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d0859fd0 0x5580d0863230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.140+0000 7fbc1e91d700 1 -- 172.21.15.45:0/20545717 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580d05b8c00 con 0x5580d085c710 2023-09-23T09:33:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.142+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/20545717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d085c710 0x5580d0867980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc10010900 tx=0x7fbc10009180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e4931c9f5dec26 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.142+0000 7fbc15587700 1 -- 172.21.15.45:0/20545717 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100998c0 con 0x5580d085c710 2023-09-23T09:33:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.142+0000 7fbc15587700 1 -- 172.21.15.45:0/20545717 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc1009dc40 con 0x5580d085c710 2023-09-23T09:33:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.142+0000 7fbc15587700 1 -- 172.21.15.45:0/20545717 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10099fb0 con 0x5580d085c710 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.143+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/20545717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d085c710 msgr2=0x5580d0867980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.143+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/20545717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d085c710 0x5580d0867980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc10010900 tx=0x7fbc10009180 comp rx=0 tx=0).stop 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/20545717 shutdown_connections 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/20545717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d085c710 0x5580d0867980 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/20545717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d0853fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/20545717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d0859fd0 0x5580d0863230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/20545717 >> 172.21.15.45:0/20545717 conn(0x5580d07c6200 msgr2=0x5580d0854810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/20545717 shutdown_connections 2023-09-23T09:33:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/20545717 wait complete. 2023-09-23T09:33:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 -- start start 2023-09-23T09:33:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.144+0000 7fbc2c56ab80 1 Processor -- start 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d08b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc2c56ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d08bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d08b5370 con 0x5580d085c710 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d08b54e0 con 0x5580d0851860 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc2c56ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580d08bfc30 con 0x5580d0859fd0 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc2c56ab80 10 client.0 _ll_register_callbacks cb 0x7fbc0400c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc1e11c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d08b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc1e11c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d08b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55452/0 (socket says 172.21.15.45:55452) 2023-09-23T09:33:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc1e11c700 1 -- 172.21.15.45:0/2907639195 learned_addr learned my addr 172.21.15.45:0/2907639195 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d08bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.147+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1e11c700 1 -- 172.21.15.45:0/2907639195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 msgr2=0x5580d08b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1e11c700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1e11c700 1 -- 172.21.15.45:0/2907639195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 msgr2=0x5580d08bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1e11c700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d08bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1e11c700 1 -- 172.21.15.45:0/2907639195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580d0807870 con 0x5580d0851860 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1e11c700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d08b4670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009a20 tx=0x7fbc0c009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a62ef3f41080a2b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.148+0000 7fbc1e91d700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d08bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.149+0000 7fbc01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:33:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.149+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2907639195 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00e040 con 0x5580d0851860 2023-09-23T09:33:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.149+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2907639195 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbc0c004d60 con 0x5580d0851860 2023-09-23T09:33:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.149+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc100a1e80 con 0x5580d0851860 2023-09-23T09:33:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.149+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2907639195 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc0c0074e0 con 0x5580d0851860 2023-09-23T09:33:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.149+0000 7fbc2c56ab80 10 client.25667 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.149+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580d05f61f0 con 0x5580d0851860 2023-09-23T09:33:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2907639195 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc100a1e80 con 0x5580d0851860 2023-09-23T09:33:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2907639195 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbc0c00b0d0 con 0x5580d0851860 2023-09-23T09:33:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc2c56ab80 10 client.25667 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc2c56ab80 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc2c56ab80 20 client.25667 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d08c1390 0x5580d08c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc2c56ab80 4 client.25667 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:33:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.150+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580d079dad0 con 0x5580d08c1390 2023-09-23T09:33:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.152+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d08c1390 0x5580d08c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.152+0000 7fbc1d91b700 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d08c1390 0x5580d08c37b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42313e62d529ba77 server_cookie=4c50c222e2b15bff in_seq=0 out_seq=0 2023-09-23T09:33:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.152+0000 7fbc01ffb700 10 client.25667 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.172+0000 7fbc01ffb700 1 -- 172.21.15.45:0/2907639195 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580d079dad0 con 0x5580d08c1390 2023-09-23T09:33:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.172+0000 7fbc01ffb700 10 client.25667 handle_command_reply: tid=0 2023-09-23T09:33:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.172+0000 7fbc2c56ab80 1 client.25667 shutdown 2023-09-23T09:33:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 msgr2=0x5580d08b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d08b4670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009a20 tx=0x7fbc0c009550 comp rx=0 tx=0).stop 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 shutdown_connections 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5580d08c1390 0x5580d08c37b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580d085c710 0x5580d08bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580d0851860 0x5580d08b4670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 --2- 172.21.15.45:0/2907639195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0859fd0 0x5580d08b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc01ffb700 0 client.25667 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 >> 172.21.15.45:0/2907639195 conn(0x5580d07c6200 msgr2=0x5580d0867980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 shutdown_connections 2023-09-23T09:33:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.175+0000 7fbc2c56ab80 1 -- 172.21.15.45:0/2907639195 wait complete. 2023-09-23T09:33:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.178+0000 7fbc2c56ab80 20 client.25667 trim_cache size 0 max 16384 2023-09-23T09:33:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:33:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:33:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:33:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:33:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.180+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbc080a1710 msgr2=0x7fbc080a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.180+0000 7fbc20380700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbc080a1710 0x7fbc080a3bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc04006fd0 tx=0x7fbc04008040 comp rx=0 tx=0).stop 2023-09-23T09:33:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.180+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc1811a600 msgr2=0x7fbc181aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.180+0000 7fbc20380700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc1811a600 0x7fbc181aa9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b3c0 tx=0x7fbc1000b6d0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 shutdown_connections 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc1811a600 0x7fbc181aa9e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbc080a1710 0x7fbc080a3bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810f570 0x7fbc181a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 --2- 172.21.15.45:0/1573344752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1810f9e0 0x7fbc181a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 >> 172.21.15.45:0/1573344752 conn(0x7fbc1806cee0 msgr2=0x7fbc1810d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 shutdown_connections 2023-09-23T09:33:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:09.181+0000 7fbc20380700 1 -- 172.21.15.45:0/1573344752 wait complete. 2023-09-23T09:33:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:09 smithi098 ceph-mon[102386]: pgmap v115: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 MiB/s wr, 2.33k op/s 2023-09-23T09:33:09.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:09 smithi045 ceph-mon[98627]: pgmap v115: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 MiB/s wr, 2.33k op/s 2023-09-23T09:33:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:09 smithi123 ceph-mon[103577]: pgmap v115: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 MiB/s wr, 2.33k op/s 2023-09-23T09:33:11.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:11 smithi098 ceph-mon[102386]: pgmap v116: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.94k op/s 2023-09-23T09:33:11.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:11 smithi045 ceph-mon[98627]: pgmap v116: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.94k op/s 2023-09-23T09:33:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:11 smithi123 ceph-mon[103577]: pgmap v116: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.94k op/s 2023-09-23T09:33:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 205.0 s, 2046.6 tps, lat 49.193 ms stddev 51.109 2023-09-23T09:33:13.835 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:13 smithi045 ceph-mon[98627]: pgmap v117: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 1.98k op/s 2023-09-23T09:33:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:13 smithi123 ceph-mon[103577]: pgmap v117: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 1.98k op/s 2023-09-23T09:33:14.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:13 smithi098 ceph-mon[102386]: pgmap v117: 97 pgs: 97 active+clean; 5.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 1.98k op/s 2023-09-23T09:33:15.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:15 smithi045 ceph-mon[98627]: pgmap v118: 97 pgs: 97 active+clean; 5.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 MiB/s wr, 2.36k op/s 2023-09-23T09:33:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:15 smithi123 ceph-mon[103577]: pgmap v118: 97 pgs: 97 active+clean; 5.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 MiB/s wr, 2.36k op/s 2023-09-23T09:33:16.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:15 smithi098 ceph-mon[102386]: pgmap v118: 97 pgs: 97 active+clean; 5.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 MiB/s wr, 2.36k op/s 2023-09-23T09:33:16.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 210.0 s, 1124.2 tps, lat 66.711 ms stddev 93.686 2023-09-23T09:33:17.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:17 smithi045 ceph-mon[98627]: pgmap v119: 97 pgs: 97 active+clean; 5.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 42 MiB/s wr, 2.07k op/s 2023-09-23T09:33:17.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:17 smithi123 ceph-mon[103577]: pgmap v119: 97 pgs: 97 active+clean; 5.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 42 MiB/s wr, 2.07k op/s 2023-09-23T09:33:18.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:17 smithi098 ceph-mon[102386]: pgmap v119: 97 pgs: 97 active+clean; 5.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 42 MiB/s wr, 2.07k op/s 2023-09-23T09:33:19.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:19 smithi045 ceph-mon[98627]: pgmap v120: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 61 MiB/s wr, 3.46k op/s 2023-09-23T09:33:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:19 smithi123 ceph-mon[103577]: pgmap v120: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 61 MiB/s wr, 3.46k op/s 2023-09-23T09:33:20.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:19 smithi098 ceph-mon[102386]: pgmap v120: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 61 MiB/s wr, 3.46k op/s 2023-09-23T09:33:21.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:21 smithi045 ceph-mon[98627]: pgmap v121: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 1.82k op/s 2023-09-23T09:33:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:21 smithi123 ceph-mon[103577]: pgmap v121: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 1.82k op/s 2023-09-23T09:33:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 215.0 s, 0.0 tps, lat -nan ms stddev -nan 2023-09-23T09:33:22.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:21 smithi098 ceph-mon[102386]: pgmap v121: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 1.82k op/s 2023-09-23T09:33:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 -- 172.21.15.45:0/1321148796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70100160 msgr2=0x7efc70100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 --2- 172.21.15.45:0/1321148796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70100160 0x7efc70100540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc600098f0 tx=0x7efc60009c00 comp rx=0 tx=0).stop 2023-09-23T09:33:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 -- 172.21.15.45:0/1321148796 shutdown_connections 2023-09-23T09:33:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 --2- 172.21.15.45:0/1321148796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70105050 0x7efc7010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 --2- 172.21.15.45:0/1321148796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70100160 0x7efc70100540 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 --2- 172.21.15.45:0/1321148796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc70100b10 0x7efc70104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 -- 172.21.15.45:0/1321148796 >> 172.21.15.45:0/1321148796 conn(0x7efc70076840 msgr2=0x7efc70076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.327+0000 7efc75a01700 1 -- 172.21.15.45:0/1321148796 shutdown_connections 2023-09-23T09:33:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.328+0000 7efc75a01700 1 -- 172.21.15.45:0/1321148796 wait complete. 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.328+0000 7efc75a01700 1 Processor -- start 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc75a01700 1 -- start start 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc75a01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70100160 0x7efc7010d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc75a01700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70105050 0x7efc7010dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc75a01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc701a30d0 0x7efc701a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc75a01700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701a5a00 con 0x7efc70100160 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc75a01700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701a5b40 con 0x7efc70105050 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc75a01700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701a5cb0 con 0x7efc701a30d0 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70100160 0x7efc7010d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70105050 0x7efc7010dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc701a30d0 0x7efc701a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70105050 0x7efc7010dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55466/0 (socket says 172.21.15.45:55466) 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70100160 0x7efc7010d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34116/0 (socket says 172.21.15.45:34116) 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 -- 172.21.15.45:0/2438528441 learned_addr learned my addr 172.21.15.45:0/2438528441 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 -- 172.21.15.45:0/2438528441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc701a30d0 msgr2=0x7efc701a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc701a30d0 0x7efc701a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 -- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70100160 msgr2=0x7efc7010d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70100160 0x7efc7010d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6e7fc700 1 -- 172.21.15.45:0/2438528441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68009d00 con 0x7efc70105050 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.329+0000 7efc6effd700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70100160 0x7efc7010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.330+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70105050 0x7efc7010dc00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e970 tx=0x7efc6400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.330+0000 7efc749ff700 1 -- 172.21.15.45:0/2438528441 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64009790 con 0x7efc70105050 2023-09-23T09:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.330+0000 7efc749ff700 1 -- 172.21.15.45:0/2438528441 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6400c7e0 con 0x7efc70105050 2023-09-23T09:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.330+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc60009520 con 0x7efc70105050 2023-09-23T09:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.330+0000 7efc749ff700 1 -- 172.21.15.45:0/2438528441 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc640054e0 con 0x7efc70105050 2023-09-23T09:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.330+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc701a6280 con 0x7efc70105050 2023-09-23T09:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.332+0000 7efc749ff700 1 -- 172.21.15.45:0/2438528441 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7efc6400c950 con 0x7efc70105050 2023-09-23T09:33:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.333+0000 7efc749ff700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efc580a1790 0x7efc580a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.333+0000 7efc749ff700 1 -- 172.21.15.45:0/2438528441 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efc64014070 con 0x7efc70105050 2023-09-23T09:33:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.333+0000 7efc6effd700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efc580a1790 0x7efc580a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 Processor -- start 2023-09-23T09:33:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 -- start start 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6effd700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efc580a1790 0x7efc580a3c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc60019040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321143320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x557321150b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573210ac400 con 0x557321143860 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732109dbc0 con 0x557321142f40 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc81bebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732109b4e0 con 0x55732114e710 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55474/0 (socket says 172.21.15.45:55474) 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6effd700 1 -- 172.21.15.45:0/4226635154 learned_addr learned my addr 172.21.15.45:0/4226635154 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x557321150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6e7fc700 1 -- 172.21.15.45:0/4226635154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 msgr2=0x557321150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x557321150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6e7fc700 1 -- 172.21.15.45:0/4226635154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 msgr2=0x557321143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6e7fc700 1 -- 172.21.15.45:0/4226635154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573210f9870 con 0x557321143860 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.334+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x557321150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114e1d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efc640cac70 tx=0x7efc6409a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5f49566213569 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc55ffb700 1 -- 172.21.15.45:0/4226635154 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6400cc50 con 0x557321143860 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc55ffb700 1 -- 172.21.15.45:0/4226635154 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6409ab90 con 0x557321143860 2023-09-23T09:33:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc55ffb700 1 -- 172.21.15.45:0/4226635154 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc640caef0 con 0x557321143860 2023-09-23T09:33:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 -- 172.21.15.45:0/4226635154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 msgr2=0x55732114e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114e1d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efc640cac70 tx=0x7efc6409a050 comp rx=0 tx=0).stop 2023-09-23T09:33:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 -- 172.21.15.45:0/4226635154 shutdown_connections 2023-09-23T09:33:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x557321150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 --2- 172.21.15.45:0/4226635154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114e1d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 -- 172.21.15.45:0/4226635154 >> 172.21.15.45:0/4226635154 conn(0x5573210b8200 msgr2=0x55732113f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.335+0000 7efc81bebb80 1 -- 172.21.15.45:0/4226635154 shutdown_connections 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.336+0000 7efc81bebb80 1 -- 172.21.15.45:0/4226635154 wait complete. 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.336+0000 7efc81bebb80 1 -- start start 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.336+0000 7efc81bebb80 1 Processor -- start 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321142f40 0x55732114e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x55732114bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc81bebb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557321096dd0 con 0x557321143860 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc81bebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573210336b0 con 0x55732114e710 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc81bebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557321033350 con 0x557321142f40 2023-09-23T09:33:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc81bebb80 10 client.0 _ll_register_callbacks cb 0x557320e907a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321142f40 0x55732114e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34132/0 (socket says 172.21.15.45:34132) 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x55732114bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6e7fc700 1 -- 172.21.15.45:0/2731083843 learned_addr learned my addr 172.21.15.45:0/2731083843 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6f7fe700 1 -- 172.21.15.45:0/2731083843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321142f40 msgr2=0x55732114e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321142f40 0x55732114e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6f7fe700 1 -- 172.21.15.45:0/2731083843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 msgr2=0x55732114b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6f7fe700 1 -- 172.21.15.45:0/2731083843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc68009160 con 0x55732114e710 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x55732114bb10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc68009cd0 tx=0x7efc68004910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a560b26a4a9d3d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc4effd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.337+0000 7efc4effd700 1 -- 172.21.15.45:0/2731083843 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68004db0 con 0x55732114e710 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc4effd700 1 -- 172.21.15.45:0/2731083843 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efc68007b20 con 0x55732114e710 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557321137090 con 0x55732114e710 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc4effd700 1 -- 172.21.15.45:0/2731083843 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc68003e50 con 0x55732114e710 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 10 client.25679 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7efc701a6280 con 0x55732114e710 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc4effd700 1 -- 172.21.15.45:0/2731083843 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557321137090 con 0x55732114e710 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc4effd700 1 -- 172.21.15.45:0/2731083843 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efc6801c030 con 0x55732114e710 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 10 client.25679 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 20 client.25679 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55732115e2d0 0x5573211606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 4 client.25679 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55732108fdb0 con 0x55732115e2d0 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.338+0000 7efc6effd700 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55732115e2d0 0x5573211606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.339+0000 7efc4effd700 10 client.25679 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.339+0000 7efc6effd700 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55732115e2d0 0x5573211606f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83fd5f92cf5f4ee server_cookie=1502817bdafc390c in_seq=0 out_seq=0 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc4effd700 1 -- 172.21.15.45:0/2731083843 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55732108fdb0 con 0x55732115e2d0 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc4effd700 10 client.25679 handle_command_reply: tid=0 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 client.25679 shutdown 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 msgr2=0x55732114bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x55732114bb10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc68009cd0 tx=0x7efc68004910 comp rx=0 tx=0).stop 2023-09-23T09:33:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 shutdown_connections 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55732115e2d0 0x5573211606f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x55732114bb10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc4effd700 0 client.25679 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321142f40 0x55732114e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 --2- 172.21.15.45:0/2731083843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x55732114b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 >> 172.21.15.45:0/2731083843 conn(0x5573210b8200 msgr2=0x55732113ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 shutdown_connections 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 1 -- 172.21.15.45:0/2731083843 wait complete. 2023-09-23T09:33:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.340+0000 7efc81bebb80 20 client.25679 trim_cache size 0 max 16384 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 Processor -- start 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 -- start start 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321142f40 0x55732114ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321143860 0x557321155260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x557321146710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557320f178c0 con 0x557321142f40 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573211557a0 con 0x55732114e710 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc81bebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557321149970 con 0x557321143860 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321142f40 0x55732114ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321143860 0x557321155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321142f40 0x55732114ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34138/0 (socket says 172.21.15.45:34138) 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc6effd700 1 -- 172.21.15.45:0/1284265249 learned_addr learned my addr 172.21.15.45:0/1284265249 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.355+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x557321146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc6effd700 1 -- 172.21.15.45:0/1284265249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321143860 msgr2=0x557321155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc6effd700 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321143860 0x557321155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc6effd700 1 -- 172.21.15.45:0/1284265249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 msgr2=0x557321146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc6effd700 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x557321146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc6effd700 1 -- 172.21.15.45:0/1284265249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557320ec1d80 con 0x557321142f40 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321143860 0x557321155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc6effd700 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321142f40 0x55732114ba70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efc600098f0 tx=0x7efc60023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c529bded5a58059 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc567fc700 1 -- 172.21.15.45:0/1284265249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc600050b0 con 0x557321142f40 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc567fc700 1 -- 172.21.15.45:0/1284265249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6001d040 con 0x557321142f40 2023-09-23T09:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc567fc700 1 -- 172.21.15.45:0/1284265249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60003cc0 con 0x557321142f40 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc81bebb80 1 -- 172.21.15.45:0/1284265249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321142f40 msgr2=0x55732114ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.356+0000 7efc81bebb80 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321142f40 0x55732114ba70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efc600098f0 tx=0x7efc60023000 comp rx=0 tx=0).stop 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- 172.21.15.45:0/1284265249 shutdown_connections 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55732114e710 0x557321146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321142f40 0x55732114ba70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 --2- 172.21.15.45:0/1284265249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557321143860 0x557321155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- 172.21.15.45:0/1284265249 >> 172.21.15.45:0/1284265249 conn(0x5573210b8200 msgr2=0x5573211599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- 172.21.15.45:0/1284265249 shutdown_connections 2023-09-23T09:33:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- 172.21.15.45:0/1284265249 wait complete. 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- start start 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 Processor -- start 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321154670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x557321154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x5573211b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573211b1640 con 0x557321143860 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573211b1780 con 0x557321142f40 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573211b18f0 con 0x55732114e710 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc81bebb80 10 client.0 _ll_register_callbacks cb 0x557320e907a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.357+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x557321154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x557321154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34146/0 (socket says 172.21.15.45:34146) 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6e7fc700 1 -- 172.21.15.45:0/3743748900 learned_addr learned my addr 172.21.15.45:0/3743748900 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x5573211b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6effd700 1 -- 172.21.15.45:0/3743748900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 msgr2=0x5573211b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6effd700 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x5573211b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6effd700 1 -- 172.21.15.45:0/3743748900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 msgr2=0x557321154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6effd700 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x557321154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6effd700 1 -- 172.21.15.45:0/3743748900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573210f9870 con 0x557321142f40 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6f7fe700 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x5573211b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc6effd700 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321154670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc60000c00 tx=0x7efc60005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c2fffb7b5e7cef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc4effd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc4effd700 1 -- 172.21.15.45:0/3743748900 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6003f070 con 0x557321142f40 2023-09-23T09:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc6409ab90 con 0x557321142f40 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc4effd700 1 -- 172.21.15.45:0/3743748900 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efc6000ba60 con 0x557321142f40 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.358+0000 7efc4effd700 1 -- 172.21.15.45:0/3743748900 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc6000bc20 con 0x557321142f40 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 10 client.25685 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7efc70068210 con 0x557321142f40 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc4effd700 1 -- 172.21.15.45:0/3743748900 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6409ab90 con 0x557321142f40 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc4effd700 1 -- 172.21.15.45:0/3743748900 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efc6003a440 con 0x557321142f40 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 10 client.25685 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 20 client.25685 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573211b2c80 0x5573211b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 4 client.25685 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55732108fad0 con 0x5573211b2c80 2023-09-23T09:33:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.359+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573211b2c80 0x5573211b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.360+0000 7efc4effd700 10 client.25685 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.360+0000 7efc6e7fc700 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573211b2c80 0x5573211b50a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824c72f948f0f82c server_cookie=8dfeef95804d1ee9 in_seq=0 out_seq=0 2023-09-23T09:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc4effd700 1 -- 172.21.15.45:0/3743748900 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55732108fad0 con 0x5573211b2c80 2023-09-23T09:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc4effd700 10 client.25685 handle_command_reply: tid=0 2023-09-23T09:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 client.25685 shutdown 2023-09-23T09:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 msgr2=0x557321154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321154670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc60000c00 tx=0x7efc60005e50 comp rx=0 tx=0).stop 2023-09-23T09:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 shutdown_connections 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573211b2c80 0x5573211b50a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55732114e710 0x5573211b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc4effd700 0 client.25685 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557321142f40 0x557321154670 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 --2- 172.21.15.45:0/3743748900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557321143860 0x557321154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 >> 172.21.15.45:0/3743748900 conn(0x5573210b8200 msgr2=0x557321146710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 shutdown_connections 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 1 -- 172.21.15.45:0/3743748900 wait complete. 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.375+0000 7efc81bebb80 20 client.25685 trim_cache size 0 max 16384 2023-09-23T09:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:33:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:33:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:33:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:33:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efc580a1790 msgr2=0x7efc580a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efc580a1790 0x7efc580a3c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc60019040 comp rx=0 tx=0).stop 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70105050 msgr2=0x7efc7010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70105050 0x7efc7010dc00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e970 tx=0x7efc6400ec80 comp rx=0 tx=0).stop 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 shutdown_connections 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc701a30d0 0x7efc701a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7efc580a1790 0x7efc580a3c50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70100160 0x7efc7010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 --2- 172.21.15.45:0/2438528441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc70105050 0x7efc7010dc00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 >> 172.21.15.45:0/2438528441 conn(0x7efc70076840 msgr2=0x7efc700fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 shutdown_connections 2023-09-23T09:33:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:22.376+0000 7efc75a01700 1 -- 172.21.15.45:0/2438528441 wait complete. 2023-09-23T09:33:23.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:23 smithi045 ceph-mon[98627]: pgmap v122: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2023-09-23T09:33:23.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:23 smithi123 ceph-mon[103577]: pgmap v122: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2023-09-23T09:33:24.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:23 smithi098 ceph-mon[102386]: pgmap v122: 97 pgs: 97 active+clean; 5.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2023-09-23T09:33:25.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:25 smithi045 ceph-mon[98627]: pgmap v123: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2023-09-23T09:33:25.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:25 smithi123 ceph-mon[103577]: pgmap v123: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2023-09-23T09:33:26.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:25 smithi098 ceph-mon[102386]: pgmap v123: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2023-09-23T09:33:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 220.0 s, 0.0 tps, lat -nan ms stddev -nan 2023-09-23T09:33:27.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:27 smithi045 ceph-mon[98627]: pgmap v124: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-23T09:33:27.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:27 smithi123 ceph-mon[103577]: pgmap v124: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-23T09:33:28.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:27 smithi098 ceph-mon[102386]: pgmap v124: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-23T09:33:29.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:33:29.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 -- 172.21.15.45:0/1273955939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 msgr2=0x7f35f010cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1273955939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 0x7f35f010cb70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f35e800b0a0 tx=0x7f35e800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 -- 172.21.15.45:0/1273955939 shutdown_connections 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1273955939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 0x7f35f010cb70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1273955939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 0x7f35f01002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1273955939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0100880 0x7f35f01048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 -- 172.21.15.45:0/1273955939 >> 172.21.15.45:0/1273955939 conn(0x7f35f00fb770 msgr2=0x7f35f00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 -- 172.21.15.45:0/1273955939 shutdown_connections 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 -- 172.21.15.45:0/1273955939 wait complete. 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 Processor -- start 2023-09-23T09:33:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.474+0000 7f35f64f8700 1 -- start start 2023-09-23T09:33:29.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f64f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 0x7f35f0195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f64f8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 0x7f35f0196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f64f8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f019a850 0x7f35f019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f64f8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f019b2d0 con 0x7f35f0104ff0 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f64f8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f019b440 con 0x7f35f019a850 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f64f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f019b600 con 0x7f35f00ffed0 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 0x7f35f0195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 0x7f35f0196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 0x7f35f0195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36264/0 (socket says 172.21.15.45:36264) 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f4a95700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f019a850 0x7f35f019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35effff700 1 -- 172.21.15.45:0/1163978633 learned_addr learned my addr 172.21.15.45:0/1163978633 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35ef7fe700 1 -- 172.21.15.45:0/1163978633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 msgr2=0x7f35f0195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 0x7f35f0195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35ef7fe700 1 -- 172.21.15.45:0/1163978633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f019a850 msgr2=0x7f35f019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f019a850 0x7f35f019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35ef7fe700 1 -- 172.21.15.45:0/1163978633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e8009d00 con 0x7f35f0104ff0 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35effff700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 0x7f35f0195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35f4a95700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f019a850 0x7f35f019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.475+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 0x7f35f0196490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f35e400e9c0 tx=0x7f35e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.476+0000 7f35ed7fa700 1 -- 172.21.15.45:0/1163978633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400c9c0 con 0x7f35f0104ff0 2023-09-23T09:33:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.476+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f01a2b60 con 0x7f35f0104ff0 2023-09-23T09:33:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.476+0000 7f35ed7fa700 1 -- 172.21.15.45:0/1163978633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e4010430 con 0x7f35f0104ff0 2023-09-23T09:33:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.476+0000 7f35ed7fa700 1 -- 172.21.15.45:0/1163978633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4018680 con 0x7f35f0104ff0 2023-09-23T09:33:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.476+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f01a3040 con 0x7f35f0104ff0 2023-09-23T09:33:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.478+0000 7f35ed7fa700 1 -- 172.21.15.45:0/1163978633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f35e400cb20 con 0x7f35f0104ff0 2023-09-23T09:33:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.478+0000 7f35ed7fa700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f35d80a1710 0x7f35d80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.479+0000 7f35ed7fa700 1 -- 172.21.15.45:0/1163978633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f35e4014070 con 0x7f35f0104ff0 2023-09-23T09:33:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.479+0000 7f35effff700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f35d80a1710 0x7f35d80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.479+0000 7f35effff700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f35d80a1710 0x7f35d80a3bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35e0011550 tx=0x7f35e0009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.479+0000 7f36026e2b80 1 Processor -- start 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.479+0000 7f36026e2b80 1 -- start start 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.479+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4400 0x55b70a1c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.479+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1d41d0 0x55b70a1d65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f36026e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a12d460 con 0x55b70a1c4d20 2023-09-23T09:33:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f36026e2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a11ec20 con 0x55b70a1c4400 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f36026e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a11c540 con 0x55b70a1d41d0 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34168/0 (socket says 172.21.15.45:34168) 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 -- 172.21.15.45:0/81649340 learned_addr learned my addr 172.21.15.45:0/81649340 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35f4a95700 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1d41d0 0x55b70a1d65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35effff700 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4400 0x55b70a1c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 -- 172.21.15.45:0/81649340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1d41d0 msgr2=0x55b70a1d65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1d41d0 0x55b70a1d65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 -- 172.21.15.45:0/81649340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4400 msgr2=0x55b70a1c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4400 0x55b70a1c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 -- 172.21.15.45:0/81649340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70a17a8d0 con 0x55b70a1c4d20 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d3c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f35e40a21d0 tx=0x7f35e4099f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3d28a3537ad20d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35d67fc700 1 -- 172.21.15.45:0/81649340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e409a370 con 0x55b70a1c4d20 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35d67fc700 1 -- 172.21.15.45:0/81649340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e409e4c0 con 0x55b70a1c4d20 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.480+0000 7f35d67fc700 1 -- 172.21.15.45:0/81649340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e409e120 con 0x55b70a1c4d20 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 -- 172.21.15.45:0/81649340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 msgr2=0x55b70a1d3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d3c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f35e40a21d0 tx=0x7f35e4099f80 comp rx=0 tx=0).stop 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 -- 172.21.15.45:0/81649340 shutdown_connections 2023-09-23T09:33:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1d41d0 0x55b70a1d65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4400 0x55b70a1c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 --2- 172.21.15.45:0/81649340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d3c90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 -- 172.21.15.45:0/81649340 >> 172.21.15.45:0/81649340 conn(0x55b70a139260 msgr2=0x55b70a1c0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 -- 172.21.15.45:0/81649340 shutdown_connections 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 -- 172.21.15.45:0/81649340 wait complete. 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.481+0000 7f36026e2b80 1 -- start start 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 1 Processor -- start 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4400 0x55b70a1d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a117e30 con 0x55b70a1c4400 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a0b4710 con 0x55b70a1d41d0 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a0b43b0 con 0x55b70a1c4d20 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f36026e2b80 10 client.0 _ll_register_callbacks cb 0x55b709f11800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35effff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4400 0x55b70a1d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36280/0 (socket says 172.21.15.45:36280) 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35ef7fe700 1 -- 172.21.15.45:0/2843439906 learned_addr learned my addr 172.21.15.45:0/2843439906 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35f4a95700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35effff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4400 0x55b70a1d47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34172/0 (socket says 172.21.15.45:34172) 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35effff700 1 -- 172.21.15.45:0/2843439906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4d20 msgr2=0x55b70a1d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35effff700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35effff700 1 -- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 msgr2=0x55b70a1df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35effff700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1df630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35effff700 1 -- 172.21.15.45:0/2843439906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f01a3040 con 0x55b70a1c4400 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35f4a95700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.482+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f35effff700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4400 0x55b70a1d47a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009cf0 tx=0x7f35e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277f48720cddf2c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f35cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f35cf7fe700 1 -- 172.21.15.45:0/2843439906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0022070 con 0x55b70a1c4400 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f35cf7fe700 1 -- 172.21.15.45:0/2843439906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f35e001dad0 con 0x55b70a1c4400 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b70a1b8250 con 0x55b70a1c4400 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f35cf7fe700 1 -- 172.21.15.45:0/2843439906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e001dc90 con 0x55b70a1c4400 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 10 client.16029 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b709f42de0 con 0x55b70a1c4400 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f35cf7fe700 1 -- 172.21.15.45:0/2843439906 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e0033020 con 0x55b70a1c4400 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f35cf7fe700 1 -- 172.21.15.45:0/2843439906 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f35e002e450 con 0x55b70a1c4400 2023-09-23T09:33:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 20 client.16029 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a1e3bf0 0x55b70a1e6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 4 client.16029 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.483+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b70a110e10 con 0x55b70a1e3bf0 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.484+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a1e3bf0 0x55b70a1e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.484+0000 7f35cf7fe700 10 client.16029 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.484+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a1e3bf0 0x55b70a1e6010 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f202ad675388715 server_cookie=b7bdda98c3c044e4 in_seq=0 out_seq=0 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f35cf7fe700 1 -- 172.21.15.45:0/2843439906 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b70a110e10 con 0x55b70a1e3bf0 2023-09-23T09:33:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f35cf7fe700 10 client.16029 handle_command_reply: tid=0 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 client.16029 shutdown 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4400 msgr2=0x55b70a1d47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4400 0x55b70a1d47a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009cf0 tx=0x7f35e00047c0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 shutdown_connections 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a1e3bf0 0x55b70a1e6010 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f35cf7fe700 0 client.16029 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4400 0x55b70a1d47a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 --2- 172.21.15.45:0/2843439906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 >> 172.21.15.45:0/2843439906 conn(0x55b70a139260 msgr2=0x55b70a1c0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 shutdown_connections 2023-09-23T09:33:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 1 -- 172.21.15.45:0/2843439906 wait complete. 2023-09-23T09:33:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.485+0000 7f36026e2b80 20 client.16029 trim_cache size 0 max 16384 2023-09-23T09:33:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 Processor -- start 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 -- start start 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1d41d0 0x55b70a1ded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b709f98920 con 0x55b70a1d41d0 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a1d5030 con 0x55b70a1c4d20 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f36026e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a1e11a0 con 0x55b70a1c4400 2023-09-23T09:33:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f35f4a95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1d41d0 0x55b70a1ded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f35f4a95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1d41d0 0x55b70a1ded50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34184/0 (socket says 172.21.15.45:34184) 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f35f4a95700 1 -- 172.21.15.45:0/1582417317 learned_addr learned my addr 172.21.15.45:0/1582417317 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.502+0000 7f35effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35f4a95700 1 -- 172.21.15.45:0/1582417317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 msgr2=0x55b70a1d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35f4a95700 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35f4a95700 1 -- 172.21.15.45:0/1582417317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4d20 msgr2=0x55b70a1d4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35f4a95700 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35f4a95700 1 -- 172.21.15.45:0/1582417317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70a1bc480 con 0x55b70a1d41d0 2023-09-23T09:33:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35f4a95700 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1d41d0 0x55b70a1ded50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f35e8015000 tx=0x7f35e800bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb59bc04f35fc14 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35d6ffd700 1 -- 172.21.15.45:0/1582417317 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800e040 con 0x55b70a1d41d0 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35d6ffd700 1 -- 172.21.15.45:0/1582417317 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e800be30 con 0x55b70a1d41d0 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.503+0000 7f35d6ffd700 1 -- 172.21.15.45:0/1582417317 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e80055e0 con 0x55b70a1d41d0 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 -- 172.21.15.45:0/1582417317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1d41d0 msgr2=0x55b70a1ded50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1d41d0 0x55b70a1ded50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f35e8015000 tx=0x7f35e800bc90 comp rx=0 tx=0).stop 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 -- 172.21.15.45:0/1582417317 shutdown_connections 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1d41d0 0x55b70a1ded50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 --2- 172.21.15.45:0/1582417317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1c4d20 0x55b70a1d4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 -- 172.21.15.45:0/1582417317 >> 172.21.15.45:0/1582417317 conn(0x55b70a139260 msgr2=0x55b70a1ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 -- 172.21.15.45:0/1582417317 shutdown_connections 2023-09-23T09:33:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 -- 172.21.15.45:0/1582417317 wait complete. 2023-09-23T09:33:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 -- start start 2023-09-23T09:33:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.504+0000 7f36026e2b80 1 Processor -- start 2023-09-23T09:33:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f36026e2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f36026e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a1d7a40 con 0x55b70a1c4d20 2023-09-23T09:33:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f36026e2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a1d7bb0 con 0x55b70a1d41d0 2023-09-23T09:33:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f36026e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a1d7d70 con 0x55b70a1c4400 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f36026e2b80 10 client.0 _ll_register_callbacks cb 0x7f35e000e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36292/0 (socket says 172.21.15.45:36292) 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34188/0 (socket says 172.21.15.45:34188) 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35effff700 1 -- 172.21.15.45:0/268764235 learned_addr learned my addr 172.21.15.45:0/268764235 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35f4a95700 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 -- 172.21.15.45:0/268764235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 msgr2=0x55b70a1d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 -- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 msgr2=0x55b70a1d7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1d7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 -- 172.21.15.45:0/268764235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f010d3e0 con 0x55b70a1c4d20 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35effff700 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35ef7fe700 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d94f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f35e40cd3e0 tx=0x7f35e4009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86826e98a04e5719 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.505+0000 7f35cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f35cf7fe700 1 -- 172.21.15.45:0/268764235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e409a3f0 con 0x55b70a1c4d20 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f35cf7fe700 1 -- 172.21.15.45:0/268764235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f35e4098710 con 0x55b70a1c4d20 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35e409e4c0 con 0x55b70a1c4d20 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f35cf7fe700 1 -- 172.21.15.45:0/268764235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e40105a0 con 0x55b70a1c4d20 2023-09-23T09:33:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 10 client.16035 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b709f2bc60 con 0x55b70a1c4d20 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f35cf7fe700 1 -- 172.21.15.45:0/268764235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e407c020 con 0x55b70a1c4d20 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f35cf7fe700 1 -- 172.21.15.45:0/268764235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f35e4099860 con 0x55b70a1c4d20 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 20 client.16035 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a2389a0 0x55b70a23adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 4 client.16035 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b70a110b30 con 0x55b70a2389a0 2023-09-23T09:33:29.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.506+0000 7f35effff700 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a2389a0 0x55b70a23adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f35cf7fe700 10 client.16035 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f35effff700 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a2389a0 0x55b70a23adc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929efef5be6ff70d server_cookie=bfe0d89305d4e726 in_seq=0 out_seq=0 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f35cf7fe700 1 -- 172.21.15.45:0/268764235 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b70a110b30 con 0x55b70a2389a0 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f35cf7fe700 10 client.16035 handle_command_reply: tid=0 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 client.16035 shutdown 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 msgr2=0x55b70a1d94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d94f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f35e40cd3e0 tx=0x7f35e4009790 comp rx=0 tx=0).stop 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 shutdown_connections 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b70a2389a0 0x55b70a23adc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b70a1d41d0 0x55b70a1d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b70a1c4400 0x55b70a1d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 --2- 172.21.15.45:0/268764235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70a1c4d20 0x55b70a1d94f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 >> 172.21.15.45:0/268764235 conn(0x55b70a139260 msgr2=0x55b70a1ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f35cf7fe700 0 client.16035 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 shutdown_connections 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 1 -- 172.21.15.45:0/268764235 wait complete. 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.507+0000 7f36026e2b80 20 client.16035 trim_cache size 0 max 16384 2023-09-23T09:33:29.518 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:33:29.519 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:33:29.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.508+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f35d80a1710 msgr2=0x7f35d80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.508+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f35d80a1710 0x7f35d80a3bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35e0011550 tx=0x7f35e0009260 comp rx=0 tx=0).stop 2023-09-23T09:33:29.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.508+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 msgr2=0x7f35f0196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:29.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.508+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 0x7f35f0196490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f35e400e9c0 tx=0x7f35e400ecd0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.522+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 shutdown_connections 2023-09-23T09:33:29.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.522+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f019a850 0x7f35f019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.522+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f00ffed0 0x7f35f0195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.522+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f35d80a1710 0x7f35d80a3bd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.522+0000 7f35f64f8700 1 --2- 172.21.15.45:0/1163978633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f0104ff0 0x7f35f0196490 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:29.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.522+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 >> 172.21.15.45:0/1163978633 conn(0x7f35f00fb770 msgr2=0x7f35f00fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:29.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.525+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 shutdown_connections 2023-09-23T09:33:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:29.527+0000 7f35f64f8700 1 -- 172.21.15.45:0/1163978633 wait complete. 2023-09-23T09:33:29.541 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:33:29.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:29 smithi045 ceph-mon[98627]: pgmap v125: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-09-23T09:33:29.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:29 smithi123 ceph-mon[103577]: pgmap v125: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-09-23T09:33:30.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:29 smithi098 ceph-mon[102386]: pgmap v125: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-09-23T09:33:31.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:31 smithi045 ceph-mon[98627]: pgmap v126: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.4 MiB/s wr, 316 op/s 2023-09-23T09:33:31.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:31 smithi123 ceph-mon[103577]: pgmap v126: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.4 MiB/s wr, 316 op/s 2023-09-23T09:33:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 225.0 s, 1073.5 tps, lat 303.167 ms stddev 1988.454 2023-09-23T09:33:32.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:31 smithi098 ceph-mon[102386]: pgmap v126: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.4 MiB/s wr, 316 op/s 2023-09-23T09:33:33.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:33 smithi123 ceph-mon[103577]: pgmap v127: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-09-23T09:33:34.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:33 smithi098 ceph-mon[102386]: pgmap v127: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-09-23T09:33:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:33 smithi045 ceph-mon[98627]: pgmap v127: 97 pgs: 97 active+clean; 5.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-09-23T09:33:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 -- 172.21.15.45:0/626037372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c0ffcd0 msgr2=0x7f8e7c100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 --2- 172.21.15.45:0/626037372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c100150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00d960 tx=0x7f8e6c00dc70 comp rx=0 tx=0).stop 2023-09-23T09:33:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 -- 172.21.15.45:0/626037372 shutdown_connections 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 --2- 172.21.15.45:0/626037372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100690 0x7f8e7c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 --2- 172.21.15.45:0/626037372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e7c105d90 0x7f8e7c106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 --2- 172.21.15.45:0/626037372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c100150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 -- 172.21.15.45:0/626037372 >> 172.21.15.45:0/626037372 conn(0x7f8e7c076e20 msgr2=0x7f8e7c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 -- 172.21.15.45:0/626037372 shutdown_connections 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.537+0000 7f8e83cad700 1 -- 172.21.15.45:0/626037372 wait complete. 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 Processor -- start 2023-09-23T09:33:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 -- start start 2023-09-23T09:33:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e7c100690 0x7f8e7c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c105d90 0x7f8e7c194b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1951a0 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c195310 con 0x7f8e7c105d90 2023-09-23T09:33:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e83cad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1954d0 con 0x7f8e7c100690 2023-09-23T09:33:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e81a49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e81248700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e7c100690 0x7f8e7c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e81a49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34200/0 (socket says 172.21.15.45:34200) 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e81a49700 1 -- 172.21.15.45:0/3780712901 learned_addr learned my addr 172.21.15.45:0/3780712901 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.538+0000 7f8e8224a700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c105d90 0x7f8e7c194b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.539+0000 7f8e81a49700 1 -- 172.21.15.45:0/3780712901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e7c100690 msgr2=0x7f8e7c19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.539+0000 7f8e81a49700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e7c100690 0x7f8e7c19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.539+0000 7f8e81a49700 1 -- 172.21.15.45:0/3780712901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c105d90 msgr2=0x7f8e7c194b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.539+0000 7f8e81a49700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c105d90 0x7f8e7c194b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.539+0000 7f8e81a49700 1 -- 172.21.15.45:0/3780712901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c00d590 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.539+0000 7f8e81a49700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c19acd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8e78011b50 tx=0x7f8e78011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.541+0000 7f8e72ffd700 1 -- 172.21.15.45:0/3780712901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800fa10 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.541+0000 7f8e72ffd700 1 -- 172.21.15.45:0/3780712901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e78013040 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.541+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c1957b0 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.541+0000 7f8e72ffd700 1 -- 172.21.15.45:0/3780712901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e780183a0 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.541+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c1a1d40 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e8fe97b80 1 Processor -- start 2023-09-23T09:33:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e72ffd700 1 -- 172.21.15.45:0/3780712901 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f8e78019400 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e8fe97b80 1 -- start start 2023-09-23T09:33:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e79320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e8c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8de2400 con 0x55d8a8e79860 2023-09-23T09:33:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.546+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8dd3bc0 con 0x55d8a8e78f40 2023-09-23T09:33:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8dd14e0 con 0x55d8a8e8cf20 2023-09-23T09:33:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e8224a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e79320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36312/0 (socket says 172.21.15.45:36312) 2023-09-23T09:33:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 -- 172.21.15.45:0/967538257 learned_addr learned my addr 172.21.15.45:0/967538257 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e72ffd700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8e680a1860 0x7f8e680a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81a49700 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e8c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e8224a700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8e680a1860 0x7f8e680a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 -- 172.21.15.45:0/967538257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 msgr2=0x55d8a8e79320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e79320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 -- 172.21.15.45:0/967538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 msgr2=0x55d8a8e8c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e72ffd700 1 -- 172.21.15.45:0/3780712901 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8e780d0900 con 0x7f8e7c0ffcd0 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e8c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 -- 172.21.15.45:0/967538257 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a8e2f870 con 0x55d8a8e8cf20 2023-09-23T09:33:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81a49700 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e8c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:33:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.547+0000 7f8e81248700 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8f310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c015000 tx=0x7f8e6c009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a9d23f2bae4e30 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.548+0000 7f8e8224a700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8e680a1860 0x7f8e680a3d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009fd0 tx=0x7f8e74004ad0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.548+0000 7f8e67fff700 1 -- 172.21.15.45:0/967538257 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c01b050 con 0x55d8a8e8cf20 2023-09-23T09:33:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.548+0000 7f8e67fff700 1 -- 172.21.15.45:0/967538257 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e6c01c040 con 0x55d8a8e8cf20 2023-09-23T09:33:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.548+0000 7f8e67fff700 1 -- 172.21.15.45:0/967538257 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c016d10 con 0x55d8a8e8cf20 2023-09-23T09:33:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/967538257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 msgr2=0x55d8a8e8f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8f310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c015000 tx=0x7f8e6c009720 comp rx=0 tx=0).stop 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/967538257 shutdown_connections 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8f310 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e79320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/967538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e8c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/967538257 >> 172.21.15.45:0/967538257 conn(0x55d8a8dee200 msgr2=0x55d8a8e753c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.549+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/967538257 shutdown_connections 2023-09-23T09:33:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.550+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/967538257 wait complete. 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.550+0000 7f8e8fe97b80 1 -- start start 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.550+0000 7f8e8fe97b80 1 Processor -- start 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.550+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e86280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.550+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e79860 0x55d8a8e867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.550+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.550+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8dccdd0 con 0x55d8a8e8cf20 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8d696b0 con 0x55d8a8e78f40 2023-09-23T09:33:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8d69350 con 0x55d8a8e79860 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e8fe97b80 10 client.0 _ll_register_callbacks cb 0x55d8a8bc67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e86280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e86280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55564/0 (socket says 172.21.15.45:55564) 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 -- 172.21.15.45:0/4043440980 learned_addr learned my addr 172.21.15.45:0/4043440980 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e8224a700 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 -- 172.21.15.45:0/4043440980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e79860 msgr2=0x55d8a8e867c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e79860 0x55d8a8e867c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 -- 172.21.15.45:0/4043440980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e8cf20 msgr2=0x55d8a8e8bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 -- 172.21.15.45:0/4043440980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c10d420 con 0x55d8a8e78f40 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e81a49700 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e86280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e78099d60 tx=0x7f8e78027560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ba34dd64e0a03fa server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e657fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.551+0000 7f8e657fa700 1 -- 172.21.15.45:0/4043440980 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e780a19b0 con 0x55d8a8e78f40 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e657fa700 1 -- 172.21.15.45:0/4043440980 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8e780a5bf0 con 0x55d8a8e78f40 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8a8e6d090 con 0x55d8a8e78f40 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e657fa700 1 -- 172.21.15.45:0/4043440980 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e78099f70 con 0x55d8a8e78f40 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 10 client.25715 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d8a8bf7d80 con 0x55d8a8e78f40 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e657fa700 1 -- 172.21.15.45:0/4043440980 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8a8e6d090 con 0x55d8a8e78f40 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e657fa700 1 -- 172.21.15.45:0/4043440980 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e7809dac0 con 0x55d8a8e78f40 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 10 client.25715 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 20 client.25715 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8ea3140 0x55d8a8ea55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 4 client.25715 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8a8dc5db0 con 0x55d8a8ea3140 2023-09-23T09:33:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.552+0000 7f8e81248700 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8ea3140 0x55d8a8ea55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.553+0000 7f8e81248700 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8ea3140 0x55d8a8ea55a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0ceb96644be271 server_cookie=443bc3a74e664fdc in_seq=0 out_seq=0 2023-09-23T09:33:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.553+0000 7f8e657fa700 10 client.25715 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e657fa700 1 -- 172.21.15.45:0/4043440980 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8a8dc5db0 con 0x55d8a8ea3140 2023-09-23T09:33:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e657fa700 10 client.25715 handle_command_reply: tid=0 2023-09-23T09:33:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 client.25715 shutdown 2023-09-23T09:33:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 msgr2=0x55d8a8e86280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e86280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e78099d60 tx=0x7f8e78027560 comp rx=0 tx=0).stop 2023-09-23T09:33:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 shutdown_connections 2023-09-23T09:33:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8ea3140 0x55d8a8ea55a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e78f40 0x55d8a8e86280 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e78099d60 tx=0x7f8e78027560 comp rx=0 tx=0).stop 2023-09-23T09:33:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/4043440980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e79860 0x55d8a8e867c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 >> 172.21.15.45:0/4043440980 conn(0x55d8a8dee200 msgr2=0x55d8a8e74f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.554+0000 7f8e657fa700 0 client.25715 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.556+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 shutdown_connections 2023-09-23T09:33:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.556+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/4043440980 wait complete. 2023-09-23T09:33:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.556+0000 7f8e8fe97b80 20 client.25715 trim_cache size 0 max 16384 2023-09-23T09:33:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 Processor -- start 2023-09-23T09:33:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 -- start start 2023-09-23T09:33:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e79320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e7bde0 0x55d8a8e73af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8c4d8c0 con 0x55d8a8e79860 2023-09-23T09:33:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8e82b30 con 0x55d8a8e8cf20 2023-09-23T09:33:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.581+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8e81760 con 0x55d8a8e7bde0 2023-09-23T09:33:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e81248700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e7bde0 0x55d8a8e73af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e7bde0 0x55d8a8e73af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36328/0 (socket says 172.21.15.45:36328) 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 -- 172.21.15.45:0/1633619509 learned_addr learned my addr 172.21.15.45:0/1633619509 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 -- 172.21.15.45:0/1633619509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e8cf20 msgr2=0x55d8a8e8bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 --2- 172.21.15.45:0/1633619509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 -- 172.21.15.45:0/1633619509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 msgr2=0x55d8a8e79320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 --2- 172.21.15.45:0/1633619509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e79320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.582+0000 7f8e8224a700 1 -- 172.21.15.45:0/1633619509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a8be0c00 con 0x55d8a8e7bde0 2023-09-23T09:33:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.583+0000 7f8e8224a700 1 --2- 172.21.15.45:0/1633619509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e7bde0 0x55d8a8e73af0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400af40 tx=0x7f8e74017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=964c001fd60acef6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.584+0000 7f8e70d86700 1 -- 172.21.15.45:0/1633619509 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74015d70 con 0x55d8a8e7bde0 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.584+0000 7f8e70d86700 1 -- 172.21.15.45:0/1633619509 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7401f050 con 0x55d8a8e7bde0 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.584+0000 7f8e70d86700 1 -- 172.21.15.45:0/1633619509 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e740196a0 con 0x55d8a8e7bde0 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.584+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1633619509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e7bde0 msgr2=0x55d8a8e73af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.584+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1633619509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e7bde0 0x55d8a8e73af0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400af40 tx=0x7f8e74017000 comp rx=0 tx=0).stop 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1633619509 shutdown_connections 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1633619509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e7bde0 0x55d8a8e73af0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1633619509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e79860 0x55d8a8e79320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1633619509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e8bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1633619509 >> 172.21.15.45:0/1633619509 conn(0x55d8a8dee200 msgr2=0x55d8a8ea9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1633619509 shutdown_connections 2023-09-23T09:33:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1633619509 wait complete. 2023-09-23T09:33:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 -- start start 2023-09-23T09:33:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.585+0000 7f8e8fe97b80 1 Processor -- start 2023-09-23T09:33:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.586+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e79860 0x55d8a8eb32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.586+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e7e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.586+0000 7f8e8fe97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e7efa0 0x55d8a8e7d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.586+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8e7f5f0 con 0x55d8a8e7efa0 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.586+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8e7dd70 con 0x55d8a8e79860 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.586+0000 7f8e8fe97b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8e7df30 con 0x55d8a8e8cf20 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.586+0000 7f8e8fe97b80 10 client.0 _ll_register_callbacks cb 0x7f8e7809d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.587+0000 7f8e81248700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e7e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.587+0000 7f8e81a49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e79860 0x55d8a8eb32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.587+0000 7f8e81a49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e79860 0x55d8a8eb32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55580/0 (socket says 172.21.15.45:55580) 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.587+0000 7f8e81a49700 1 -- 172.21.15.45:0/1860966875 learned_addr learned my addr 172.21.15.45:0/1860966875 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e81248700 1 -- 172.21.15.45:0/1860966875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e79860 msgr2=0x55d8a8eb32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e81248700 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e79860 0x55d8a8eb32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e81248700 1 -- 172.21.15.45:0/1860966875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e7efa0 msgr2=0x55d8a8e7d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e81248700 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e7efa0 0x55d8a8e7d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e81248700 1 -- 172.21.15.45:0/1860966875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e74013040 con 0x55d8a8e8cf20 2023-09-23T09:33:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e81248700 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e7e920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005510 tx=0x7f8e6c005570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8338bacff987af server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e657fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:33:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e657fa700 1 -- 172.21.15.45:0/1860966875 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c01b040 con 0x55d8a8e8cf20 2023-09-23T09:33:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e657fa700 1 -- 172.21.15.45:0/1860966875 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8e6c02e070 con 0x55d8a8e8cf20 2023-09-23T09:33:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.588+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e6c01c040 con 0x55d8a8e8cf20 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.589+0000 7f8e657fa700 1 -- 172.21.15.45:0/1860966875 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e6c00d020 con 0x55d8a8e8cf20 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.589+0000 7f8e8fe97b80 10 client.25723 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.589+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d8a8c1e1f0 con 0x55d8a8e8cf20 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.589+0000 7f8e657fa700 1 -- 172.21.15.45:0/1860966875 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e6c01c040 con 0x55d8a8e8cf20 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e657fa700 1 -- 172.21.15.45:0/1860966875 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e6c016ca0 con 0x55d8a8e8cf20 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e8fe97b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e8fe97b80 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e8fe97b80 20 client.25723 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8f1c570 0x55d8a8f1e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e8fe97b80 4 client.25723 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:33:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8a8dc5ad0 con 0x55d8a8f1c570 2023-09-23T09:33:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.590+0000 7f8e81a49700 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8f1c570 0x55d8a8f1e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.591+0000 7f8e81a49700 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8f1c570 0x55d8a8f1e990 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa2056b4d9c5825 server_cookie=ef7c228ebf94b07a in_seq=0 out_seq=0 2023-09-23T09:33:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.591+0000 7f8e657fa700 10 client.25723 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.611+0000 7f8e657fa700 1 -- 172.21.15.45:0/1860966875 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8a8dc5ad0 con 0x55d8a8f1c570 2023-09-23T09:33:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.611+0000 7f8e657fa700 10 client.25723 handle_command_reply: tid=0 2023-09-23T09:33:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.611+0000 7f8e8fe97b80 1 client.25723 shutdown 2023-09-23T09:33:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.611+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 msgr2=0x55d8a8e7e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.611+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e7e920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005510 tx=0x7f8e6c005570 comp rx=0 tx=0).stop 2023-09-23T09:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 shutdown_connections 2023-09-23T09:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d8a8f1c570 0x55d8a8f1e990 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a8e7efa0 0x55d8a8e7d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8a8e79860 0x55d8a8eb32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e8fe97b80 1 --2- 172.21.15.45:0/1860966875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8a8e8cf20 0x55d8a8e7e920 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 >> 172.21.15.45:0/1860966875 conn(0x55d8a8dee200 msgr2=0x55d8a8e7b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e657fa700 0 client.25723 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.612+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 shutdown_connections 2023-09-23T09:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.613+0000 7f8e8fe97b80 1 -- 172.21.15.45:0/1860966875 wait complete. 2023-09-23T09:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.613+0000 7f8e8fe97b80 20 client.25723 trim_cache size 0 max 16384 2023-09-23T09:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.613+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8e680a1860 msgr2=0x7f8e680a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.613+0000 7f8e83cad700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8e680a1860 0x7f8e680a3d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009fd0 tx=0x7f8e74004ad0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.613+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c0ffcd0 msgr2=0x7f8e7c19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.613+0000 7f8e83cad700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c19acd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8e78011b50 tx=0x7f8e78011e60 comp rx=0 tx=0).stop 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 shutdown_connections 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7c105d90 0x7f8e7c194b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8e680a1860 0x7f8e680a3d20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c0ffcd0 0x7f8e7c19acd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 --2- 172.21.15.45:0/3780712901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e7c100690 0x7f8e7c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 >> 172.21.15.45:0/3780712901 conn(0x7f8e7c076e20 msgr2=0x7f8e7c0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 shutdown_connections 2023-09-23T09:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:35.614+0000 7f8e83cad700 1 -- 172.21.15.45:0/3780712901 wait complete. 2023-09-23T09:33:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:35 smithi123 ceph-mon[103577]: pgmap v128: 97 pgs: 97 active+clean; 5.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.1 MiB/s wr, 434 op/s 2023-09-23T09:33:36.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:35 smithi098 ceph-mon[102386]: pgmap v128: 97 pgs: 97 active+clean; 5.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.1 MiB/s wr, 434 op/s 2023-09-23T09:33:36.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:35 smithi045 ceph-mon[98627]: pgmap v128: 97 pgs: 97 active+clean; 5.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.1 MiB/s wr, 434 op/s 2023-09-23T09:33:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 230.0 s, 2768.4 tps, lat 36.039 ms stddev 41.468 2023-09-23T09:33:37.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:36 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:33:37.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:36 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:33:37.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:36 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:33:38.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:37 smithi098 ceph-mon[102386]: pgmap v129: 97 pgs: 97 active+clean; 5.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.1 MiB/s wr, 246 op/s 2023-09-23T09:33:38.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:37 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:33:38.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:37 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:33:38.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:37 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:33:38.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:37 smithi045 ceph-mon[98627]: pgmap v129: 97 pgs: 97 active+clean; 5.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.1 MiB/s wr, 246 op/s 2023-09-23T09:33:38.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:37 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:33:38.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:37 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:33:38.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:37 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:33:38.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:37 smithi123 ceph-mon[103577]: pgmap v129: 97 pgs: 97 active+clean; 5.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.1 MiB/s wr, 246 op/s 2023-09-23T09:33:38.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:37 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:33:38.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:37 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:33:38.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:37 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:33:40.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:39 smithi098 ceph-mon[102386]: pgmap v130: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 14 MiB/s wr, 547 op/s 2023-09-23T09:33:40.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:39 smithi045 ceph-mon[98627]: pgmap v130: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 14 MiB/s wr, 547 op/s 2023-09-23T09:33:40.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:39 smithi123 ceph-mon[103577]: pgmap v130: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 14 MiB/s wr, 547 op/s 2023-09-23T09:33:41.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 235.0 s, 2831.7 tps, lat 35.393 ms stddev 38.113 2023-09-23T09:33:42.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:41 smithi098 ceph-mon[102386]: pgmap v131: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 13 MiB/s wr, 432 op/s 2023-09-23T09:33:42.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:41 smithi045 ceph-mon[98627]: pgmap v131: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 13 MiB/s wr, 432 op/s 2023-09-23T09:33:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:41 smithi123 ceph-mon[103577]: pgmap v131: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 13 MiB/s wr, 432 op/s 2023-09-23T09:33:44.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:43 smithi098 ceph-mon[102386]: pgmap v132: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 470 op/s 2023-09-23T09:33:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:43 smithi045 ceph-mon[98627]: pgmap v132: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 470 op/s 2023-09-23T09:33:44.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:43 smithi123 ceph-mon[103577]: pgmap v132: 97 pgs: 97 active+clean; 5.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 470 op/s 2023-09-23T09:33:46.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:45 smithi098 ceph-mon[102386]: pgmap v133: 97 pgs: 97 active+clean; 5.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 19 MiB/s wr, 722 op/s 2023-09-23T09:33:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:45 smithi045 ceph-mon[98627]: pgmap v133: 97 pgs: 97 active+clean; 5.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 19 MiB/s wr, 722 op/s 2023-09-23T09:33:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:45 smithi123 ceph-mon[103577]: pgmap v133: 97 pgs: 97 active+clean; 5.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 19 MiB/s wr, 722 op/s 2023-09-23T09:33:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 240.0 s, 2906.6 tps, lat 34.553 ms stddev 40.256 2023-09-23T09:33:48.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:47 smithi098 ceph-mon[102386]: pgmap v134: 97 pgs: 97 active+clean; 5.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T09:33:48.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:47 smithi045 ceph-mon[98627]: pgmap v134: 97 pgs: 97 active+clean; 5.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T09:33:48.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:47 smithi123 ceph-mon[103577]: pgmap v134: 97 pgs: 97 active+clean; 5.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T09:33:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.833+0000 7fe04fc66700 1 -- 172.21.15.45:0/1206731200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe04810f9e0 msgr2=0x7fe048108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.833+0000 7fe04fc66700 1 --2- 172.21.15.45:0/1206731200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe04810f9e0 0x7fe048108360 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe044009990 tx=0x7fe044009ca0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.835+0000 7fe04fc66700 1 -- 172.21.15.45:0/1206731200 shutdown_connections 2023-09-23T09:33:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.835+0000 7fe04fc66700 1 --2- 172.21.15.45:0/1206731200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 0x7fe04811c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.835+0000 7fe04fc66700 1 --2- 172.21.15.45:0/1206731200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe04810f570 0x7fe048107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.835+0000 7fe04fc66700 1 --2- 172.21.15.45:0/1206731200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe04810f9e0 0x7fe048108360 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.835+0000 7fe04fc66700 1 -- 172.21.15.45:0/1206731200 >> 172.21.15.45:0/1206731200 conn(0x7fe04806cee0 msgr2=0x7fe04806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.835+0000 7fe04fc66700 1 -- 172.21.15.45:0/1206731200 shutdown_connections 2023-09-23T09:33:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.836+0000 7fe04fc66700 1 -- 172.21.15.45:0/1206731200 wait complete. 2023-09-23T09:33:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.836+0000 7fe04fc66700 1 Processor -- start 2023-09-23T09:33:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.837+0000 7fe04fc66700 1 -- start start 2023-09-23T09:33:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.837+0000 7fe04fc66700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe04810f570 0x7fe0481a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.837+0000 7fe04fc66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe04810f9e0 0x7fe0481a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.837+0000 7fe04fc66700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 0x7fe0481aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.837+0000 7fe04fc66700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481ab010 con 0x7fe04810f9e0 2023-09-23T09:33:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.837+0000 7fe04fc66700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481ab180 con 0x7fe04810f570 2023-09-23T09:33:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.837+0000 7fe04fc66700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481ab340 con 0x7fe04811a600 2023-09-23T09:33:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 0x7fe0481aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 0x7fe0481aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36348/0 (socket says 172.21.15.45:36348) 2023-09-23T09:33:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 -- 172.21.15.45:0/3581118910 learned_addr learned my addr 172.21.15.45:0/3581118910 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 -- 172.21.15.45:0/3581118910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe04810f570 msgr2=0x7fe0481a5e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:33:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe04810f570 0x7fe0481a5e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 -- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe04810f9e0 msgr2=0x7fe0481a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe04810f9e0 0x7fe0481a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 -- 172.21.15.45:0/3581118910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0440095f0 con 0x7fe04811a600 2023-09-23T09:33:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.838+0000 7fe04e203700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 0x7fe0481aa9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe04000ada0 tx=0x7fe04000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.839+0000 7fe03effd700 1 -- 172.21.15.45:0/3581118910 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04000f070 con 0x7fe04811a600 2023-09-23T09:33:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.839+0000 7fe03effd700 1 -- 172.21.15.45:0/3581118910 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe04000ce40 con 0x7fe04811a600 2023-09-23T09:33:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.839+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0481ab620 con 0x7fe04811a600 2023-09-23T09:33:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.839+0000 7fe03effd700 1 -- 172.21.15.45:0/3581118910 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400054e0 con 0x7fe04811a600 2023-09-23T09:33:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.839+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04810ffc0 con 0x7fe04811a600 2023-09-23T09:33:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.844+0000 7fe03effd700 1 -- 172.21.15.45:0/3581118910 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fe040004ad0 con 0x7fe04811a600 2023-09-23T09:33:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.845+0000 7fe03effd700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fe0340a1790 0x7fe0340a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.845+0000 7fe03effd700 1 -- 172.21.15.45:0/3581118910 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe0400cc810 con 0x7fe04811a600 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe04da02700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fe0340a1790 0x7fe0340a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 Processor -- start 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 -- start start 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb57f40 0x556a1eb58320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb58860 0x556a1eb631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb65b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1eac1400 con 0x556a1eb58860 2023-09-23T09:33:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1eab2bc0 con 0x556a1eb63710 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe05be50b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1eab04e0 con 0x556a1eb57f40 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe04da02700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fe0340a1790 0x7fe0340a3c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe03800a9b0 tx=0x7fe038005c90 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe04d201700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb58860 0x556a1eb631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.846+0000 7fe04da02700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb57f40 0x556a1eb58320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04d201700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb58860 0x556a1eb631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34248/0 (socket says 172.21.15.45:34248) 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04d201700 1 -- 172.21.15.45:0/1054562317 learned_addr learned my addr 172.21.15.45:0/1054562317 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04e203700 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb65b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04da02700 1 -- 172.21.15.45:0/1054562317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 msgr2=0x556a1eb65b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04da02700 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb65b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04da02700 1 -- 172.21.15.45:0/1054562317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb58860 msgr2=0x556a1eb631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04da02700 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb58860 0x556a1eb631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.847+0000 7fe04da02700 1 -- 172.21.15.45:0/1054562317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a1eb0e870 con 0x556a1eb57f40 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.848+0000 7fe04da02700 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb57f40 0x556a1eb58320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe03800a2c0 tx=0x7fe03800f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc10817eef0ccd85 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.849+0000 7fe033fff700 1 -- 172.21.15.45:0/1054562317 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038042070 con 0x556a1eb57f40 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.849+0000 7fe033fff700 1 -- 172.21.15.45:0/1054562317 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe038004020 con 0x556a1eb57f40 2023-09-23T09:33:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.849+0000 7fe033fff700 1 -- 172.21.15.45:0/1054562317 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038003b10 con 0x556a1eb57f40 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 -- 172.21.15.45:0/1054562317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb57f40 msgr2=0x556a1eb58320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb57f40 0x556a1eb58320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe03800a2c0 tx=0x7fe03800f000 comp rx=0 tx=0).stop 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 -- 172.21.15.45:0/1054562317 shutdown_connections 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb65b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb57f40 0x556a1eb58320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 --2- 172.21.15.45:0/1054562317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb58860 0x556a1eb631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 -- 172.21.15.45:0/1054562317 >> 172.21.15.45:0/1054562317 conn(0x556a1eacd200 msgr2=0x556a1eb543c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 -- 172.21.15.45:0/1054562317 shutdown_connections 2023-09-23T09:33:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 -- 172.21.15.45:0/1054562317 wait complete. 2023-09-23T09:33:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.850+0000 7fe05be50b80 1 -- start start 2023-09-23T09:33:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 1 Processor -- start 2023-09-23T09:33:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb63120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb58860 0x556a1eb60330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb63710 0x556a1eb60960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1eaabdd0 con 0x556a1eb57f40 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1ea486b0 con 0x556a1eb58860 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1ea48350 con 0x556a1eb63710 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.851+0000 7fe05be50b80 10 client.0 _ll_register_callbacks cb 0x556a1e8a57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04da02700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb63120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb63710 0x556a1eb60960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb63710 0x556a1eb60960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36360/0 (socket says 172.21.15.45:36360) 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 -- 172.21.15.45:0/3196865666 learned_addr learned my addr 172.21.15.45:0/3196865666 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04d201700 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb58860 0x556a1eb60330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 -- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb58860 msgr2=0x556a1eb60330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb58860 0x556a1eb60330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 -- 172.21.15.45:0/3196865666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 msgr2=0x556a1eb63120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb63120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 -- 172.21.15.45:0/3196865666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03800c860 con 0x556a1eb63710 2023-09-23T09:33:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe04e203700 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb63710 0x556a1eb60960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe040099500 tx=0x7fe040099f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3f30f04c504821 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe0317fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.852+0000 7fe0317fa700 1 -- 172.21.15.45:0/3196865666 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400a1b00 con 0x556a1eb63710 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe0317fa700 1 -- 172.21.15.45:0/3196865666 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe0400840b0 con 0x556a1eb63710 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a1eb4c090 con 0x556a1eb63710 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe0317fa700 1 -- 172.21.15.45:0/3196865666 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe040084270 con 0x556a1eb63710 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 10 client.25732 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556a1e8d6d80 con 0x556a1eb63710 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe0317fa700 1 -- 172.21.15.45:0/3196865666 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe04007c080 con 0x556a1eb63710 2023-09-23T09:33:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe0317fa700 1 -- 172.21.15.45:0/3196865666 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe040098cf0 con 0x556a1eb63710 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 10 client.25732 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 20 client.25732 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1eb73150 0x556a1eb75570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 4 client.25732 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a1eaa4db0 con 0x556a1eb73150 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.853+0000 7fe04da02700 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1eb73150 0x556a1eb75570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.854+0000 7fe0317fa700 10 client.25732 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.854+0000 7fe04da02700 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1eb73150 0x556a1eb75570 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3649d91dfb73d7a8 server_cookie=9435803db8192f7e in_seq=0 out_seq=0 2023-09-23T09:33:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe0317fa700 1 -- 172.21.15.45:0/3196865666 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a1eaa4db0 con 0x556a1eb73150 2023-09-23T09:33:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe0317fa700 10 client.25732 handle_command_reply: tid=0 2023-09-23T09:33:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 client.25732 shutdown 2023-09-23T09:33:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb63710 msgr2=0x556a1eb60960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb63710 0x556a1eb60960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe040099500 tx=0x7fe040099f80 comp rx=0 tx=0).stop 2023-09-23T09:33:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 shutdown_connections 2023-09-23T09:33:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1eb73150 0x556a1eb75570 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb63710 0x556a1eb60960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb63120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3196865666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb58860 0x556a1eb60330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 >> 172.21.15.45:0/3196865666 conn(0x556a1eacd200 msgr2=0x556a1eb53fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.855+0000 7fe0317fa700 0 client.25732 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.856+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 shutdown_connections 2023-09-23T09:33:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.856+0000 7fe05be50b80 1 -- 172.21.15.45:0/3196865666 wait complete. 2023-09-23T09:33:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.856+0000 7fe05be50b80 20 client.25732 trim_cache size 0 max 16384 2023-09-23T09:33:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 Processor -- start 2023-09-23T09:33:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 -- start start 2023-09-23T09:33:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb5faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb62c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1e92c8c0 con 0x556a1eb57f40 2023-09-23T09:33:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1eb6a790 con 0x556a1eb63710 2023-09-23T09:33:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.883+0000 7fe05be50b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1eb53170 con 0x556a1eb58860 2023-09-23T09:33:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb6a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34260/0 (socket says 172.21.15.45:34260) 2023-09-23T09:33:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 -- 172.21.15.45:0/34609401 learned_addr learned my addr 172.21.15.45:0/34609401 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04e203700 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb62c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04d201700 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb5faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 -- 172.21.15.45:0/34609401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 msgr2=0x556a1eb5faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb5faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 -- 172.21.15.45:0/34609401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 msgr2=0x556a1eb62c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb62c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.885+0000 7fe04da02700 1 -- 172.21.15.45:0/34609401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a1e8bfc00 con 0x556a1eb57f40 2023-09-23T09:33:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.886+0000 7fe04da02700 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb6a250 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe0380130e0 tx=0x7fe03800afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133315f7310213be server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.886+0000 7fe03cd86700 1 -- 172.21.15.45:0/34609401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038042040 con 0x556a1eb57f40 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.886+0000 7fe03cd86700 1 -- 172.21.15.45:0/34609401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe03800a6c0 con 0x556a1eb57f40 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.886+0000 7fe03cd86700 1 -- 172.21.15.45:0/34609401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038050c70 con 0x556a1eb57f40 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.887+0000 7fe05be50b80 1 -- 172.21.15.45:0/34609401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 msgr2=0x556a1eb6a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.887+0000 7fe05be50b80 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb6a250 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe0380130e0 tx=0x7fe03800afb0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.887+0000 7fe05be50b80 1 -- 172.21.15.45:0/34609401 shutdown_connections 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.887+0000 7fe05be50b80 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb63710 0x556a1eb62c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.887+0000 7fe05be50b80 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb57f40 0x556a1eb6a250 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.887+0000 7fe05be50b80 1 --2- 172.21.15.45:0/34609401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb5faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.887+0000 7fe05be50b80 1 -- 172.21.15.45:0/34609401 >> 172.21.15.45:0/34609401 conn(0x556a1eacd200 msgr2=0x556a1eb6e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.888+0000 7fe05be50b80 1 -- 172.21.15.45:0/34609401 shutdown_connections 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.889+0000 7fe05be50b80 1 -- 172.21.15.45:0/34609401 wait complete. 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.889+0000 7fe05be50b80 1 -- start start 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.889+0000 7fe05be50b80 1 Processor -- start 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb57f40 0x556a1eb66230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb66770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe05be50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb63710 0x556a1eb66cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe05be50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1ebc5fc0 con 0x556a1eb63710 2023-09-23T09:33:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe05be50b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1ebc6130 con 0x556a1eb57f40 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe05be50b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1ebc62f0 con 0x556a1eb58860 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe05be50b80 10 client.0 _ll_register_callbacks cb 0x7fe040099380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04d201700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb66770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04e203700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb63710 0x556a1eb66cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04d201700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb66770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36372/0 (socket says 172.21.15.45:36372) 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04d201700 1 -- 172.21.15.45:0/3947025096 learned_addr learned my addr 172.21.15.45:0/3947025096 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04da02700 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb57f40 0x556a1eb66230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04e203700 1 -- 172.21.15.45:0/3947025096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 msgr2=0x556a1eb66770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04e203700 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb66770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04e203700 1 -- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb57f40 msgr2=0x556a1eb66230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04e203700 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb57f40 0x556a1eb66230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.890+0000 7fe04e203700 1 -- 172.21.15.45:0/3947025096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a1eb0e870 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe04e203700 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb63710 0x556a1eb66cb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe040095d00 tx=0x7fe040004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcabc1926db7c129 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe0317fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe0317fa700 1 -- 172.21.15.45:0/3947025096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040099c70 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe0317fa700 1 -- 172.21.15.45:0/3947025096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe0400a1b00 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe038004020 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe0317fa700 1 -- 172.21.15.45:0/3947025096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0400a1710 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe05be50b80 10 client.16053 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556a1e8fd1f0 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.891+0000 7fe0317fa700 1 -- 172.21.15.45:0/3947025096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe038004020 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe0317fa700 1 -- 172.21.15.45:0/3947025096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe040098760 con 0x556a1eb63710 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe05be50b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe05be50b80 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe05be50b80 20 client.16053 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1ebc7720 0x556a1ebc9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe05be50b80 4 client.16053 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:33:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a1eaa4ad0 con 0x556a1ebc7720 2023-09-23T09:33:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe04da02700 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1ebc7720 0x556a1ebc9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe0317fa700 10 client.16053 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.892+0000 7fe04da02700 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1ebc7720 0x556a1ebc9b40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e3acb723324758 server_cookie=2550657aed2dfd6 in_seq=0 out_seq=0 2023-09-23T09:33:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.926+0000 7fe0317fa700 1 -- 172.21.15.45:0/3947025096 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a1eaa4ad0 con 0x556a1ebc7720 2023-09-23T09:33:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.926+0000 7fe0317fa700 10 client.16053 handle_command_reply: tid=0 2023-09-23T09:33:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.926+0000 7fe05be50b80 1 client.16053 shutdown 2023-09-23T09:33:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb63710 msgr2=0x556a1eb66cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb63710 0x556a1eb66cb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe040095d00 tx=0x7fe040004fb0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 shutdown_connections 2023-09-23T09:33:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556a1ebc7720 0x556a1ebc9b40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a1eb63710 0x556a1eb66cb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1eb57f40 0x556a1eb66230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 --2- 172.21.15.45:0/3947025096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a1eb58860 0x556a1eb66770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 >> 172.21.15.45:0/3947025096 conn(0x556a1eacd200 msgr2=0x556a1eb5b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe0317fa700 0 client.16053 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 shutdown_connections 2023-09-23T09:33:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 1 -- 172.21.15.45:0/3947025096 wait complete. 2023-09-23T09:33:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.927+0000 7fe05be50b80 20 client.16053 trim_cache size 0 max 16384 2023-09-23T09:33:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:33:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:33:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:33:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:33:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fe0340a1790 msgr2=0x7fe0340a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fe0340a1790 0x7fe0340a3c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe03800a9b0 tx=0x7fe038005c90 comp rx=0 tx=0).stop 2023-09-23T09:33:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 msgr2=0x7fe0481aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 0x7fe0481aa9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe04000ada0 tx=0x7fe04000c330 comp rx=0 tx=0).stop 2023-09-23T09:33:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 shutdown_connections 2023-09-23T09:33:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04811a600 0x7fe0481aa9d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe04810f570 0x7fe0481a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fe0340a1790 0x7fe0340a3c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 --2- 172.21.15.45:0/3581118910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe04810f9e0 0x7fe0481a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 >> 172.21.15.45:0/3581118910 conn(0x7fe04806cee0 msgr2=0x7fe0481185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 shutdown_connections 2023-09-23T09:33:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:33:48.928+0000 7fe04fc66700 1 -- 172.21.15.45:0/3581118910 wait complete. 2023-09-23T09:33:50.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:49 smithi098 ceph-mon[102386]: pgmap v135: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 21 MiB/s wr, 912 op/s 2023-09-23T09:33:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:49 smithi045 ceph-mon[98627]: pgmap v135: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 21 MiB/s wr, 912 op/s 2023-09-23T09:33:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:49 smithi123 ceph-mon[103577]: pgmap v135: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 21 MiB/s wr, 912 op/s 2023-09-23T09:33:51.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:51 smithi098 ceph-mon[102386]: pgmap v136: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 611 op/s 2023-09-23T09:33:51.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:51 smithi045 ceph-mon[98627]: pgmap v136: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 611 op/s 2023-09-23T09:33:51.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:51 smithi123 ceph-mon[103577]: pgmap v136: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 611 op/s 2023-09-23T09:33:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 245.0 s, 1972.5 tps, lat 50.443 ms stddev 59.561 2023-09-23T09:33:53.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:53 smithi098 ceph-mon[102386]: pgmap v137: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 675 op/s 2023-09-23T09:33:53.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:53 smithi045 ceph-mon[98627]: pgmap v137: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 675 op/s 2023-09-23T09:33:53.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:53 smithi123 ceph-mon[103577]: pgmap v137: 97 pgs: 97 active+clean; 5.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 675 op/s 2023-09-23T09:33:55.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:55 smithi098 ceph-mon[102386]: pgmap v138: 97 pgs: 97 active+clean; 6.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-09-23T09:33:55.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:55 smithi045 ceph-mon[98627]: pgmap v138: 97 pgs: 97 active+clean; 6.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-09-23T09:33:55.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:55 smithi123 ceph-mon[103577]: pgmap v138: 97 pgs: 97 active+clean; 6.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-09-23T09:33:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 250.0 s, 2593.8 tps, lat 38.487 ms stddev 41.536 2023-09-23T09:33:57.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:57 smithi098 ceph-mon[102386]: pgmap v139: 97 pgs: 97 active+clean; 6.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 808 op/s 2023-09-23T09:33:57.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:57 smithi045 ceph-mon[98627]: pgmap v139: 97 pgs: 97 active+clean; 6.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 808 op/s 2023-09-23T09:33:57.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:57 smithi123 ceph-mon[103577]: pgmap v139: 97 pgs: 97 active+clean; 6.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 808 op/s 2023-09-23T09:33:59.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:33:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 -- 172.21.15.45:0/2840416614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4106110 msgr2=0x7fdea410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 --2- 172.21.15.45:0/2840416614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4106110 0x7fdea410dc90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00b0a0 tx=0x7fde9c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 -- 172.21.15.45:0/2840416614 shutdown_connections 2023-09-23T09:33:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 --2- 172.21.15.45:0/2840416614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4106110 0x7fdea410dc90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 --2- 172.21.15.45:0/2840416614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4100ff0 0x7fdea41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 --2- 172.21.15.45:0/2840416614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea41019a0 0x7fdea41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 -- 172.21.15.45:0/2840416614 >> 172.21.15.45:0/2840416614 conn(0x7fdea40fc890 msgr2=0x7fdea40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 -- 172.21.15.45:0/2840416614 shutdown_connections 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.735+0000 7fdeab27c700 1 -- 172.21.15.45:0/2840416614 wait complete. 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 Processor -- start 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 -- start start 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4100ff0 0x7fdea4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4106110 0x7fdea4073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea4073ba0 0x7fdea41a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41a5640 con 0x7fdea4073ba0 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41a57b0 con 0x7fdea4100ff0 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdeab27c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41a5970 con 0x7fdea4106110 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea9819700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea4073ba0 0x7fdea41a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea8817700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4106110 0x7fdea4073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea9018700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4100ff0 0x7fdea4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea9819700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea4073ba0 0x7fdea41a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34284/0 (socket says 172.21.15.45:34284) 2023-09-23T09:33:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea8817700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4106110 0x7fdea4073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36388/0 (socket says 172.21.15.45:36388) 2023-09-23T09:33:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea9819700 1 -- 172.21.15.45:0/978212084 learned_addr learned my addr 172.21.15.45:0/978212084 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea8817700 1 -- 172.21.15.45:0/978212084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4100ff0 msgr2=0x7fdea4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea8817700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4100ff0 0x7fdea4073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea8817700 1 -- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea4073ba0 msgr2=0x7fdea41a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.736+0000 7fdea8817700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea4073ba0 0x7fdea41a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.737+0000 7fdea8817700 1 -- 172.21.15.45:0/978212084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c009d00 con 0x7fdea4106110 2023-09-23T09:33:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.737+0000 7fdea8817700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4106110 0x7fdea4073660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e9c0 tx=0x7fde9400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.737+0000 7fde9a7fc700 1 -- 172.21.15.45:0/978212084 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400c9c0 con 0x7fdea4106110 2023-09-23T09:33:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.737+0000 7fde9a7fc700 1 -- 172.21.15.45:0/978212084 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde94010430 con 0x7fdea4106110 2023-09-23T09:33:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.737+0000 7fde9a7fc700 1 -- 172.21.15.45:0/978212084 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94018680 con 0x7fdea4106110 2023-09-23T09:33:59.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.738+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea41a5c50 con 0x7fdea4106110 2023-09-23T09:33:59.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.738+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea41a6110 con 0x7fdea4106110 2023-09-23T09:33:59.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.741+0000 7fde9a7fc700 1 -- 172.21.15.45:0/978212084 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fde9400cb20 con 0x7fdea4106110 2023-09-23T09:33:59.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.743+0000 7fdeb7466b80 1 Processor -- start 2023-09-23T09:33:59.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdeb7466b80 1 -- start start 2023-09-23T09:33:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fde9a7fc700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fde900a1710 0x7fde900a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed7ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed717460 con 0x55a9ed7aed20 2023-09-23T09:33:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed708c20 con 0x55a9ed7ae400 2023-09-23T09:33:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9819700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fde900a1710 0x7fde900a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed706540 con 0x55a9ed7b9bd0 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fde9a7fc700 1 -- 172.21.15.45:0/978212084 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fde94014070 con 0x7fdea4106110 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed7ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea8817700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed7ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55638/0 (socket says 172.21.15.45:55638) 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 -- 172.21.15.45:0/929480568 learned_addr learned my addr 172.21.15.45:0/929480568 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9819700 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea8817700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34290/0 (socket says 172.21.15.45:34290) 2023-09-23T09:33:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 -- 172.21.15.45:0/929480568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 msgr2=0x55a9ed7bbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7bbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 -- 172.21.15.45:0/929480568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 msgr2=0x55a9ed7b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.744+0000 7fdea9018700 1 -- 172.21.15.45:0/929480568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9ed7648d0 con 0x55a9ed7ae400 2023-09-23T09:33:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.745+0000 7fdea9819700 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.745+0000 7fdea9819700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fde900a1710 0x7fde900a3bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00bf30 tx=0x7fde9c00bf60 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.745+0000 7fdea9018700 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed7ae7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d80 tx=0x7fdea0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f2a8cebc86bad7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.745+0000 7fde8f7fe700 1 -- 172.21.15.45:0/929480568 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0004230 con 0x55a9ed7ae400 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.745+0000 7fde8f7fe700 1 -- 172.21.15.45:0/929480568 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea0004390 con 0x55a9ed7ae400 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.745+0000 7fde8f7fe700 1 -- 172.21.15.45:0/929480568 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea00197e0 con 0x55a9ed7ae400 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 -- 172.21.15.45:0/929480568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 msgr2=0x55a9ed7ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed7ae7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d80 tx=0x7fdea0011720 comp rx=0 tx=0).stop 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 -- 172.21.15.45:0/929480568 shutdown_connections 2023-09-23T09:33:59.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed7ae7e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/929480568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 -- 172.21.15.45:0/929480568 >> 172.21.15.45:0/929480568 conn(0x55a9ed723260 msgr2=0x55a9ed7aa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 -- 172.21.15.45:0/929480568 shutdown_connections 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.746+0000 7fdeb7466b80 1 -- 172.21.15.45:0/929480568 wait complete. 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 -- start start 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 Processor -- start 2023-09-23T09:33:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed701e30 con 0x55a9ed7aed20 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed69e710 con 0x55a9ed7b9bd0 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed69e3b0 con 0x55a9ed7ae400 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdeb7466b80 10 client.0 _ll_register_callbacks cb 0x55a9ed4fb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdea8817700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdea8817700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34296/0 (socket says 172.21.15.45:34296) 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdea9018700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdea8817700 1 -- 172.21.15.45:0/244773462 learned_addr learned my addr 172.21.15.45:0/244773462 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.747+0000 7fdea9819700 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fdea9018700 1 -- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 msgr2=0x55a9ed7b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fdea9018700 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fdea9018700 1 -- 172.21.15.45:0/244773462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 msgr2=0x55a9ed7b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fdea9018700 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fdea9018700 1 -- 172.21.15.45:0/244773462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c026050 con 0x55a9ed7ae400 2023-09-23T09:33:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fdea8817700 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fdea9018700 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7b6940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdea0003f10 tx=0x7fdea00038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b57593e9a0f0d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.748+0000 7fde8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.749+0000 7fde8cff9700 1 -- 172.21.15.45:0/244773462 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0003d30 con 0x55a9ed7ae400 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.749+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9ed7a2250 con 0x55a9ed7ae400 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.749+0000 7fde8cff9700 1 -- 172.21.15.45:0/244773462 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdea0005040 con 0x55a9ed7ae400 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.750+0000 7fde8cff9700 1 -- 172.21.15.45:0/244773462 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea0005220 con 0x55a9ed7ae400 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.750+0000 7fdeb7466b80 10 client.25747 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.750+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fdea00099e0 con 0x55a9ed7ae400 2023-09-23T09:33:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fde8cff9700 1 -- 172.21.15.45:0/244773462 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9ed7a2250 con 0x55a9ed7ae400 2023-09-23T09:33:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fde8cff9700 1 -- 172.21.15.45:0/244773462 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdea0021040 con 0x55a9ed7ae400 2023-09-23T09:33:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fdeb7466b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fdeb7466b80 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fdeb7466b80 20 client.25747 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed7c9600 0x55a9ed7cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fdeb7466b80 4 client.25747 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:33:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.751+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9ed6fae10 con 0x55a9ed7c9600 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.752+0000 7fdea8817700 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed7c9600 0x55a9ed7cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.752+0000 7fde8cff9700 10 client.25747 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.752+0000 7fdea8817700 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed7c9600 0x55a9ed7cba20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e5b6d948faf33f server_cookie=25009e0e030f834d in_seq=0 out_seq=0 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fde8cff9700 1 -- 172.21.15.45:0/244773462 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a9ed6fae10 con 0x55a9ed7c9600 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fde8cff9700 10 client.25747 handle_command_reply: tid=0 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 client.25747 shutdown 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 msgr2=0x55a9ed7b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7b6940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdea0003f10 tx=0x7fdea00038b0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 shutdown_connections 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed7c9600 0x55a9ed7cba20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fde8cff9700 0 client.25747 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7b6940 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/244773462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 >> 172.21.15.45:0/244773462 conn(0x55a9ed723260 msgr2=0x55a9ed7aa320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 shutdown_connections 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 1 -- 172.21.15.45:0/244773462 wait complete. 2023-09-23T09:33:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.753+0000 7fdeb7466b80 20 client.25747 trim_cache size 0 max 16384 2023-09-23T09:33:59.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 Processor -- start 2023-09-23T09:33:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 -- start start 2023-09-23T09:33:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7c0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed582920 con 0x55a9ed7aed20 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed7c0cf0 con 0x55a9ed7b9bd0 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.785+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed7b4f20 con 0x55a9ed7ae400 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea8817700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea8817700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34304/0 (socket says 172.21.15.45:34304) 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea8817700 1 -- 172.21.15.45:0/3173371806 learned_addr learned my addr 172.21.15.45:0/3173371806 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9819700 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9018700 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9018700 1 -- 172.21.15.45:0/3173371806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 msgr2=0x55a9ed7b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9018700 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9018700 1 -- 172.21.15.45:0/3173371806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 msgr2=0x55a9ed7b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9018700 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9018700 1 -- 172.21.15.45:0/3173371806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9ed52cde0 con 0x55a9ed7ae400 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.786+0000 7fdea9018700 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7c0720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdea0004360 tx=0x7fdea0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a86993bf35222c26 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.787+0000 7fde8ffff700 1 -- 172.21.15.45:0/3173371806 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0003db0 con 0x55a9ed7ae400 2023-09-23T09:33:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.787+0000 7fde8ffff700 1 -- 172.21.15.45:0/3173371806 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea0023da0 con 0x55a9ed7ae400 2023-09-23T09:33:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.787+0000 7fde8ffff700 1 -- 172.21.15.45:0/3173371806 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0005630 con 0x55a9ed7ae400 2023-09-23T09:33:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.788+0000 7fdeb7466b80 1 -- 172.21.15.45:0/3173371806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 msgr2=0x55a9ed7c0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.788+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7c0720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdea0004360 tx=0x7fdea0009f90 comp rx=0 tx=0).stop 2023-09-23T09:33:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.789+0000 7fdeb7466b80 1 -- 172.21.15.45:0/3173371806 shutdown_connections 2023-09-23T09:33:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.789+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed7b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.789+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7ae400 0x55a9ed7c0720 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.789+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/3173371806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed7b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.789+0000 7fdeb7466b80 1 -- 172.21.15.45:0/3173371806 >> 172.21.15.45:0/3173371806 conn(0x55a9ed723260 msgr2=0x55a9ed7c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.789+0000 7fdeb7466b80 1 -- 172.21.15.45:0/3173371806 shutdown_connections 2023-09-23T09:33:59.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.789+0000 7fdeb7466b80 1 -- 172.21.15.45:0/3173371806 wait complete. 2023-09-23T09:33:59.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.790+0000 7fdeb7466b80 1 -- start start 2023-09-23T09:33:59.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.790+0000 7fdeb7466b80 1 Processor -- start 2023-09-23T09:33:59.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:33:59 smithi098 ceph-mon[102386]: pgmap v140: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 27 MiB/s wr, 1.17k op/s 2023-09-23T09:33:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed8113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed811920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdeb7466b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed81c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed812100 con 0x55a9ed7aed20 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed812270 con 0x55a9ed7ae400 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdeb7466b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ed81c9f0 con 0x55a9ed7b9bd0 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdeb7466b80 10 client.0 _ll_register_callbacks cb 0x7fde9c00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.791+0000 7fdea9018700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed8113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed8113e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55658/0 (socket says 172.21.15.45:55658) 2023-09-23T09:33:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 -- 172.21.15.45:0/1252379492 learned_addr learned my addr 172.21.15.45:0/1252379492 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 -- 172.21.15.45:0/1252379492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 msgr2=0x55a9ed81c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9819700 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed81c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed81c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 -- 172.21.15.45:0/1252379492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 msgr2=0x55a9ed811920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed811920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 -- 172.21.15.45:0/1252379492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea00099e0 con 0x55a9ed7ae400 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9819700 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed81c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fdea9018700 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed8113e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdea00236f0 tx=0x7fdea0003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c19b71a485c538 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:33:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.792+0000 7fde8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fde8cff9700 1 -- 172.21.15.45:0/1252379492 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0023af0 con 0x55a9ed7ae400 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fde8cff9700 1 -- 172.21.15.45:0/1252379492 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdea0000e40 con 0x55a9ed7ae400 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdea0004390 con 0x55a9ed7ae400 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fde8cff9700 1 -- 172.21.15.45:0/1252379492 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea0005210 con 0x55a9ed7ae400 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fdeb7466b80 10 client.25754 fetch_fsmap learned FSMap version 30 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a9ed515c60 con 0x55a9ed7ae400 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fde8cff9700 1 -- 172.21.15.45:0/1252379492 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea004c020 con 0x55a9ed7ae400 2023-09-23T09:33:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.793+0000 7fde8cff9700 1 -- 172.21.15.45:0/1252379492 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdea0011790 con 0x55a9ed7ae400 2023-09-23T09:33:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.794+0000 7fdeb7466b80 10 client.25754 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:33:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.794+0000 7fdeb7466b80 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:33:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.794+0000 7fdeb7466b80 20 client.25754 populate_metadata read hostname 'smithi045' 2023-09-23T09:33:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.794+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed81dd20 0x55a9ed820140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:33:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.794+0000 7fdeb7466b80 4 client.25754 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.794+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9ed6fab30 con 0x55a9ed81dd20 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.794+0000 7fdea8817700 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed81dd20 0x55a9ed820140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.795+0000 7fdea8817700 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed81dd20 0x55a9ed820140 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4514b08c8e543b92 server_cookie=40669e19a0920887 in_seq=0 out_seq=0 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.795+0000 7fde8cff9700 10 client.25754 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.795+0000 7fde8cff9700 1 -- 172.21.15.45:0/1252379492 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a9ed6fab30 con 0x55a9ed81dd20 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.795+0000 7fde8cff9700 10 client.25754 handle_command_reply: tid=0 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 client.25754 shutdown 2023-09-23T09:33:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 msgr2=0x55a9ed8113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed8113e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdea00236f0 tx=0x7fdea0003e90 comp rx=0 tx=0).stop 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 shutdown_connections 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a9ed81dd20 0x55a9ed820140 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9ed7b9bd0 0x55a9ed81c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9ed7ae400 0x55a9ed8113e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 --2- 172.21.15.45:0/1252379492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9ed7aed20 0x55a9ed811920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 >> 172.21.15.45:0/1252379492 conn(0x55a9ed723260 msgr2=0x55a9ed7b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fde8cff9700 0 client.25754 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:33:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.796+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 shutdown_connections 2023-09-23T09:33:59.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.797+0000 7fdeb7466b80 1 -- 172.21.15.45:0/1252379492 wait complete. 2023-09-23T09:33:59.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.797+0000 7fdeb7466b80 20 client.25754 trim_cache size 0 max 16384 2023-09-23T09:33:59.803 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:33:59.803 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:33:59.803 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:33:59.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:33:59.804 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:33:59.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:33:59.804 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:33:59.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:33:59.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:33:59.804 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:33:59.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fde900a1710 msgr2=0x7fde900a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fde900a1710 0x7fde900a3bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00bf30 tx=0x7fde9c00bf60 comp rx=0 tx=0).stop 2023-09-23T09:33:59.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4106110 msgr2=0x7fdea4073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:33:59.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4106110 0x7fdea4073660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e9c0 tx=0x7fde9400ecd0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 shutdown_connections 2023-09-23T09:33:59.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fde900a1710 0x7fde900a3bd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea4073ba0 0x7fdea41a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea4100ff0 0x7fdea4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 --2- 172.21.15.45:0/978212084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdea4106110 0x7fdea4073660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:33:59.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 >> 172.21.15.45:0/978212084 conn(0x7fdea40fc890 msgr2=0x7fdea41049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:33:59.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 shutdown_connections 2023-09-23T09:33:59.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:33:59.798+0000 7fdeab27c700 1 -- 172.21.15.45:0/978212084 wait complete. 2023-09-23T09:33:59.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:33:59 smithi045 ceph-mon[98627]: pgmap v140: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 27 MiB/s wr, 1.17k op/s 2023-09-23T09:33:59.818 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:33:59.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:33:59 smithi123 ceph-mon[103577]: pgmap v140: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 27 MiB/s wr, 1.17k op/s 2023-09-23T09:34:01.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:01 smithi098 ceph-mon[102386]: pgmap v141: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 866 op/s 2023-09-23T09:34:01.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:01 smithi045 ceph-mon[98627]: pgmap v141: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 866 op/s 2023-09-23T09:34:01.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:01 smithi123 ceph-mon[103577]: pgmap v141: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 866 op/s 2023-09-23T09:34:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 255.0 s, 1823.8 tps, lat 41.803 ms stddev 45.910 2023-09-23T09:34:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 -- 172.21.15.45:0/273455028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15781019c0 msgr2=0x7f1578105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 --2- 172.21.15.45:0/273455028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15781019c0 0x7f1578105a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15680098f0 tx=0x7f1568009c00 comp rx=0 tx=0).stop 2023-09-23T09:34:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 -- 172.21.15.45:0/273455028 shutdown_connections 2023-09-23T09:34:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 --2- 172.21.15.45:0/273455028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1578106130 0x7f157810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 --2- 172.21.15.45:0/273455028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1578101010 0x7f15781013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 --2- 172.21.15.45:0/273455028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15781019c0 0x7f1578105a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 -- 172.21.15.45:0/273455028 >> 172.21.15.45:0/273455028 conn(0x7f15780fc8d0 msgr2=0x7f15780fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 -- 172.21.15.45:0/273455028 shutdown_connections 2023-09-23T09:34:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.103+0000 7f157da5c700 1 -- 172.21.15.45:0/273455028 wait complete. 2023-09-23T09:34:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.104+0000 7f157da5c700 1 Processor -- start 2023-09-23T09:34:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f157da5c700 1 -- start start 2023-09-23T09:34:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f157da5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1578101010 0x7f1578199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f157da5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1578106130 0x7f1578199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f157da5c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f157819dcf0 0x7f157819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f157da5c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157819e770 con 0x7f1578106130 2023-09-23T09:34:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f157da5c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157819e8e0 con 0x7f157819dcf0 2023-09-23T09:34:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f157da5c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157819eaa0 con 0x7f1578101010 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f1576ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1578101010 0x7f1578199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f1576ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1578101010 0x7f1578199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36428/0 (socket says 172.21.15.45:36428) 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f15777fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f157819dcf0 0x7f157819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.106+0000 7f1576ffd700 1 -- 172.21.15.45:0/1631583395 learned_addr learned my addr 172.21.15.45:0/1631583395 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15767fc700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1578106130 0x7f1578199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15777fe700 1 -- 172.21.15.45:0/1631583395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1578101010 msgr2=0x7f1578199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15777fe700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1578101010 0x7f1578199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15777fe700 1 -- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1578106130 msgr2=0x7f1578199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15777fe700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1578106130 0x7f1578199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15777fe700 1 -- 172.21.15.45:0/1631583395 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568009520 con 0x7f157819dcf0 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15767fc700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1578106130 0x7f1578199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f15777fe700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f157819dcf0 0x7f157819e170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f157000be20 tx=0x7f1570009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f157ca5a700 1 -- 172.21.15.45:0/1631583395 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000cc10 con 0x7f157819dcf0 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f157ca5a700 1 -- 172.21.15.45:0/1631583395 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f157000cd70 con 0x7f157819dcf0 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f157ca5a700 1 -- 172.21.15.45:0/1631583395 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15700126c0 con 0x7f157819dcf0 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15781a6000 con 0x7f157819dcf0 2023-09-23T09:34:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.107+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15781a6490 con 0x7f157819dcf0 2023-09-23T09:34:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.110+0000 7f157ca5a700 1 -- 172.21.15.45:0/1631583395 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f1570012860 con 0x7f157819dcf0 2023-09-23T09:34:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 Processor -- start 2023-09-23T09:34:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 -- start start 2023-09-23T09:34:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83430400 con 0x563a834c6f40 2023-09-23T09:34:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83421bc0 con 0x563a834c7860 2023-09-23T09:34:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f1589c46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8341f4e0 con 0x563a834daf20 2023-09-23T09:34:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.112+0000 7f157ca5a700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f15600a1790 0x7f15600a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.113+0000 7f157ca5a700 1 -- 172.21.15.45:0/1631583395 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1570006e40 con 0x7f157819dcf0 2023-09-23T09:34:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.114+0000 7f15767fc700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f15600a1790 0x7f15600a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.114+0000 7f15767fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.114+0000 7f15767fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834da9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55680/0 (socket says 172.21.15.45:55680) 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.114+0000 7f15767fc700 1 -- 172.21.15.45:0/2903138531 learned_addr learned my addr 172.21.15.45:0/2903138531 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.115+0000 7f1576ffd700 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.115+0000 7f15767fc700 1 -- 172.21.15.45:0/2903138531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 msgr2=0x563a834dd310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.115+0000 7f15767fc700 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834dd310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.115+0000 7f15767fc700 1 -- 172.21.15.45:0/2903138531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 msgr2=0x563a834c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.115+0000 7f15767fc700 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.115+0000 7f15767fc700 1 -- 172.21.15.45:0/2903138531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a8347d870 con 0x563a834c7860 2023-09-23T09:34:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.116+0000 7f15767fc700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f15600a1790 0x7f15600a3c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15680098f0 tx=0x7f156801c000 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.116+0000 7f15767fc700 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834da9e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f157806a480 tx=0x7f156800f930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990ad62348b8794a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.116+0000 7f155dffb700 1 -- 172.21.15.45:0/2903138531 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800f600 con 0x563a834c7860 2023-09-23T09:34:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.116+0000 7f155dffb700 1 -- 172.21.15.45:0/2903138531 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15680268b0 con 0x563a834c7860 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.117+0000 7f155dffb700 1 -- 172.21.15.45:0/2903138531 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800e620 con 0x563a834c7860 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.117+0000 7f1589c46b80 1 -- 172.21.15.45:0/2903138531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 msgr2=0x563a834da9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.117+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834da9e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f157806a480 tx=0x7f156800f930 comp rx=0 tx=0).stop 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 -- 172.21.15.45:0/2903138531 shutdown_connections 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2903138531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834da9e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 -- 172.21.15.45:0/2903138531 >> 172.21.15.45:0/2903138531 conn(0x563a8343c200 msgr2=0x563a834c33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 -- 172.21.15.45:0/2903138531 shutdown_connections 2023-09-23T09:34:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 -- 172.21.15.45:0/2903138531 wait complete. 2023-09-23T09:34:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.118+0000 7f1589c46b80 1 -- start start 2023-09-23T09:34:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 1 Processor -- start 2023-09-23T09:34:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834da940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8341add0 con 0x563a834c6f40 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a833b76b0 con 0x563a834c7860 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a833b7350 con 0x563a834daf20 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1589c46b80 10 client.0 _ll_register_callbacks cb 0x563a832147a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1576ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834da940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1576ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834da940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34334/0 (socket says 172.21.15.45:34334) 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f1576ffd700 1 -- 172.21.15.45:0/3348868735 learned_addr learned my addr 172.21.15.45:0/3348868735 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.119+0000 7f15767fc700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f15777fe700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1576ffd700 1 -- 172.21.15.45:0/3348868735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 msgr2=0x563a834d82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1576ffd700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834d82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1576ffd700 1 -- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 msgr2=0x563a834d7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1576ffd700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834d7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1576ffd700 1 -- 172.21.15.45:0/3348868735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157810e520 con 0x563a834c6f40 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f15777fe700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:34:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f15767fc700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1576ffd700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834da940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c010 tx=0x7f156c0127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68a47470675be8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1556ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1556ffd700 1 -- 172.21.15.45:0/3348868735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c010870 con 0x563a834c6f40 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1556ffd700 1 -- 172.21.15.45:0/3348868735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f156c0109d0 con 0x563a834c6f40 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a834bb090 con 0x563a834c6f40 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1556ffd700 1 -- 172.21.15.45:0/3348868735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f156c010bd0 con 0x563a834c6f40 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1589c46b80 10 client.16074 fetch_fsmap learned FSMap version 30 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563a83245d80 con 0x563a834c6f40 2023-09-23T09:34:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.120+0000 7f1556ffd700 1 -- 172.21.15.45:0/3348868735 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c027080 con 0x563a834c6f40 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1556ffd700 1 -- 172.21.15.45:0/3348868735 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f156c023410 con 0x563a834c6f40 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1589c46b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1589c46b80 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1589c46b80 20 client.16074 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1589c46b80 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a834ea910 0x563a834ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1589c46b80 4 client.16074 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a83413db0 con 0x563a834ea910 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f15767fc700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a834ea910 0x563a834ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f1556ffd700 10 client.16074 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.121+0000 7f15767fc700 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a834ea910 0x563a834ecd30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9677d218d0cf4e server_cookie=f885f90c6532480 in_seq=0 out_seq=0 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1556ffd700 1 -- 172.21.15.45:0/3348868735 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a83413db0 con 0x563a834ea910 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1556ffd700 10 client.16074 handle_command_reply: tid=0 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 client.16074 shutdown 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 msgr2=0x563a834da940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834da940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c010 tx=0x7f156c0127e0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 shutdown_connections 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a834ea910 0x563a834ecd30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1556ffd700 0 client.16074 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834da940 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.122+0000 7f1589c46b80 1 --2- 172.21.15.45:0/3348868735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.123+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 >> 172.21.15.45:0/3348868735 conn(0x563a8343c200 msgr2=0x563a834c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.123+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 shutdown_connections 2023-09-23T09:34:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.123+0000 7f1589c46b80 1 -- 172.21.15.45:0/3348868735 wait complete. 2023-09-23T09:34:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.123+0000 7f1589c46b80 20 client.16074 trim_cache size 0 max 16384 2023-09-23T09:34:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.143+0000 7f1589c46b80 1 Processor -- start 2023-09-23T09:34:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1589c46b80 1 -- start start 2023-09-23T09:34:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c6f40 0x563a834dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c7860 0x563a834e1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1589c46b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8329b8c0 con 0x563a834c7860 2023-09-23T09:34:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1589c46b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a834d6190 con 0x563a834c6f40 2023-09-23T09:34:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1589c46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a834d3a80 con 0x563a834daf20 2023-09-23T09:34:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c7860 0x563a834e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c7860 0x563a834e1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34342/0 (socket says 172.21.15.45:34342) 2023-09-23T09:34:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 -- 172.21.15.45:0/2645125146 learned_addr learned my addr 172.21.15.45:0/2645125146 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f1576ffd700 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c6f40 0x563a834dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15777fe700 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 -- 172.21.15.45:0/2645125146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 msgr2=0x563a834e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 -- 172.21.15.45:0/2645125146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c6f40 msgr2=0x563a834dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c6f40 0x563a834dd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.144+0000 7f15767fc700 1 -- 172.21.15.45:0/2645125146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a8322ec00 con 0x563a834c7860 2023-09-23T09:34:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.146+0000 7f15767fc700 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c7860 0x563a834e1940 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f157806a480 tx=0x7f156800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54224029d8bd670a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.146+0000 7f155e7fc700 1 -- 172.21.15.45:0/2645125146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800bb10 con 0x563a834c7860 2023-09-23T09:34:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.146+0000 7f155e7fc700 1 -- 172.21.15.45:0/2645125146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156800bc70 con 0x563a834c7860 2023-09-23T09:34:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.146+0000 7f155e7fc700 1 -- 172.21.15.45:0/2645125146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568026c80 con 0x563a834c7860 2023-09-23T09:34:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.147+0000 7f1589c46b80 1 -- 172.21.15.45:0/2645125146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c7860 msgr2=0x563a834e1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.147+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c7860 0x563a834e1940 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f157806a480 tx=0x7f156800f8b0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 -- 172.21.15.45:0/2645125146 shutdown_connections 2023-09-23T09:34:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a834e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c6f40 0x563a834dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 --2- 172.21.15.45:0/2645125146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c7860 0x563a834e1940 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 -- 172.21.15.45:0/2645125146 >> 172.21.15.45:0/2645125146 conn(0x563a8343c200 msgr2=0x563a834f16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 -- 172.21.15.45:0/2645125146 shutdown_connections 2023-09-23T09:34:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 -- 172.21.15.45:0/2645125146 wait complete. 2023-09-23T09:34:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.148+0000 7f1589c46b80 1 -- start start 2023-09-23T09:34:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 1 Processor -- start 2023-09-23T09:34:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a83538740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83538c80 con 0x563a834c6f40 2023-09-23T09:34:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83538dc0 con 0x563a834c7860 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83538f80 con 0x563a834daf20 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1589c46b80 10 client.0 _ll_register_callbacks cb 0x563a832147a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1576ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f15767fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f15767fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834e0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55698/0 (socket says 172.21.15.45:55698) 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f1576ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834e05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34348/0 (socket says 172.21.15.45:34348) 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f15767fc700 1 -- 172.21.15.45:0/1456646034 learned_addr learned my addr 172.21.15.45:0/1456646034 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.149+0000 7f15777fe700 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a83538740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f15767fc700 1 -- 172.21.15.45:0/1456646034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 msgr2=0x563a83538740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f15767fc700 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a83538740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f15767fc700 1 -- 172.21.15.45:0/1456646034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 msgr2=0x563a834e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f15767fc700 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834e05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f15767fc700 1 -- 172.21.15.45:0/1456646034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1568034040 con 0x563a834c7860 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1576ffd700 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f15767fc700 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834e0b30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1568011040 tx=0x7f156800b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88db99cdef313779 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1556ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1556ffd700 1 -- 172.21.15.45:0/1456646034 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568005700 con 0x563a834c7860 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1556ffd700 1 -- 172.21.15.45:0/1456646034 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f15680263f0 con 0x563a834c7860 2023-09-23T09:34:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15680268b0 con 0x563a834c7860 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1556ffd700 1 -- 172.21.15.45:0/1456646034 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15680265f0 con 0x563a834c7860 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1589c46b80 10 client.25772 fetch_fsmap learned FSMap version 30 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.150+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563a8326c1f0 con 0x563a834c7860 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1556ffd700 1 -- 172.21.15.45:0/1456646034 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1568057020 con 0x563a834c7860 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1556ffd700 1 -- 172.21.15.45:0/1456646034 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f156802e070 con 0x563a834c7860 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1589c46b80 10 client.25772 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1589c46b80 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1589c46b80 20 client.25772 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1589c46b80 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a8353a2b0 0x563a834d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1589c46b80 4 client.25772 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:34:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a83413ad0 con 0x563a8353a2b0 2023-09-23T09:34:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.151+0000 7f1576ffd700 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a8353a2b0 0x563a834d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.152+0000 7f1576ffd700 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a8353a2b0 0x563a834d0e10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b15e81f421d42fa8 server_cookie=378dd5f3237c2f2c in_seq=0 out_seq=0 2023-09-23T09:34:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.152+0000 7f1556ffd700 10 client.25772 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.300+0000 7f1556ffd700 1 -- 172.21.15.45:0/1456646034 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a83413ad0 con 0x563a8353a2b0 2023-09-23T09:34:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.300+0000 7f1556ffd700 10 client.25772 handle_command_reply: tid=0 2023-09-23T09:34:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.300+0000 7f1589c46b80 1 client.25772 shutdown 2023-09-23T09:34:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.302+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 msgr2=0x563a834e0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.302+0000 7f1589c46b80 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834e0b30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1568011040 tx=0x7f156800b3c0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 shutdown_connections 2023-09-23T09:34:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1589c46b80 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a8353a2b0 0x563a834d0e10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1589c46b80 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a834daf20 0x563a83538740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1589c46b80 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a834c6f40 0x563a834e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1589c46b80 1 --2- 172.21.15.45:0/1456646034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a834c7860 0x563a834e0b30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1556ffd700 0 client.25772 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 >> 172.21.15.45:0/1456646034 conn(0x563a8343c200 msgr2=0x563a834e6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.304+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 shutdown_connections 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f1589c46b80 1 -- 172.21.15.45:0/1456646034 wait complete. 2023-09-23T09:34:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f1589c46b80 20 client.25772 trim_cache size 0 max 16384 2023-09-23T09:34:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:34:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:34:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:34:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:34:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f15600a1790 msgr2=0x7f15600a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f15600a1790 0x7f15600a3c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15680098f0 tx=0x7f156801c000 comp rx=0 tx=0).stop 2023-09-23T09:34:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f157819dcf0 msgr2=0x7f157819e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f157819dcf0 0x7f157819e170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f157000be20 tx=0x7f1570009ed0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 shutdown_connections 2023-09-23T09:34:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f157819dcf0 0x7f157819e170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1578101010 0x7f1578199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f15600a1790 0x7f15600a3c50 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15680098f0 tx=0x7f156801c000 comp rx=0 tx=0).stop 2023-09-23T09:34:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 --2- 172.21.15.45:0/1631583395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1578106130 0x7f1578199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.305+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 >> 172.21.15.45:0/1631583395 conn(0x7f15780fc8d0 msgr2=0x7f15780fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.307+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 shutdown_connections 2023-09-23T09:34:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:02.307+0000 7f157da5c700 1 -- 172.21.15.45:0/1631583395 wait complete. 2023-09-23T09:34:03.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:03 smithi098 ceph-mon[102386]: pgmap v142: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 866 op/s 2023-09-23T09:34:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:03 smithi045 ceph-mon[98627]: pgmap v142: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 866 op/s 2023-09-23T09:34:03.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:03 smithi123 ceph-mon[103577]: pgmap v142: 97 pgs: 97 active+clean; 6.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 866 op/s 2023-09-23T09:34:05.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:05 smithi098 ceph-mon[102386]: pgmap v143: 97 pgs: 97 active+clean; 6.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 MiB/s wr, 2.35k op/s 2023-09-23T09:34:05.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:05 smithi045 ceph-mon[98627]: pgmap v143: 97 pgs: 97 active+clean; 6.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 MiB/s wr, 2.35k op/s 2023-09-23T09:34:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:05 smithi123 ceph-mon[103577]: pgmap v143: 97 pgs: 97 active+clean; 6.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 MiB/s wr, 2.35k op/s 2023-09-23T09:34:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 260.0 s, 2482.7 tps, lat 49.629 ms stddev 126.086 2023-09-23T09:34:07.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:07 smithi098 ceph-mon[102386]: pgmap v144: 97 pgs: 97 active+clean; 6.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 1.91k op/s 2023-09-23T09:34:07.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:07 smithi045 ceph-mon[98627]: pgmap v144: 97 pgs: 97 active+clean; 6.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 1.91k op/s 2023-09-23T09:34:07.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:07 smithi123 ceph-mon[103577]: pgmap v144: 97 pgs: 97 active+clean; 6.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 1.91k op/s 2023-09-23T09:34:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:09 smithi098 ceph-mon[102386]: pgmap v145: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 48 MiB/s wr, 2.31k op/s 2023-09-23T09:34:09.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:09 smithi045 ceph-mon[98627]: pgmap v145: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 48 MiB/s wr, 2.31k op/s 2023-09-23T09:34:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:09 smithi123 ceph-mon[103577]: pgmap v145: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 48 MiB/s wr, 2.31k op/s 2023-09-23T09:34:11.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:11 smithi098 ceph-mon[102386]: pgmap v146: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-09-23T09:34:11.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:11 smithi045 ceph-mon[98627]: pgmap v146: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-09-23T09:34:11.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:11 smithi123 ceph-mon[103577]: pgmap v146: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-09-23T09:34:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 265.0 s, 2677.8 tps, lat 37.560 ms stddev 43.220 2023-09-23T09:34:13.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:13 smithi045 ceph-mon[98627]: pgmap v147: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-09-23T09:34:13.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:13 smithi123 ceph-mon[103577]: pgmap v147: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-09-23T09:34:14.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:13 smithi098 ceph-mon[102386]: pgmap v147: 97 pgs: 97 active+clean; 6.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-09-23T09:34:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 -- 172.21.15.45:0/472386689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 msgr2=0x7f48e01013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 --2- 172.21.15.45:0/472386689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 0x7f48e01013f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b0a0 tx=0x7f48d800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 -- 172.21.15.45:0/472386689 shutdown_connections 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 --2- 172.21.15.45:0/472386689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e0106130 0x7f48e010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 --2- 172.21.15.45:0/472386689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 0x7f48e01013f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 --2- 172.21.15.45:0/472386689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e01019c0 0x7f48e0105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 -- 172.21.15.45:0/472386689 >> 172.21.15.45:0/472386689 conn(0x7f48e00fc8d0 msgr2=0x7f48e00fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 -- 172.21.15.45:0/472386689 shutdown_connections 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.477+0000 7f48e68e7700 1 -- 172.21.15.45:0/472386689 wait complete. 2023-09-23T09:34:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 Processor -- start 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 -- start start 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 0x7f48e0073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e0106130 0x7f48e0073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e0073ba0 0x7f48e01a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e01a9da0 con 0x7f48e0106130 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e01a9ee0 con 0x7f48e0101010 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.478+0000 7f48e68e7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e01aa020 con 0x7f48e0073ba0 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.479+0000 7f48e4e84700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e0073ba0 0x7f48e01a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.479+0000 7f48e4e84700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e0073ba0 0x7f48e01a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36468/0 (socket says 172.21.15.45:36468) 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.479+0000 7f48e4e84700 1 -- 172.21.15.45:0/1212175818 learned_addr learned my addr 172.21.15.45:0/1212175818 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.479+0000 7f48dffff700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 0x7f48e0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.479+0000 7f48df7fe700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e0106130 0x7f48e0073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dffff700 1 -- 172.21.15.45:0/1212175818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e0073ba0 msgr2=0x7f48e01a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dffff700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e0073ba0 0x7f48e01a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dffff700 1 -- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e0106130 msgr2=0x7f48e0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dffff700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e0106130 0x7f48e0073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dffff700 1 -- 172.21.15.45:0/1212175818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8009d00 con 0x7f48e0101010 2023-09-23T09:34:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48df7fe700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e0106130 0x7f48e0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dffff700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 0x7f48e0073120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b0a0 tx=0x7f48d8007640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dd7fa700 1 -- 172.21.15.45:0/1212175818 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d800e040 con 0x7f48e0101010 2023-09-23T09:34:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dd7fa700 1 -- 172.21.15.45:0/1212175818 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d8003c00 con 0x7f48e0101010 2023-09-23T09:34:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48dd7fa700 1 -- 172.21.15.45:0/1212175818 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80052a0 con 0x7f48e0101010 2023-09-23T09:34:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e01aa2a0 con 0x7f48e0101010 2023-09-23T09:34:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.480+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e01aa6d0 con 0x7f48e0101010 2023-09-23T09:34:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.483+0000 7f48dd7fa700 1 -- 172.21.15.45:0/1212175818 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f48d801d470 con 0x7f48e0101010 2023-09-23T09:34:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.484+0000 7f48dd7fa700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f48c80a1790 0x7f48c80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.484+0000 7f48dd7fa700 1 -- 172.21.15.45:0/1212175818 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f48d80ce5a0 con 0x7f48e0101010 2023-09-23T09:34:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.485+0000 7f48df7fe700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f48c80a1790 0x7f48c80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.485+0000 7f48df7fe700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f48c80a1790 0x7f48c80a3c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48e0074ab0 tx=0x7f48d0009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.485+0000 7f48f2ad1b80 1 Processor -- start 2023-09-23T09:34:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.485+0000 7f48f2ad1b80 1 -- start start 2023-09-23T09:34:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f01320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8e6a400 con 0x5585e8f01860 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8e5bbc0 con 0x5585e8f00f40 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8e594e0 con 0x5585e8f0c710 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34370/0 (socket says 172.21.15.45:34370) 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 -- 172.21.15.45:0/2630718911 learned_addr learned my addr 172.21.15.45:0/2630718911 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48dffff700 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f01320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48e4e84700 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 -- 172.21.15.45:0/2630718911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 msgr2=0x5585e8f0eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f0eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 -- 172.21.15.45:0/2630718911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 msgr2=0x5585e8f01320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f01320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.486+0000 7f48df7fe700 1 -- 172.21.15.45:0/2630718911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e8eb7870 con 0x5585e8f01860 2023-09-23T09:34:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.487+0000 7f48e4e84700 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f0eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:34:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.487+0000 7f48dffff700 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f01320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.487+0000 7f48df7fe700 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0c1d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f48d00043c0 tx=0x7f48d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15413fad08724676 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.487+0000 7f48c67fc700 1 -- 172.21.15.45:0/2630718911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0003d10 con 0x5585e8f01860 2023-09-23T09:34:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.487+0000 7f48c67fc700 1 -- 172.21.15.45:0/2630718911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d0003e70 con 0x5585e8f01860 2023-09-23T09:34:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.487+0000 7f48c67fc700 1 -- 172.21.15.45:0/2630718911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d001e7e0 con 0x5585e8f01860 2023-09-23T09:34:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/2630718911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 msgr2=0x5585e8f0c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0c1d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f48d00043c0 tx=0x7f48d0014000 comp rx=0 tx=0).stop 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/2630718911 shutdown_connections 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f0eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f01320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/2630718911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0c1d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/2630718911 >> 172.21.15.45:0/2630718911 conn(0x5585e8e76200 msgr2=0x5585e8efd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/2630718911 shutdown_connections 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.488+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/2630718911 wait complete. 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.489+0000 7f48f2ad1b80 1 -- start start 2023-09-23T09:34:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.489+0000 7f48f2ad1b80 1 Processor -- start 2023-09-23T09:34:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.490+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f00f40 0x5585e8f0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.490+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.490+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f0c710 0x5585e8f09b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.490+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8e54dd0 con 0x5585e8f0c710 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.490+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8df16b0 con 0x5585e8f01860 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.490+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8df1350 con 0x5585e8f00f40 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.490+0000 7f48f2ad1b80 10 client.0 _ll_register_callbacks cb 0x5585e8c4e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48e4e84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f0c710 0x5585e8f09b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48dffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f00f40 0x5585e8f0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48e4e84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f0c710 0x5585e8f09b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34378/0 (socket says 172.21.15.45:34378) 2023-09-23T09:34:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48e4e84700 1 -- 172.21.15.45:0/35738657 learned_addr learned my addr 172.21.15.45:0/35738657 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48dffff700 1 -- 172.21.15.45:0/35738657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 msgr2=0x5585e8f093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48dffff700 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48dffff700 1 -- 172.21.15.45:0/35738657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f0c710 msgr2=0x5585e8f09b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48dffff700 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f0c710 0x5585e8f09b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48dffff700 1 -- 172.21.15.45:0/35738657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e01aa6d0 con 0x5585e8f00f40 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48dffff700 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f00f40 0x5585e8f0c140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f48d80cbce0 tx=0x7f48d800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88ed46e900151a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48bf7fe700 1 -- 172.21.15.45:0/35738657 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d809b590 con 0x5585e8f00f40 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48bf7fe700 1 -- 172.21.15.45:0/35738657 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f48d80b1060 con 0x5585e8f00f40 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585e8ef5090 con 0x5585e8f00f40 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48bf7fe700 1 -- 172.21.15.45:0/35738657 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d809f6e0 con 0x5585e8f00f40 2023-09-23T09:34:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48f2ad1b80 10 client.25777 fetch_fsmap learned FSMap version 30 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.491+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f48d0017070 con 0x5585e8f00f40 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48bf7fe700 1 -- 172.21.15.45:0/35738657 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585e8ef5090 con 0x5585e8f00f40 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48bf7fe700 1 -- 172.21.15.45:0/35738657 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48d801d800 con 0x5585e8f00f40 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48f2ad1b80 10 client.25777 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48f2ad1b80 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48f2ad1b80 20 client.25777 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f2ced0 0x5585e8f2f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48f2ad1b80 4 client.25777 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585e8e4ddb0 con 0x5585e8f2ced0 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.492+0000 7f48df7fe700 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f2ced0 0x5585e8f2f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.493+0000 7f48bf7fe700 10 client.25777 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.493+0000 7f48df7fe700 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f2ced0 0x5585e8f2f2f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ebe7dc41d745d7 server_cookie=6ebc3a81545ae7c4 in_seq=0 out_seq=0 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.493+0000 7f48bf7fe700 1 -- 172.21.15.45:0/35738657 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5585e8e4ddb0 con 0x5585e8f2ced0 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.493+0000 7f48bf7fe700 10 client.25777 handle_command_reply: tid=0 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.493+0000 7f48f2ad1b80 1 client.25777 shutdown 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f00f40 msgr2=0x5585e8f0c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f00f40 0x5585e8f0c140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f48d80cbce0 tx=0x7f48d800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 shutdown_connections 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f2ced0 0x5585e8f2f2f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f0c710 0x5585e8f09b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f00f40 0x5585e8f0c140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/35738657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 >> 172.21.15.45:0/35738657 conn(0x5585e8e76200 msgr2=0x5585e8efcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48bf7fe700 0 client.25777 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 shutdown_connections 2023-09-23T09:34:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/35738657 wait complete. 2023-09-23T09:34:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.494+0000 7f48f2ad1b80 20 client.25777 trim_cache size 0 max 16384 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 Processor -- start 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 -- start start 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f08930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8cd58c0 con 0x5585e8f01860 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8f02190 con 0x5585e8f00f40 2023-09-23T09:34:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.516+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8efe180 con 0x5585e8f0c710 2023-09-23T09:34:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.523+0000 7f48dffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f08930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.523+0000 7f48dffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f08930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55736/0 (socket says 172.21.15.45:55736) 2023-09-23T09:34:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.523+0000 7f48dffff700 1 -- 172.21.15.45:0/4136251367 learned_addr learned my addr 172.21.15.45:0/4136251367 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.524+0000 7f48df7fe700 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.525+0000 7f48df7fe700 1 -- 172.21.15.45:0/4136251367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 msgr2=0x5585e8f046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.525+0000 7f48df7fe700 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.525+0000 7f48df7fe700 1 -- 172.21.15.45:0/4136251367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 msgr2=0x5585e8f08930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.525+0000 7f48df7fe700 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f08930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.525+0000 7f48df7fe700 1 -- 172.21.15.45:0/4136251367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e8c7fd80 con 0x5585e8f01860 2023-09-23T09:34:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.528+0000 7f48df7fe700 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0bce0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f48d0004390 tx=0x7f48d0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ab14a7a307b38f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.528+0000 7f48c6ffd700 1 -- 172.21.15.45:0/4136251367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d002b040 con 0x5585e8f01860 2023-09-23T09:34:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.528+0000 7f48c6ffd700 1 -- 172.21.15.45:0/4136251367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d002c040 con 0x5585e8f01860 2023-09-23T09:34:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.528+0000 7f48c6ffd700 1 -- 172.21.15.45:0/4136251367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d00269e0 con 0x5585e8f01860 2023-09-23T09:34:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.529+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/4136251367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 msgr2=0x5585e8f0bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.529+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0bce0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f48d0004390 tx=0x7f48d0005ce0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.536+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/4136251367 shutdown_connections 2023-09-23T09:34:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.536+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.536+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f00f40 0x5585e8f08930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.536+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/4136251367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f01860 0x5585e8f0bce0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.536+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/4136251367 >> 172.21.15.45:0/4136251367 conn(0x5585e8e76200 msgr2=0x5585e8f2eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.536+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/4136251367 shutdown_connections 2023-09-23T09:34:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.536+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/4136251367 wait complete. 2023-09-23T09:34:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.537+0000 7f48f2ad1b80 1 -- start start 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.537+0000 7f48f2ad1b80 1 Processor -- start 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.537+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f00f40 0x5585e8f10bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.537+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48f2ad1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f7fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8f11a00 con 0x5585e8f00f40 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8f80240 con 0x5585e8f01860 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48f2ad1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8f91390 con 0x5585e8f0c710 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48f2ad1b80 10 client.0 _ll_register_callbacks cb 0x7f48d809b0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f00f40 0x5585e8f10bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f00f40 0x5585e8f10bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34390/0 (socket says 172.21.15.45:34390) 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 -- 172.21.15.45:0/3906831832 learned_addr learned my addr 172.21.15.45:0/3906831832 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48df7fe700 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48e4e84700 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f7fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 -- 172.21.15.45:0/3906831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 msgr2=0x5585e8f7fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f7fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 -- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 msgr2=0x5585e8f110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 -- 172.21.15.45:0/3906831832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585e8eb7870 con 0x5585e8f00f40 2023-09-23T09:34:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48dffff700 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f00f40 0x5585e8f10bb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f48d80cc2e0 tx=0x7f48d8097210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a73004dbe7dd43 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.538+0000 7f48bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:34:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.539+0000 7f48bf7fe700 1 -- 172.21.15.45:0/3906831832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d809ac80 con 0x5585e8f00f40 2023-09-23T09:34:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.539+0000 7f48bf7fe700 1 -- 172.21.15.45:0/3906831832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f48d801d930 con 0x5585e8f00f40 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.539+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48d0003e70 con 0x5585e8f00f40 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.539+0000 7f48bf7fe700 1 -- 172.21.15.45:0/3906831832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d809b590 con 0x5585e8f00f40 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.539+0000 7f48f2ad1b80 10 client.16095 fetch_fsmap learned FSMap version 30 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.539+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5585e8c68c00 con 0x5585e8f00f40 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48bf7fe700 1 -- 172.21.15.45:0/3906831832 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d0003e70 con 0x5585e8f00f40 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48bf7fe700 1 -- 172.21.15.45:0/3906831832 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48d80a3400 con 0x5585e8f00f40 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48f2ad1b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48f2ad1b80 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48f2ad1b80 20 client.16095 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f92540 0x5585e8f94930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48f2ad1b80 4 client.16095 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:34:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5585e8e4dad0 con 0x5585e8f92540 2023-09-23T09:34:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.540+0000 7f48df7fe700 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f92540 0x5585e8f94930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.541+0000 7f48bf7fe700 10 client.16095 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.541+0000 7f48df7fe700 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f92540 0x5585e8f94930 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69742b456dbac701 server_cookie=bce65c667be1f815 in_seq=0 out_seq=0 2023-09-23T09:34:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.564+0000 7f48bf7fe700 1 -- 172.21.15.45:0/3906831832 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5585e8e4dad0 con 0x5585e8f92540 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.564+0000 7f48bf7fe700 10 client.16095 handle_command_reply: tid=0 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.564+0000 7f48f2ad1b80 1 client.16095 shutdown 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.564+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f00f40 msgr2=0x5585e8f10bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.564+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f00f40 0x5585e8f10bb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f48d80cc2e0 tx=0x7f48d8097210 comp rx=0 tx=0).stop 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 shutdown_connections 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5585e8f92540 0x5585e8f94930 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585e8f0c710 0x5585e8f7fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e8f00f40 0x5585e8f10bb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48bf7fe700 0 client.16095 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 --2- 172.21.15.45:0/3906831832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585e8f01860 0x5585e8f110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 >> 172.21.15.45:0/3906831832 conn(0x5585e8e76200 msgr2=0x5585e8f046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 shutdown_connections 2023-09-23T09:34:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 1 -- 172.21.15.45:0/3906831832 wait complete. 2023-09-23T09:34:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48f2ad1b80 20 client.16095 trim_cache size 0 max 16384 2023-09-23T09:34:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:34:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:34:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:34:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:34:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f48c80a1790 msgr2=0x7f48c80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f48c80a1790 0x7f48c80a3c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48e0074ab0 tx=0x7f48d0009260 comp rx=0 tx=0).stop 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 msgr2=0x7f48e0073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 0x7f48e0073120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b0a0 tx=0x7f48d8007640 comp rx=0 tx=0).stop 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 shutdown_connections 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e0073ba0 0x7f48e01a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0101010 0x7f48e0073120 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f48c80a1790 0x7f48c80a3c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 --2- 172.21.15.45:0/1212175818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e0106130 0x7f48e0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 >> 172.21.15.45:0/1212175818 conn(0x7f48e00fc8d0 msgr2=0x7f48e00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.565+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 shutdown_connections 2023-09-23T09:34:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:15.566+0000 7f48e68e7700 1 -- 172.21.15.45:0/1212175818 wait complete. 2023-09-23T09:34:15.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:15 smithi045 ceph-mon[98627]: pgmap v148: 97 pgs: 97 active+clean; 6.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 44 MiB/s wr, 2.25k op/s 2023-09-23T09:34:15.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:15 smithi123 ceph-mon[103577]: pgmap v148: 97 pgs: 97 active+clean; 6.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 44 MiB/s wr, 2.25k op/s 2023-09-23T09:34:16.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:15 smithi098 ceph-mon[102386]: pgmap v148: 97 pgs: 97 active+clean; 6.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 44 MiB/s wr, 2.25k op/s 2023-09-23T09:34:16.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 270.0 s, 2444.8 tps, lat 40.799 ms stddev 52.913 2023-09-23T09:34:18.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:17 smithi098 ceph-mon[102386]: pgmap v149: 97 pgs: 97 active+clean; 6.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 14 MiB/s wr, 701 op/s 2023-09-23T09:34:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:17 smithi045 ceph-mon[98627]: pgmap v149: 97 pgs: 97 active+clean; 6.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 14 MiB/s wr, 701 op/s 2023-09-23T09:34:18.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:17 smithi123 ceph-mon[103577]: pgmap v149: 97 pgs: 97 active+clean; 6.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 14 MiB/s wr, 701 op/s 2023-09-23T09:34:19.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:18 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:19.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:18 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:19.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:18 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:20.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:19 smithi098 ceph-mon[102386]: pgmap v150: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 19 MiB/s wr, 999 op/s 2023-09-23T09:34:20.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:19 smithi045 ceph-mon[98627]: pgmap v150: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 19 MiB/s wr, 999 op/s 2023-09-23T09:34:20.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:19 smithi123 ceph-mon[103577]: pgmap v150: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 19 MiB/s wr, 999 op/s 2023-09-23T09:34:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 275.0 s, 2691.2 tps, lat 37.265 ms stddev 44.921 2023-09-23T09:34:22.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:21 smithi098 ceph-mon[102386]: pgmap v151: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 597 op/s 2023-09-23T09:34:22.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:21 smithi045 ceph-mon[98627]: pgmap v151: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 597 op/s 2023-09-23T09:34:22.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:21 smithi123 ceph-mon[103577]: pgmap v151: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 597 op/s 2023-09-23T09:34:24.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:23 smithi098 ceph-mon[102386]: pgmap v152: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 597 op/s 2023-09-23T09:34:24.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:23 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:24.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:23 smithi045 ceph-mon[98627]: pgmap v152: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 597 op/s 2023-09-23T09:34:24.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:23 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:24.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:23 smithi123 ceph-mon[103577]: pgmap v152: 97 pgs: 97 active+clean; 6.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 597 op/s 2023-09-23T09:34:24.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:23 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:26.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:25 smithi098 ceph-mon[102386]: pgmap v153: 97 pgs: 97 active+clean; 6.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 943 op/s 2023-09-23T09:34:26.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:25 smithi045 ceph-mon[98627]: pgmap v153: 97 pgs: 97 active+clean; 6.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 943 op/s 2023-09-23T09:34:26.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:25 smithi123 ceph-mon[103577]: pgmap v153: 97 pgs: 97 active+clean; 6.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 943 op/s 2023-09-23T09:34:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 280.0 s, 2272.5 tps, lat 43.986 ms stddev 52.020 2023-09-23T09:34:28.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:27 smithi098 ceph-mon[102386]: pgmap v154: 97 pgs: 97 active+clean; 6.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 643 op/s 2023-09-23T09:34:28.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:27 smithi045 ceph-mon[98627]: pgmap v154: 97 pgs: 97 active+clean; 6.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 643 op/s 2023-09-23T09:34:28.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:27 smithi123 ceph-mon[103577]: pgmap v154: 97 pgs: 97 active+clean; 6.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 643 op/s 2023-09-23T09:34:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 -- 172.21.15.45:0/596209223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254103800 msgr2=0x7f5254103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 --2- 172.21.15.45:0/596209223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254103800 0x7f5254103c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5244009990 tx=0x7f5244009ca0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 -- 172.21.15.45:0/596209223 shutdown_connections 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 --2- 172.21.15.45:0/596209223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254103800 0x7f5254103c90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 --2- 172.21.15.45:0/596209223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254105d00 0x7f5254069140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 --2- 172.21.15.45:0/596209223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5254069680 0x7f5254069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 -- 172.21.15.45:0/596209223 >> 172.21.15.45:0/596209223 conn(0x7f5254076dd0 msgr2=0x7f52540771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 -- 172.21.15.45:0/596209223 shutdown_connections 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 -- 172.21.15.45:0/596209223 wait complete. 2023-09-23T09:34:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.732+0000 7f5259f39700 1 Processor -- start 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5259f39700 1 -- start start 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5259f39700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5254069680 0x7f52541a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5259f39700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254103800 0x7f52541a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5259f39700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254105d00 0x7f525419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5259f39700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525419db50 con 0x7f5254103800 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5259f39700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525419dcc0 con 0x7f5254069680 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5259f39700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525419de80 con 0x7f5254105d00 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254103800 0x7f52541a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5253fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254105d00 0x7f525419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254103800 0x7f52541a3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34402/0 (socket says 172.21.15.45:34402) 2023-09-23T09:34:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 -- 172.21.15.45:0/2133588916 learned_addr learned my addr 172.21.15.45:0/2133588916 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f52537fe700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5254069680 0x7f52541a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 -- 172.21.15.45:0/2133588916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254105d00 msgr2=0x7f525419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254105d00 0x7f525419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 -- 172.21.15.45:0/2133588916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5254069680 msgr2=0x7f52541a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5254069680 0x7f52541a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 -- 172.21.15.45:0/2133588916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52440095f0 con 0x7f5254103800 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.733+0000 7f5252ffd700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254103800 0x7f52541a3af0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f524800d950 tx=0x7f524800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.734+0000 7f5250ff9700 1 -- 172.21.15.45:0/2133588916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800f890 con 0x7f5254103800 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.734+0000 7f5250ff9700 1 -- 172.21.15.45:0/2133588916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5248010460 con 0x7f5254103800 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.734+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525419e160 con 0x7f5254103800 2023-09-23T09:34:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.734+0000 7f5250ff9700 1 -- 172.21.15.45:0/2133588916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800e610 con 0x7f5254103800 2023-09-23T09:34:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.734+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52541aa580 con 0x7f5254103800 2023-09-23T09:34:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.736+0000 7f5250ff9700 1 -- 172.21.15.45:0/2133588916 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f52480108d0 con 0x7f5254103800 2023-09-23T09:34:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.738+0000 7f5266123b80 1 Processor -- start 2023-09-23T09:34:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.738+0000 7f5266123b80 1 -- start start 2023-09-23T09:34:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.738+0000 7f5250ff9700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f523c0a1810 0x7f523c0a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.738+0000 7f5266123b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966bf40 0x563bf966c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.738+0000 7f5266123b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966c860 0x563bf96771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5266123b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9679b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5266123b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf95d5400 con 0x563bf9677710 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5266123b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf95c6bc0 con 0x563bf966bf40 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5250ff9700 1 -- 172.21.15.45:0/2133588916 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f52480cd0c0 con 0x7f5254103800 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5266123b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf95c44e0 con 0x563bf966c860 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5253fff700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f523c0a1810 0x7f523c0a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5253fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9679b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966c860 0x563bf96771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5253fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9679b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34412/0 (socket says 172.21.15.45:34412) 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f52537fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966bf40 0x563bf966c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5253fff700 1 -- 172.21.15.45:0/569374438 learned_addr learned my addr 172.21.15.45:0/569374438 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5252ffd700 1 -- 172.21.15.45:0/569374438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966bf40 msgr2=0x563bf966c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5252ffd700 1 --2- 172.21.15.45:0/569374438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966bf40 0x563bf966c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5252ffd700 1 -- 172.21.15.45:0/569374438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 msgr2=0x563bf9679b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5252ffd700 1 --2- 172.21.15.45:0/569374438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9679b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.739+0000 7f5252ffd700 1 -- 172.21.15.45:0/569374438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bf9622870 con 0x563bf966c860 2023-09-23T09:34:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.740+0000 7f5252ffd700 1 --2- 172.21.15.45:0/569374438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966c860 0x563bf96771d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f52480ca7a0 tx=0x7f524809a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7337a34146a173dd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.740+0000 7f5239ffb700 1 -- 172.21.15.45:0/569374438 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480a2350 con 0x563bf966c860 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.740+0000 7f5239ffb700 1 -- 172.21.15.45:0/569374438 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52480caa20 con 0x563bf966c860 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.740+0000 7f5239ffb700 1 -- 172.21.15.45:0/569374438 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248099710 con 0x563bf966c860 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 -- 172.21.15.45:0/569374438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966c860 msgr2=0x563bf96771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5253fff700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f523c0a1810 0x7f523c0a3cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5244015000 tx=0x7f5244005be0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 --2- 172.21.15.45:0/569374438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966c860 0x563bf96771d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f52480ca7a0 tx=0x7f524809a6c0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 -- 172.21.15.45:0/569374438 shutdown_connections 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 --2- 172.21.15.45:0/569374438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9679b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 --2- 172.21.15.45:0/569374438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966bf40 0x563bf966c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 --2- 172.21.15.45:0/569374438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966c860 0x563bf96771d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 -- 172.21.15.45:0/569374438 >> 172.21.15.45:0/569374438 conn(0x563bf95e1200 msgr2=0x563bf96683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 -- 172.21.15.45:0/569374438 shutdown_connections 2023-09-23T09:34:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 -- 172.21.15.45:0/569374438 wait complete. 2023-09-23T09:34:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.741+0000 7f5266123b80 1 -- start start 2023-09-23T09:34:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 1 Processor -- start 2023-09-23T09:34:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf9677150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf96743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9674b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf95bfdd0 con 0x563bf9677710 2023-09-23T09:34:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf955c6b0 con 0x563bf966c860 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf955c350 con 0x563bf966bf40 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.742+0000 7f5266123b80 10 client.0 _ll_register_callbacks cb 0x563bf93b97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf9677150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf9677150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36522/0 (socket says 172.21.15.45:36522) 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 -- 172.21.15.45:0/875722365 learned_addr learned my addr 172.21.15.45:0/875722365 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f5253fff700 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9674b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 -- 172.21.15.45:0/875722365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 msgr2=0x563bf96743d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf96743d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 -- 172.21.15.45:0/875722365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 msgr2=0x563bf9674b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9674b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f52537fe700 1 -- 172.21.15.45:0/875722365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52541aa580 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.743+0000 7f5253fff700 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9674b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f52537fe700 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf9677150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f524c007b20 tx=0x7f524c007ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9eaaacef340ac71 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5232ffd700 1 -- 172.21.15.45:0/875722365 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00e040 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5232ffd700 1 -- 172.21.15.45:0/875722365 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f524c012b80 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bf9660090 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5232ffd700 1 -- 172.21.15.45:0/875722365 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f524c012d80 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5266123b80 10 client.25792 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563bf93ead80 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5232ffd700 1 -- 172.21.15.45:0/875722365 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bf9660090 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.744+0000 7f5232ffd700 1 -- 172.21.15.45:0/875722365 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f524c014040 con 0x563bf966bf40 2023-09-23T09:34:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5266123b80 10 client.25792 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5266123b80 10 client.25792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5266123b80 20 client.25792 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5266123b80 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96872d0 0x563bf96896f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5266123b80 4 client.25792 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bf95b8db0 con 0x563bf96872d0 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5252ffd700 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96872d0 0x563bf96896f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5232ffd700 10 client.25792 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.745+0000 7f5252ffd700 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96872d0 0x563bf96896f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988624343b4fbcac server_cookie=34393a03e13b8a06 in_seq=0 out_seq=0 2023-09-23T09:34:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.746+0000 7f5232ffd700 1 -- 172.21.15.45:0/875722365 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563bf95b8db0 con 0x563bf96872d0 2023-09-23T09:34:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.746+0000 7f5232ffd700 10 client.25792 handle_command_reply: tid=0 2023-09-23T09:34:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.746+0000 7f5266123b80 1 client.25792 shutdown 2023-09-23T09:34:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 msgr2=0x563bf9677150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf9677150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f524c007b20 tx=0x7f524c007ee0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 shutdown_connections 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96872d0 0x563bf96896f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf9677710 0x563bf9674b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf9677150 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 --2- 172.21.15.45:0/875722365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf96743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 >> 172.21.15.45:0/875722365 conn(0x563bf95e1200 msgr2=0x563bf9667f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5232ffd700 0 client.25792 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 shutdown_connections 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 1 -- 172.21.15.45:0/875722365 wait complete. 2023-09-23T09:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.747+0000 7f5266123b80 20 client.25792 trim_cache size 0 max 16384 2023-09-23T09:34:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.767+0000 7f5266123b80 1 Processor -- start 2023-09-23T09:34:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.767+0000 7f5266123b80 1 -- start start 2023-09-23T09:34:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.767+0000 7f5266123b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf966ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.767+0000 7f5266123b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966c860 0x563bf967e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.767+0000 7f5266123b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf9677710 0x563bf96892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5266123b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf94408c0 con 0x563bf966c860 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5266123b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf967e810 con 0x563bf9677710 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5266123b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf96729b0 con 0x563bf966bf40 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966c860 0x563bf967e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f52537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf966ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966c860 0x563bf967e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34426/0 (socket says 172.21.15.45:34426) 2023-09-23T09:34:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 -- 172.21.15.45:0/7765660 learned_addr learned my addr 172.21.15.45:0/7765660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 -- 172.21.15.45:0/7765660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 msgr2=0x563bf966ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 --2- 172.21.15.45:0/7765660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf966ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 -- 172.21.15.45:0/7765660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf9677710 msgr2=0x563bf96892e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:34:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 --2- 172.21.15.45:0/7765660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf9677710 0x563bf96892e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.768+0000 7f5252ffd700 1 -- 172.21.15.45:0/7765660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bf93d3c00 con 0x563bf966c860 2023-09-23T09:34:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.769+0000 7f5252ffd700 1 --2- 172.21.15.45:0/7765660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966c860 0x563bf967e260 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5248098310 tx=0x7f5248000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa00f3c6210784ce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.769+0000 7f523a7fc700 1 -- 172.21.15.45:0/7765660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480b6070 con 0x563bf966c860 2023-09-23T09:34:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.769+0000 7f523a7fc700 1 -- 172.21.15.45:0/7765660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52480a2350 con 0x563bf966c860 2023-09-23T09:34:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.769+0000 7f523a7fc700 1 -- 172.21.15.45:0/7765660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248014070 con 0x563bf966c860 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.770+0000 7f5266123b80 1 -- 172.21.15.45:0/7765660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966c860 msgr2=0x563bf967e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.770+0000 7f5266123b80 1 --2- 172.21.15.45:0/7765660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966c860 0x563bf967e260 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5248098310 tx=0x7f5248000bc0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.770+0000 7f5266123b80 1 -- 172.21.15.45:0/7765660 shutdown_connections 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.770+0000 7f5266123b80 1 --2- 172.21.15.45:0/7765660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf9677710 0x563bf96892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.770+0000 7f5266123b80 1 --2- 172.21.15.45:0/7765660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf966bf40 0x563bf966ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.770+0000 7f5266123b80 1 --2- 172.21.15.45:0/7765660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966c860 0x563bf967e260 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.770+0000 7f5266123b80 1 -- 172.21.15.45:0/7765660 >> 172.21.15.45:0/7765660 conn(0x563bf95e1200 msgr2=0x563bf9667590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.771+0000 7f5266123b80 1 -- 172.21.15.45:0/7765660 shutdown_connections 2023-09-23T09:34:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.771+0000 7f5266123b80 1 -- 172.21.15.45:0/7765660 wait complete. 2023-09-23T09:34:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 -- start start 2023-09-23T09:34:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 Processor -- start 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966bf40 0x563bf967d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf967db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf9677710 0x563bf96da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf967e2d0 con 0x563bf966bf40 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf96da6b0 con 0x563bf966c860 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf96da870 con 0x563bf9677710 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f5266123b80 10 client.0 _ll_register_callbacks cb 0x7f524c009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.772+0000 7f52537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966bf40 0x563bf967d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf967db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966bf40 0x563bf967d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34428/0 (socket says 172.21.15.45:34428) 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 -- 172.21.15.45:0/581434981 learned_addr learned my addr 172.21.15.45:0/581434981 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5253fff700 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf9677710 0x563bf96da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 -- 172.21.15.45:0/581434981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf9677710 msgr2=0x563bf96da140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf9677710 0x563bf96da140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 -- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 msgr2=0x563bf967db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf967db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 -- 172.21.15.45:0/581434981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bf9622870 con 0x563bf966bf40 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5252ffd700 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf967db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f52537fe700 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966bf40 0x563bf967d620 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f524c009ee0 tx=0x7f524c016e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78deab30e2229cd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5232ffd700 1 -- 172.21.15.45:0/581434981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00e040 con 0x563bf966bf40 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5232ffd700 1 -- 172.21.15.45:0/581434981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f524c016460 con 0x563bf966bf40 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52480caa20 con 0x563bf966bf40 2023-09-23T09:34:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5232ffd700 1 -- 172.21.15.45:0/581434981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f524c0095e0 con 0x563bf966bf40 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5266123b80 10 client.16113 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.773+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563bf94111f0 con 0x563bf966bf40 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5232ffd700 1 -- 172.21.15.45:0/581434981 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524c037020 con 0x563bf966bf40 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5232ffd700 1 -- 172.21.15.45:0/581434981 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f524c014070 con 0x563bf966bf40 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5266123b80 10 client.16113 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5266123b80 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5266123b80 20 client.16113 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5266123b80 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96dc0e0 0x563bf96de500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5266123b80 4 client.16113 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563bf95b8ad0 con 0x563bf96dc0e0 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5252ffd700 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96dc0e0 0x563bf96de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.774+0000 7f5252ffd700 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96dc0e0 0x563bf96de500 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44dc9bcacbb303f server_cookie=10d9c5364ede3831 in_seq=0 out_seq=0 2023-09-23T09:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.775+0000 7f5232ffd700 10 client.16113 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5232ffd700 1 -- 172.21.15.45:0/581434981 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563bf95b8ad0 con 0x563bf96dc0e0 2023-09-23T09:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5232ffd700 10 client.16113 handle_command_reply: tid=0 2023-09-23T09:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 client.16113 shutdown 2023-09-23T09:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966bf40 msgr2=0x563bf967d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966bf40 0x563bf967d620 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f524c009ee0 tx=0x7f524c016e40 comp rx=0 tx=0).stop 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 shutdown_connections 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563bf96dc0e0 0x563bf96de500 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bf9677710 0x563bf96da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bf966bf40 0x563bf967d620 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 --2- 172.21.15.45:0/581434981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bf966c860 0x563bf967db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5232ffd700 0 client.16113 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 >> 172.21.15.45:0/581434981 conn(0x563bf95e1200 msgr2=0x563bf96892e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 shutdown_connections 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 1 -- 172.21.15.45:0/581434981 wait complete. 2023-09-23T09:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.797+0000 7f5266123b80 20 client.16113 trim_cache size 0 max 16384 2023-09-23T09:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f523c0a1810 msgr2=0x7f523c0a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f523c0a1810 0x7f523c0a3cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5244015000 tx=0x7f5244005be0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254103800 msgr2=0x7f52541a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254103800 0x7f52541a3af0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f524800d950 tx=0x7f524800dc60 comp rx=0 tx=0).stop 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 shutdown_connections 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254105d00 0x7f525419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5254069680 0x7f52541a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f523c0a1810 0x7f523c0a3cd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 --2- 172.21.15.45:0/2133588916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5254103800 0x7f52541a3af0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.798+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 >> 172.21.15.45:0/2133588916 conn(0x7f5254076dd0 msgr2=0x7f52540fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.799+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 shutdown_connections 2023-09-23T09:34:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:28.799+0000 7f5259f39700 1 -- 172.21.15.45:0/2133588916 wait complete. 2023-09-23T09:34:29.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:28 smithi098 ceph-mon[102386]: pgmap v155: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 993 op/s 2023-09-23T09:34:29.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:28 smithi045 ceph-mon[98627]: pgmap v155: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 993 op/s 2023-09-23T09:34:29.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:28 smithi123 ceph-mon[103577]: pgmap v155: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 993 op/s 2023-09-23T09:34:29.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:34:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.071+0000 7fcc587a9700 1 -- 172.21.15.45:0/4164212931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010f940 msgr2=0x7fcc5010d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.071+0000 7fcc587a9700 1 --2- 172.21.15.45:0/4164212931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010f940 0x7fcc5010d5a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800b0a0 tx=0x7fcc4800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.078+0000 7fcc587a9700 1 -- 172.21.15.45:0/4164212931 shutdown_connections 2023-09-23T09:34:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.078+0000 7fcc587a9700 1 --2- 172.21.15.45:0/4164212931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5011a600 0x7fcc5011c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.078+0000 7fcc587a9700 1 --2- 172.21.15.45:0/4164212931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5010f4d0 0x7fcc5010d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.078+0000 7fcc587a9700 1 --2- 172.21.15.45:0/4164212931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010f940 0x7fcc5010d5a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.078+0000 7fcc587a9700 1 -- 172.21.15.45:0/4164212931 >> 172.21.15.45:0/4164212931 conn(0x7fcc5006d0e0 msgr2=0x7fcc5006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.080+0000 7fcc587a9700 1 -- 172.21.15.45:0/4164212931 shutdown_connections 2023-09-23T09:34:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.083+0000 7fcc587a9700 1 -- 172.21.15.45:0/4164212931 wait complete. 2023-09-23T09:34:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.084+0000 7fcc587a9700 1 Processor -- start 2023-09-23T09:34:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.085+0000 7fcc587a9700 1 -- start start 2023-09-23T09:34:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.085+0000 7fcc587a9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010f4d0 0x7fcc501a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.085+0000 7fcc587a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010f940 0x7fcc501a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.085+0000 7fcc587a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5011a600 0x7fcc501aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.086+0000 7fcc587a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501aaea0 con 0x7fcc5010f940 2023-09-23T09:34:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.086+0000 7fcc587a9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501ab010 con 0x7fcc5010f4d0 2023-09-23T09:34:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.086+0000 7fcc587a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501ab1d0 con 0x7fcc5011a600 2023-09-23T09:34:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.088+0000 7fcc56d46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5011a600 0x7fcc501aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.088+0000 7fcc56d46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5011a600 0x7fcc501aa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36550/0 (socket says 172.21.15.45:36550) 2023-09-23T09:34:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.088+0000 7fcc56d46700 1 -- 172.21.15.45:0/2325507552 learned_addr learned my addr 172.21.15.45:0/2325507552 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.088+0000 7fcc56545700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010f4d0 0x7fcc501a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.089+0000 7fcc56d46700 1 -- 172.21.15.45:0/2325507552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010f4d0 msgr2=0x7fcc501a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.089+0000 7fcc56d46700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010f4d0 0x7fcc501a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.089+0000 7fcc56d46700 1 -- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010f940 msgr2=0x7fcc501a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.089+0000 7fcc56d46700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010f940 0x7fcc501a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.089+0000 7fcc56d46700 1 -- 172.21.15.45:0/2325507552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc48009d00 con 0x7fcc5011a600 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.089+0000 7fcc56d46700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5011a600 0x7fcc501aa860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b410 tx=0x7fcc4c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.090+0000 7fcc477fe700 1 -- 172.21.15.45:0/2325507552 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c011830 con 0x7fcc5011a600 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.090+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc501ab4b0 con 0x7fcc5011a600 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.090+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5010ffc0 con 0x7fcc5011a600 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.090+0000 7fcc477fe700 1 -- 172.21.15.45:0/2325507552 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4c00be60 con 0x7fcc5011a600 2023-09-23T09:34:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.090+0000 7fcc477fe700 1 -- 172.21.15.45:0/2325507552 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c010de0 con 0x7fcc5011a600 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.095+0000 7fcc64993b80 1 Processor -- start 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.096+0000 7fcc477fe700 1 -- 172.21.15.45:0/2325507552 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fcc4c011990 con 0x7fcc5011a600 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.096+0000 7fcc64993b80 1 -- start start 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.096+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4da400 0x55dceb4da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.096+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4dad20 0x55dceb4e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.096+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc64993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb443460 con 0x55dceb4e5bd0 2023-09-23T09:34:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc64993b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb434c20 con 0x55dceb4dad20 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc64993b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb432540 con 0x55dceb4da400 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4dad20 0x55dceb4e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4dad20 0x55dceb4e5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55798/0 (socket says 172.21.15.45:55798) 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 -- 172.21.15.45:0/2624103192 learned_addr learned my addr 172.21.15.45:0/2624103192 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc55d44700 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 -- 172.21.15.45:0/2624103192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4da400 msgr2=0x55dceb4da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4da400 0x55dceb4da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 -- 172.21.15.45:0/2624103192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4e5bd0 msgr2=0x55dceb4e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc56545700 1 -- 172.21.15.45:0/2624103192 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dceb4908d0 con 0x55dceb4dad20 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.097+0000 7fcc55d44700 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:34:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.098+0000 7fcc56545700 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4dad20 0x55dceb4e5690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009ad0 tx=0x7fcc40009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f1b21ff206408c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.098+0000 7fcc44ff9700 1 -- 172.21.15.45:0/2624103192 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc400095d0 con 0x55dceb4dad20 2023-09-23T09:34:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.098+0000 7fcc44ff9700 1 -- 172.21.15.45:0/2624103192 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4000ee50 con 0x55dceb4dad20 2023-09-23T09:34:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.098+0000 7fcc44ff9700 1 -- 172.21.15.45:0/2624103192 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc400123d0 con 0x55dceb4dad20 2023-09-23T09:34:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.098+0000 7fcc477fe700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a1790 0x7fcc3c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc477fe700 1 -- 172.21.15.45:0/2325507552 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc4c0cca30 con 0x7fcc5011a600 2023-09-23T09:34:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc55d44700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a1790 0x7fcc3c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 -- 172.21.15.45:0/2624103192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4dad20 msgr2=0x55dceb4e5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4dad20 0x55dceb4e5690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009ad0 tx=0x7fcc40009de0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 -- 172.21.15.45:0/2624103192 shutdown_connections 2023-09-23T09:34:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4da400 0x55dceb4da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 --2- 172.21.15.45:0/2624103192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4dad20 0x55dceb4e5690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 -- 172.21.15.45:0/2624103192 >> 172.21.15.45:0/2624103192 conn(0x55dceb44f260 msgr2=0x55dceb4d6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 -- 172.21.15.45:0/2624103192 shutdown_connections 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.099+0000 7fcc64993b80 1 -- 172.21.15.45:0/2624103192 wait complete. 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc55d44700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a1790 0x7fcc3c0a3c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009850 tx=0x7fcc480096a0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc64993b80 1 -- start start 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc64993b80 1 Processor -- start 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4da400 0x55dceb4e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4dad20 0x55dceb4e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc64993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb42de30 con 0x55dceb4da400 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.100+0000 7fcc64993b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb3ca710 con 0x55dceb4e5bd0 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc64993b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb3ca3b0 con 0x55dceb4dad20 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc64993b80 10 client.0 _ll_register_callbacks cb 0x55dceb227800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4da400 0x55dceb4e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4da400 0x55dceb4e5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34456/0 (socket says 172.21.15.45:34456) 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 -- 172.21.15.45:0/3285854657 learned_addr learned my addr 172.21.15.45:0/3285854657 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 -- 172.21.15.45:0/3285854657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4dad20 msgr2=0x55dceb4e2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4dad20 0x55dceb4e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 -- 172.21.15.45:0/3285854657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 msgr2=0x55dceb4e2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 -- 172.21.15.45:0/3285854657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc40009730 con 0x55dceb4da400 2023-09-23T09:34:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc56545700 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4da400 0x55dceb4e5570 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcc40000c00 tx=0x7fcc40013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300168ef191b56a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.101+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3285854657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4001c450 con 0x55dceb4da400 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3285854657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc4001c5b0 con 0x55dceb4da400 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dceb4ce250 con 0x55dceb4da400 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3285854657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc4001c770 con 0x55dceb4da400 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 10 client.16119 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dceb258de0 con 0x55dceb4da400 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3285854657 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc40031080 con 0x55dceb4da400 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3285854657 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcc40011070 con 0x55dceb4da400 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 20 client.16119 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb4f5630 0x55dceb4f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 4 client.16119 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:34:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dceb426e10 con 0x55dceb4f5630 2023-09-23T09:34:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.102+0000 7fcc55d44700 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb4f5630 0x55dceb4f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.104+0000 7fcc55d44700 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb4f5630 0x55dceb4f7a90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd25a3872f930f00 server_cookie=25a1a14c965d57 in_seq=0 out_seq=0 2023-09-23T09:34:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.104+0000 7fcc39ffb700 10 client.16119 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3285854657 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dceb426e10 con 0x55dceb4f5630 2023-09-23T09:34:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc39ffb700 10 client.16119 handle_command_reply: tid=0 2023-09-23T09:34:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 client.16119 shutdown 2023-09-23T09:34:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4da400 msgr2=0x55dceb4e5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4da400 0x55dceb4e5570 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcc40000c00 tx=0x7fcc40013e90 comp rx=0 tx=0).stop 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 shutdown_connections 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb4f5630 0x55dceb4f7a90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4da400 0x55dceb4e5570 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3285854657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4dad20 0x55dceb4e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc39ffb700 0 client.16119 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 >> 172.21.15.45:0/3285854657 conn(0x55dceb44f260 msgr2=0x55dceb4d6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 shutdown_connections 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.105+0000 7fcc64993b80 1 -- 172.21.15.45:0/3285854657 wait complete. 2023-09-23T09:34:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.106+0000 7fcc64993b80 20 client.16119 trim_cache size 0 max 16384 2023-09-23T09:34:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 Processor -- start 2023-09-23T09:34:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 -- start start 2023-09-23T09:34:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4da400 0x55dceb4ec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4dad20 0x55dceb4f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e5bd0 0x55dceb4ddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb2ae920 con 0x55dceb4dad20 2023-09-23T09:34:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb4db100 con 0x55dceb4da400 2023-09-23T09:34:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.134+0000 7fcc64993b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb4e0dc0 con 0x55dceb4e5bd0 2023-09-23T09:34:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e5bd0 0x55dceb4ddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e5bd0 0x55dceb4ddb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36570/0 (socket says 172.21.15.45:36570) 2023-09-23T09:34:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 -- 172.21.15.45:0/3171568887 learned_addr learned my addr 172.21.15.45:0/3171568887 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 -- 172.21.15.45:0/3171568887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4da400 msgr2=0x55dceb4ec780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 --2- 172.21.15.45:0/3171568887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4da400 0x55dceb4ec780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 -- 172.21.15.45:0/3171568887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4dad20 msgr2=0x55dceb4f0ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 --2- 172.21.15.45:0/3171568887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4dad20 0x55dceb4f0ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.135+0000 7fcc56d46700 1 -- 172.21.15.45:0/3171568887 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dceb241c60 con 0x55dceb4e5bd0 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.136+0000 7fcc56d46700 1 --2- 172.21.15.45:0/3171568887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e5bd0 0x55dceb4ddb90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00bfd0 tx=0x7fcc4c0993a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f49ff4b6909dd42 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.136+0000 7fcc457fa700 1 -- 172.21.15.45:0/3171568887 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c09db90 con 0x55dceb4e5bd0 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.136+0000 7fcc457fa700 1 -- 172.21.15.45:0/3171568887 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4c0a1dd0 con 0x55dceb4e5bd0 2023-09-23T09:34:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.136+0000 7fcc457fa700 1 -- 172.21.15.45:0/3171568887 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c098f30 con 0x55dceb4e5bd0 2023-09-23T09:34:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.137+0000 7fcc64993b80 1 -- 172.21.15.45:0/3171568887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e5bd0 msgr2=0x55dceb4ddb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.137+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3171568887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e5bd0 0x55dceb4ddb90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00bfd0 tx=0x7fcc4c0993a0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.137+0000 7fcc64993b80 1 -- 172.21.15.45:0/3171568887 shutdown_connections 2023-09-23T09:34:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.137+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3171568887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e5bd0 0x55dceb4ddb90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.137+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3171568887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4da400 0x55dceb4ec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.137+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3171568887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4dad20 0x55dceb4f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.137+0000 7fcc64993b80 1 -- 172.21.15.45:0/3171568887 >> 172.21.15.45:0/3171568887 conn(0x55dceb44f260 msgr2=0x55dceb4f1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.140+0000 7fcc64993b80 1 -- 172.21.15.45:0/3171568887 shutdown_connections 2023-09-23T09:34:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.140+0000 7fcc64993b80 1 -- 172.21.15.45:0/3171568887 wait complete. 2023-09-23T09:34:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.140+0000 7fcc64993b80 1 -- start start 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.140+0000 7fcc64993b80 1 Processor -- start 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.140+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4f5630 0x55dceb4e9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc64993b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e9880 0x55dceb5494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc64993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb549a20 con 0x55dceb4f5630 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc64993b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb549b60 con 0x55dceb4e5bd0 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc64993b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dceb549cf0 con 0x55dceb4e9880 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc64993b80 10 client.0 _ll_register_callbacks cb 0x7fcc40009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55818/0 (socket says 172.21.15.45:55818) 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 -- 172.21.15.45:0/3608922194 learned_addr learned my addr 172.21.15.45:0/3608922194 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56d46700 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e9880 0x55dceb5494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 -- 172.21.15.45:0/3608922194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e9880 msgr2=0x55dceb5494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e9880 0x55dceb5494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 -- 172.21.15.45:0/3608922194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4f5630 msgr2=0x55dceb4e9340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4f5630 0x55dceb4e9340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.141+0000 7fcc56545700 1 -- 172.21.15.45:0/3608922194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5010ffc0 con 0x55dceb4e5bd0 2023-09-23T09:34:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.142+0000 7fcc56545700 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e8e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009e20 tx=0x7fcc40009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74993c8f9e4aafd3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.142+0000 7fcc39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:34:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3608922194 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40025640 con 0x55dceb4e5bd0 2023-09-23T09:34:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3608922194 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc400257a0 con 0x55dceb4e5bd0 2023-09-23T09:34:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc4000ee50 con 0x55dceb4e5bd0 2023-09-23T09:34:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3608922194 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc40004040 con 0x55dceb4e5bd0 2023-09-23T09:34:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 10 client.25808 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dceb27f250 con 0x55dceb4e5bd0 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3608922194 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4000ee50 con 0x55dceb4e5bd0 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3608922194 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcc40011070 con 0x55dceb4e5bd0 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 10 client.25808 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 20 client.25808 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb54b0b0 0x55dceb54d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 4 client.25808 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.143+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dceb426b30 con 0x55dceb54b0b0 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.144+0000 7fcc55d44700 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb54b0b0 0x55dceb54d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.144+0000 7fcc55d44700 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb54b0b0 0x55dceb54d4d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f531930d3eb2603 server_cookie=830d731a43cbe57c in_seq=0 out_seq=0 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.144+0000 7fcc39ffb700 10 client.25808 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.144+0000 7fcc39ffb700 1 -- 172.21.15.45:0/3608922194 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dceb426b30 con 0x55dceb54b0b0 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.144+0000 7fcc39ffb700 10 client.25808 handle_command_reply: tid=0 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.144+0000 7fcc64993b80 1 client.25808 shutdown 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.145+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 msgr2=0x55dceb4e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.145+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e8e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009e20 tx=0x7fcc40009320 comp rx=0 tx=0).stop 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.146+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 shutdown_connections 2023-09-23T09:34:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.146+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dceb54b0b0 0x55dceb54d4d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.146+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dceb4e9880 0x55dceb5494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.146+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dceb4e5bd0 0x55dceb4e8e00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.146+0000 7fcc64993b80 1 --2- 172.21.15.45:0/3608922194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dceb4f5630 0x55dceb4e9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.146+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 >> 172.21.15.45:0/3608922194 conn(0x55dceb44f260 msgr2=0x55dceb4f0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.148+0000 7fcc39ffb700 0 client.25808 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.148+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 shutdown_connections 2023-09-23T09:34:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.149+0000 7fcc64993b80 1 -- 172.21.15.45:0/3608922194 wait complete. 2023-09-23T09:34:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.149+0000 7fcc64993b80 20 client.25808 trim_cache size 0 max 16384 2023-09-23T09:34:30.150 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:34:30.151 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:34:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a1790 msgr2=0x7fcc3c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a1790 0x7fcc3c0a3c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009850 tx=0x7fcc480096a0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5011a600 msgr2=0x7fcc501aa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5011a600 0x7fcc501aa860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b410 tx=0x7fcc4c009f90 comp rx=0 tx=0).stop 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 shutdown_connections 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc5011a600 0x7fcc501aa860 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc5010f4d0 0x7fcc501a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcc3c0a1790 0x7fcc3c0a3c50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 --2- 172.21.15.45:0/2325507552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5010f940 0x7fcc501a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.150+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 >> 172.21.15.45:0/2325507552 conn(0x7fcc5006d0e0 msgr2=0x7fcc501185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.151+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 shutdown_connections 2023-09-23T09:34:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:34:30.151+0000 7fcc587a9700 1 -- 172.21.15.45:0/2325507552 wait complete. 2023-09-23T09:34:30.172 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:34:31.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:31 smithi098 ceph-mon[102386]: pgmap v156: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 695 op/s 2023-09-23T09:34:31.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:31 smithi045 ceph-mon[98627]: pgmap v156: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 695 op/s 2023-09-23T09:34:31.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:31 smithi123 ceph-mon[103577]: pgmap v156: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 695 op/s 2023-09-23T09:34:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 285.0 s, 2385.1 tps, lat 41.727 ms stddev 47.276 2023-09-23T09:34:33.801 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:33 smithi098 ceph-mon[102386]: pgmap v157: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 695 op/s 2023-09-23T09:34:33.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:33 smithi045 ceph-mon[98627]: pgmap v157: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 695 op/s 2023-09-23T09:34:33.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:33 smithi123 ceph-mon[103577]: pgmap v157: 97 pgs: 97 active+clean; 6.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 695 op/s 2023-09-23T09:34:35.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:35 smithi098 ceph-mon[102386]: pgmap v158: 97 pgs: 97 active+clean; 6.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2023-09-23T09:34:35.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:35 smithi045 ceph-mon[98627]: pgmap v158: 97 pgs: 97 active+clean; 6.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2023-09-23T09:34:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:35 smithi123 ceph-mon[103577]: pgmap v158: 97 pgs: 97 active+clean; 6.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2023-09-23T09:34:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 290.0 s, 1515.0 tps, lat 65.740 ms stddev 114.467 2023-09-23T09:34:37.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:37 smithi098 ceph-mon[102386]: pgmap v159: 97 pgs: 97 active+clean; 6.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 730 op/s 2023-09-23T09:34:37.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:37 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:34:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:37 smithi045 ceph-mon[98627]: pgmap v159: 97 pgs: 97 active+clean; 6.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 730 op/s 2023-09-23T09:34:37.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:37 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:34:37.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:37 smithi123 ceph-mon[103577]: pgmap v159: 97 pgs: 97 active+clean; 6.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 730 op/s 2023-09-23T09:34:37.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:37 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:34:38.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:38 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:34:38.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:38 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:34:38.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:38 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:34:38.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:38 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:34:38.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:38 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:34:38.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:38 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:34:38.842 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:38 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:34:38.842 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:38 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:34:38.842 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:38 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:34:39.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:39 smithi098 ceph-mon[102386]: pgmap v160: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 MiB/s wr, 2.43k op/s 2023-09-23T09:34:39.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:39 smithi045 ceph-mon[98627]: pgmap v160: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 MiB/s wr, 2.43k op/s 2023-09-23T09:34:39.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:39 smithi123 ceph-mon[103577]: pgmap v160: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 MiB/s wr, 2.43k op/s 2023-09-23T09:34:41.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:41 smithi098 ceph-mon[102386]: pgmap v161: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-09-23T09:34:41.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:41 smithi045 ceph-mon[98627]: pgmap v161: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-09-23T09:34:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:41 smithi123 ceph-mon[103577]: pgmap v161: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-09-23T09:34:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 295.0 s, 2235.2 tps, lat 45.003 ms stddev 49.583 2023-09-23T09:34:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.971+0000 7f8284d2e700 1 -- 172.21.15.45:0/3369432348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800ffd50 msgr2=0x7f82801040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.971+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3369432348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800ffd50 0x7f82801040e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f82740099c0 tx=0x7f8274009cd0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.972+0000 7f8284d2e700 1 -- 172.21.15.45:0/3369432348 shutdown_connections 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.972+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3369432348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82801001d0 0x7f8280104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.972+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3369432348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82800688c0 0x7f82800ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.972+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3369432348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800ffd50 0x7f82801040e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.972+0000 7f8284d2e700 1 -- 172.21.15.45:0/3369432348 >> 172.21.15.45:0/3369432348 conn(0x7f8280074f80 msgr2=0x7f8280075390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.972+0000 7f8284d2e700 1 -- 172.21.15.45:0/3369432348 shutdown_connections 2023-09-23T09:34:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.972+0000 7f8284d2e700 1 -- 172.21.15.45:0/3369432348 wait complete. 2023-09-23T09:34:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 Processor -- start 2023-09-23T09:34:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 -- start start 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800688c0 0x7f82801990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82800ffd50 0x7f8280199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82801001d0 0x7f828019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828019e2c0 con 0x7f82800688c0 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828019e430 con 0x7f82800ffd50 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f8284d2e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828019e5f0 con 0x7f82801001d0 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f827e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800688c0 0x7f82801990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f827e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800688c0 0x7f82801990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34480/0 (socket says 172.21.15.45:34480) 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f827e59c700 1 -- 172.21.15.45:0/3464425378 learned_addr learned my addr 172.21.15.45:0/3464425378 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f827ed9d700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82801001d0 0x7f828019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.973+0000 7f827dd9b700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82800ffd50 0x7f8280199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f827ed9d700 1 -- 172.21.15.45:0/3464425378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82800ffd50 msgr2=0x7f8280199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f827ed9d700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82800ffd50 0x7f8280199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f827ed9d700 1 -- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800688c0 msgr2=0x7f82801990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f827ed9d700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800688c0 0x7f82801990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f827ed9d700 1 -- 172.21.15.45:0/3464425378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740095f0 con 0x7f82801001d0 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f827ed9d700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82801001d0 0x7f828019dc80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f827800ef90 tx=0x7f827800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f826f7fe700 1 -- 172.21.15.45:0/3464425378 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278018070 con 0x7f82801001d0 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f826f7fe700 1 -- 172.21.15.45:0/3464425378 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827800f040 con 0x7f82801001d0 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828019e870 con 0x7f82801001d0 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f826f7fe700 1 -- 172.21.15.45:0/3464425378 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800cbb0 con 0x7f82801001d0 2023-09-23T09:34:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.974+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82801a6110 con 0x7f82801001d0 2023-09-23T09:34:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.979+0000 7f8290f18b80 1 Processor -- start 2023-09-23T09:34:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.980+0000 7f826f7fe700 1 -- 172.21.15.45:0/3464425378 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f8278003a00 con 0x7f82801001d0 2023-09-23T09:34:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.980+0000 7f8290f18b80 1 -- start start 2023-09-23T09:34:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.981+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.981+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.981+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab76b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.981+0000 7f8290f18b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620baad2400 con 0x5620bab68f40 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.981+0000 7f8290f18b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620baac3bc0 con 0x5620bab74710 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.981+0000 7f8290f18b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620baac14e0 con 0x5620bab69860 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f826f7fe700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82680a17d0 0x7f82680a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab69320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34488/0 (socket says 172.21.15.45:34488) 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827dd9b700 1 -- 172.21.15.45:0/3493701420 learned_addr learned my addr 172.21.15.45:0/3493701420 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827e59c700 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f826f7fe700 1 -- 172.21.15.45:0/3464425378 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f82780ccdb0 con 0x7f82801001d0 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827ed9d700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82680a17d0 0x7f82680a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827e59c700 1 -- 172.21.15.45:0/3493701420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 msgr2=0x5620bab76b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827e59c700 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab76b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827e59c700 1 -- 172.21.15.45:0/3493701420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 msgr2=0x5620bab69320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827e59c700 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab69320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827e59c700 1 -- 172.21.15.45:0/3493701420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620bab1f870 con 0x5620bab69860 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.982+0000 7f827dd9b700 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab76b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.983+0000 7f827e59c700 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab741d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8270005950 tx=0x7f8270011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd47d1df174e0ac server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.983+0000 7f827ed9d700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82680a17d0 0x7f82680a3c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f827800c6f0 tx=0x7f8278004c50 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.983+0000 7f826cff9700 1 -- 172.21.15.45:0/3493701420 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270011d80 con 0x5620bab69860 2023-09-23T09:34:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.983+0000 7f826cff9700 1 -- 172.21.15.45:0/3493701420 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82700044e0 con 0x5620bab69860 2023-09-23T09:34:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.983+0000 7f826cff9700 1 -- 172.21.15.45:0/3493701420 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270005280 con 0x5620bab69860 2023-09-23T09:34:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 -- 172.21.15.45:0/3493701420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 msgr2=0x5620bab741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab741d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8270005950 tx=0x7f8270011440 comp rx=0 tx=0).stop 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 -- 172.21.15.45:0/3493701420 shutdown_connections 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab76b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab69320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3493701420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab741d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 -- 172.21.15.45:0/3493701420 >> 172.21.15.45:0/3493701420 conn(0x5620baade200 msgr2=0x5620bab653c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 -- 172.21.15.45:0/3493701420 shutdown_connections 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 -- 172.21.15.45:0/3493701420 wait complete. 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 -- start start 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.984+0000 7f8290f18b80 1 Processor -- start 2023-09-23T09:34:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab68f40 0x5620bab74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f8290f18b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620baabcdd0 con 0x5620bab74710 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f8290f18b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620baa596b0 con 0x5620bab68f40 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f8290f18b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620baa59350 con 0x5620bab69860 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f8290f18b80 10 client.0 _ll_register_callbacks cb 0x5620ba8b67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab68f40 0x5620bab74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36602/0 (socket says 172.21.15.45:36602) 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 -- 172.21.15.45:0/1815759999 learned_addr learned my addr 172.21.15.45:0/1815759999 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827dd9b700 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 -- 172.21.15.45:0/1815759999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab68f40 msgr2=0x5620bab74150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab68f40 0x5620bab74150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 -- 172.21.15.45:0/1815759999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 msgr2=0x5620bab71b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab71b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.985+0000 7f827e59c700 1 -- 172.21.15.45:0/1815759999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8280106360 con 0x5620bab69860 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f827e59c700 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab713d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8270011f90 tx=0x7f8270004ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2cc97f000e713ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8265ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8265ffb700 1 -- 172.21.15.45:0/1815759999 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270005ce0 con 0x5620bab69860 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8265ffb700 1 -- 172.21.15.45:0/1815759999 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f827001f040 con 0x5620bab69860 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620bab5d090 con 0x5620bab69860 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8265ffb700 1 -- 172.21.15.45:0/1815759999 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82700053c0 con 0x5620bab69860 2023-09-23T09:34:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 10 client.25816 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f8270009970 con 0x5620bab69860 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8265ffb700 1 -- 172.21.15.45:0/1815759999 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620bab5d090 con 0x5620bab69860 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8265ffb700 1 -- 172.21.15.45:0/1815759999 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8270019430 con 0x5620bab69860 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 10 client.25816 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 20 client.25816 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620bab841c0 0x5620bab865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 4 client.25816 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.986+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620baab5db0 con 0x5620bab841c0 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.987+0000 7f827dd9b700 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620bab841c0 0x5620bab865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.987+0000 7f8265ffb700 10 client.25816 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.987+0000 7f827dd9b700 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620bab841c0 0x5620bab865e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933f59377af99320 server_cookie=5cdb0a886db64acd in_seq=0 out_seq=0 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8265ffb700 1 -- 172.21.15.45:0/1815759999 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5620baab5db0 con 0x5620bab841c0 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8265ffb700 10 client.25816 handle_command_reply: tid=0 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 client.25816 shutdown 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 msgr2=0x5620bab713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab713d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8270011f90 tx=0x7f8270004ac0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 shutdown_connections 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620bab841c0 0x5620bab865e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab71b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8265ffb700 0 client.25816 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab68f40 0x5620bab74150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 --2- 172.21.15.45:0/1815759999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab713d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 >> 172.21.15.45:0/1815759999 conn(0x5620baade200 msgr2=0x5620bab64fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 shutdown_connections 2023-09-23T09:34:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 1 -- 172.21.15.45:0/1815759999 wait complete. 2023-09-23T09:34:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:41.988+0000 7f8290f18b80 20 client.25816 trim_cache size 0 max 16384 2023-09-23T09:34:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.010+0000 7f8290f18b80 1 Processor -- start 2023-09-23T09:34:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f8290f18b80 1 -- start start 2023-09-23T09:34:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab6cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f8290f18b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ba93d8c0 con 0x5620bab68f40 2023-09-23T09:34:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f8290f18b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bab61ba0 con 0x5620bab74710 2023-09-23T09:34:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f8290f18b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bab5cf60 con 0x5620bab69860 2023-09-23T09:34:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f827e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f827e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab6c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36608/0 (socket says 172.21.15.45:36608) 2023-09-23T09:34:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.011+0000 7f827e59c700 1 -- 172.21.15.45:0/3306300793 learned_addr learned my addr 172.21.15.45:0/3306300793 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.012+0000 7f827e59c700 1 -- 172.21.15.45:0/3306300793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 msgr2=0x5620bab6cc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:34:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.012+0000 7f827e59c700 1 --2- 172.21.15.45:0/3306300793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab6cc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.012+0000 7f827e59c700 1 -- 172.21.15.45:0/3306300793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 msgr2=0x5620bab865b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.012+0000 7f827e59c700 1 --2- 172.21.15.45:0/3306300793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.012+0000 7f827e59c700 1 -- 172.21.15.45:0/3306300793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ba8e7d80 con 0x5620bab69860 2023-09-23T09:34:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.013+0000 7f827e59c700 1 --2- 172.21.15.45:0/3306300793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab6c6d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8270009c60 tx=0x7f8270011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78bd6968d3b2f772 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.013+0000 7f826d7fa700 1 -- 172.21.15.45:0/3306300793 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270009260 con 0x5620bab69860 2023-09-23T09:34:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.013+0000 7f826d7fa700 1 -- 172.21.15.45:0/3306300793 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8270011790 con 0x5620bab69860 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.013+0000 7f826d7fa700 1 -- 172.21.15.45:0/3306300793 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270024690 con 0x5620bab69860 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.014+0000 7f8290f18b80 1 -- 172.21.15.45:0/3306300793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 msgr2=0x5620bab6c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.014+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3306300793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab6c6d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8270009c60 tx=0x7f8270011440 comp rx=0 tx=0).stop 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.014+0000 7f8290f18b80 1 -- 172.21.15.45:0/3306300793 shutdown_connections 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.014+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3306300793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab74710 0x5620bab6cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.014+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3306300793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab68f40 0x5620bab865b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.014+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3306300793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab69860 0x5620bab6c6d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.014+0000 7f8290f18b80 1 -- 172.21.15.45:0/3306300793 >> 172.21.15.45:0/3306300793 conn(0x5620baade200 msgr2=0x5620bab7f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.015+0000 7f8290f18b80 1 -- 172.21.15.45:0/3306300793 shutdown_connections 2023-09-23T09:34:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.015+0000 7f8290f18b80 1 -- 172.21.15.45:0/3306300793 wait complete. 2023-09-23T09:34:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.015+0000 7f8290f18b80 1 -- start start 2023-09-23T09:34:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.015+0000 7f8290f18b80 1 Processor -- start 2023-09-23T09:34:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab68f40 0x5620bab79240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab69860 0x5620bab79780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f8290f18b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab79ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f8290f18b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620babdf9a0 con 0x5620bab74710 2023-09-23T09:34:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f8290f18b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620babdfb10 con 0x5620bab69860 2023-09-23T09:34:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f8290f18b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620babdfcd0 con 0x5620bab68f40 2023-09-23T09:34:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f8290f18b80 10 client.0 _ll_register_callbacks cb 0x7f8274009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f827dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab68f40 0x5620bab79240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f827dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab68f40 0x5620bab79240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36612/0 (socket says 172.21.15.45:36612) 2023-09-23T09:34:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f827dd9b700 1 -- 172.21.15.45:0/3509802236 learned_addr learned my addr 172.21.15.45:0/3509802236 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f827dd9b700 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab79ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.016+0000 7f827e59c700 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab69860 0x5620bab79780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f827dd9b700 1 -- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab69860 msgr2=0x5620bab79780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f827dd9b700 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab69860 0x5620bab79780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f827dd9b700 1 -- 172.21.15.45:0/3509802236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 msgr2=0x5620bab79ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f827dd9b700 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab79ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f827dd9b700 1 -- 172.21.15.45:0/3509802236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8270009970 con 0x5620bab68f40 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f827dd9b700 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab68f40 0x5620bab79240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8274005850 tx=0x7f8274005150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93fd80027fcaab91 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f8265ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f8265ffb700 1 -- 172.21.15.45:0/3509802236 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827401d040 con 0x5620bab68f40 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f8265ffb700 1 -- 172.21.15.45:0/3509802236 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8274004d70 con 0x5620bab68f40 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82700044e0 con 0x5620bab68f40 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f8265ffb700 1 -- 172.21.15.45:0/3509802236 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8274004f30 con 0x5620bab68f40 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f8290f18b80 10 client.25822 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.017+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5620ba8d0c00 con 0x5620bab68f40 2023-09-23T09:34:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8265ffb700 1 -- 172.21.15.45:0/3509802236 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82700044e0 con 0x5620bab68f40 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8265ffb700 1 -- 172.21.15.45:0/3509802236 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f827400f7f0 con 0x5620bab68f40 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8290f18b80 10 client.25822 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8290f18b80 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8290f18b80 20 client.25822 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620babe14a0 0x5620babe38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8290f18b80 4 client.25822 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620baab5ad0 con 0x5620babe14a0 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f827e59c700 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620babe14a0 0x5620babe38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f827e59c700 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620babe14a0 0x5620babe38c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c42f47fce83822 server_cookie=5fec03d9003290b7 in_seq=0 out_seq=0 2023-09-23T09:34:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.018+0000 7f8265ffb700 10 client.25822 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8265ffb700 1 -- 172.21.15.45:0/3509802236 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620baab5ad0 con 0x5620babe14a0 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8265ffb700 10 client.25822 handle_command_reply: tid=0 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 client.25822 shutdown 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab68f40 msgr2=0x5620bab79240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab68f40 0x5620bab79240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8274005850 tx=0x7f8274005150 comp rx=0 tx=0).stop 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 shutdown_connections 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5620babe14a0 0x5620babe38c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5620bab74710 0x5620bab79ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620bab68f40 0x5620bab79240 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 --2- 172.21.15.45:0/3509802236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bab69860 0x5620bab79780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 >> 172.21.15.45:0/3509802236 conn(0x5620baade200 msgr2=0x5620bab6c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.036+0000 7f8265ffb700 0 client.25822 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 shutdown_connections 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8290f18b80 1 -- 172.21.15.45:0/3509802236 wait complete. 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8290f18b80 20 client.25822 trim_cache size 0 max 16384 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:34:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:34:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82680a17d0 msgr2=0x7f82680a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82680a17d0 0x7f82680a3c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f827800c6f0 tx=0x7f8278004c50 comp rx=0 tx=0).stop 2023-09-23T09:34:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82801001d0 msgr2=0x7f828019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82801001d0 0x7f828019dc80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f827800ef90 tx=0x7f827800c3a0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 shutdown_connections 2023-09-23T09:34:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82801001d0 0x7f828019dc80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82680a17d0 0x7f82680a3c90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82800688c0 0x7f82801990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 --2- 172.21.15.45:0/3464425378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82800ffd50 0x7f8280199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 >> 172.21.15.45:0/3464425378 conn(0x7f8280074f80 msgr2=0x7f82800fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 shutdown_connections 2023-09-23T09:34:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:42.037+0000 7f8284d2e700 1 -- 172.21.15.45:0/3464425378 wait complete. 2023-09-23T09:34:43.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:43 smithi098 ceph-mon[102386]: pgmap v162: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-09-23T09:34:43.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:43 smithi045 ceph-mon[98627]: pgmap v162: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-09-23T09:34:43.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:43 smithi123 ceph-mon[103577]: pgmap v162: 97 pgs: 97 active+clean; 6.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-09-23T09:34:46.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:45 smithi098 ceph-mon[102386]: pgmap v163: 97 pgs: 97 active+clean; 7.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 44 MiB/s wr, 2.38k op/s 2023-09-23T09:34:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:45 smithi045 ceph-mon[98627]: pgmap v163: 97 pgs: 97 active+clean; 7.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 44 MiB/s wr, 2.38k op/s 2023-09-23T09:34:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:45 smithi123 ceph-mon[103577]: pgmap v163: 97 pgs: 97 active+clean; 7.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 44 MiB/s wr, 2.38k op/s 2023-09-23T09:34:46.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 300.0 s, 1598.8 tps, lat 54.688 ms stddev 68.992 2023-09-23T09:34:48.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:47 smithi098 ceph-mon[102386]: pgmap v164: 97 pgs: 97 active+clean; 7.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 2.00k op/s 2023-09-23T09:34:48.088 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:47 smithi045 ceph-mon[98627]: pgmap v164: 97 pgs: 97 active+clean; 7.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 2.00k op/s 2023-09-23T09:34:48.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:47 smithi123 ceph-mon[103577]: pgmap v164: 97 pgs: 97 active+clean; 7.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 2.00k op/s 2023-09-23T09:34:50.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:49 smithi098 ceph-mon[102386]: pgmap v165: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 MiB/s wr, 3.09k op/s 2023-09-23T09:34:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:49 smithi045 ceph-mon[98627]: pgmap v165: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 MiB/s wr, 3.09k op/s 2023-09-23T09:34:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:49 smithi123 ceph-mon[103577]: pgmap v165: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 MiB/s wr, 3.09k op/s 2023-09-23T09:34:51.887 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 305.0 s, 1623.9 tps, lat 69.776 ms stddev 201.096 2023-09-23T09:34:52.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:51 smithi098 ceph-mon[102386]: pgmap v166: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.39k op/s 2023-09-23T09:34:52.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:51 smithi045 ceph-mon[98627]: pgmap v166: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.39k op/s 2023-09-23T09:34:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:51 smithi123 ceph-mon[103577]: pgmap v166: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.39k op/s 2023-09-23T09:34:54.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:53 smithi098 ceph-mon[102386]: pgmap v167: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.39k op/s 2023-09-23T09:34:54.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:53 smithi045 ceph-mon[98627]: pgmap v167: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.39k op/s 2023-09-23T09:34:54.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:53 smithi123 ceph-mon[103577]: pgmap v167: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.39k op/s 2023-09-23T09:34:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 -- 172.21.15.45:0/2429830738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b0105050 msgr2=0x7f63b010cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 --2- 172.21.15.45:0/2429830738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b0105050 0x7f63b010cbd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009fc0 tx=0x7f63a800b330 comp rx=0 tx=0).stop 2023-09-23T09:34:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 -- 172.21.15.45:0/2429830738 shutdown_connections 2023-09-23T09:34:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 --2- 172.21.15.45:0/2429830738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b0105050 0x7f63b010cbd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 --2- 172.21.15.45:0/2429830738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0100160 0x7f63b0100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 --2- 172.21.15.45:0/2429830738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b0100b10 0x7f63b0104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 -- 172.21.15.45:0/2429830738 >> 172.21.15.45:0/2429830738 conn(0x7f63b0076840 msgr2=0x7f63b0076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 -- 172.21.15.45:0/2429830738 shutdown_connections 2023-09-23T09:34:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.197+0000 7f63b484d700 1 -- 172.21.15.45:0/2429830738 wait complete. 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 Processor -- start 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 -- start start 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b0100160 0x7f63b0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0105050 0x7f63b0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01a0ea0 0x7f63b01a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01a3860 con 0x7f63b0100160 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01a39d0 con 0x7f63b0105050 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63b484d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01a3b90 con 0x7f63b01a0ea0 2023-09-23T09:34:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63add9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0105050 0x7f63b0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b0100160 0x7f63b0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01a0ea0 0x7f63b01a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63add9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0105050 0x7f63b0198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55870/0 (socket says 172.21.15.45:55870) 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b0100160 0x7f63b0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34520/0 (socket says 172.21.15.45:34520) 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.198+0000 7f63add9b700 1 -- 172.21.15.45:0/3004478660 learned_addr learned my addr 172.21.15.45:0/3004478660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63aed9d700 1 -- 172.21.15.45:0/3004478660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0105050 msgr2=0x7f63b0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63aed9d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0105050 0x7f63b0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63aed9d700 1 -- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b0100160 msgr2=0x7f63b0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63aed9d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b0100160 0x7f63b0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63aed9d700 1 -- 172.21.15.45:0/3004478660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a8009c40 con 0x7f63b01a0ea0 2023-09-23T09:34:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63aed9d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01a0ea0 0x7f63b01a3290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63a8015000 tx=0x7f63a8009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f639f7fe700 1 -- 172.21.15.45:0/3004478660 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800e040 con 0x7f63b01a0ea0 2023-09-23T09:34:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b01a3e10 con 0x7f63b01a0ea0 2023-09-23T09:34:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f639f7fe700 1 -- 172.21.15.45:0/3004478660 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a801d470 con 0x7f63b01a0ea0 2023-09-23T09:34:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f639f7fe700 1 -- 172.21.15.45:0/3004478660 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a80052a0 con 0x7f63b01a0ea0 2023-09-23T09:34:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.199+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b01a42e0 con 0x7f63b01a0ea0 2023-09-23T09:34:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.202+0000 7f639f7fe700 1 -- 172.21.15.45:0/3004478660 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f63a800b9f0 con 0x7f63b01a0ea0 2023-09-23T09:34:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.203+0000 7f639f7fe700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f63980a17c0 0x7f63980a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.203+0000 7f639f7fe700 1 -- 172.21.15.45:0/3004478660 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f63a80ce5d0 con 0x7f63b01a0ea0 2023-09-23T09:34:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.203+0000 7f63ae59c700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f63980a17c0 0x7f63980a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.203+0000 7f63ae59c700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f63980a17c0 0x7f63980a3c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005950 tx=0x7f63a000b410 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.204+0000 7f63c0a37b80 1 Processor -- start 2023-09-23T09:34:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.204+0000 7f63c0a37b80 1 -- start start 2023-09-23T09:34:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f03f40 0x558d29f04320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f0f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f0f710 0x558d29f11b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29e6d400 con 0x558d29f0f710 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29e5ebc0 con 0x558d29f03f40 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29e5c4e0 con 0x558d29f04860 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f0f710 0x558d29f11b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63add9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f0f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f03f40 0x558d29f04320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f0f710 0x558d29f11b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34532/0 (socket says 172.21.15.45:34532) 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 -- 172.21.15.45:0/3829465485 learned_addr learned my addr 172.21.15.45:0/3829465485 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63add9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f0f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36636/0 (socket says 172.21.15.45:36636) 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 -- 172.21.15.45:0/3829465485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 msgr2=0x558d29f0f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f0f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 -- 172.21.15.45:0/3829465485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f03f40 msgr2=0x558d29f04320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f03f40 0x558d29f04320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63aed9d700 1 -- 172.21.15.45:0/3829465485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d29eba870 con 0x558d29f0f710 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63add9b700 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f0f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.205+0000 7f63ae59c700 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f03f40 0x558d29f04320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.207+0000 7f63aed9d700 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f0f710 0x558d29f11b00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f63a8097ac0 tx=0x7f63a809b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414866a0eb731574 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.207+0000 7f639cff9700 1 -- 172.21.15.45:0/3829465485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a809bcc0 con 0x558d29f0f710 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.207+0000 7f639cff9700 1 -- 172.21.15.45:0/3829465485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a809b5c0 con 0x558d29f0f710 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.207+0000 7f639cff9700 1 -- 172.21.15.45:0/3829465485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a801d8c0 con 0x558d29f0f710 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 -- 172.21.15.45:0/3829465485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f0f710 msgr2=0x558d29f11b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f0f710 0x558d29f11b00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f63a8097ac0 tx=0x7f63a809b410 comp rx=0 tx=0).stop 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 -- 172.21.15.45:0/3829465485 shutdown_connections 2023-09-23T09:34:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f0f710 0x558d29f11b00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f03f40 0x558d29f04320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/3829465485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f0f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 -- 172.21.15.45:0/3829465485 >> 172.21.15.45:0/3829465485 conn(0x558d29e79200 msgr2=0x558d29f003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 -- 172.21.15.45:0/3829465485 shutdown_connections 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.208+0000 7f63c0a37b80 1 -- 172.21.15.45:0/3829465485 wait complete. 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 -- start start 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 Processor -- start 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f03f40 0x558d29f0f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f04860 0x558d29f0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f0cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29e57dd0 con 0x558d29f04860 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29df46b0 con 0x558d29f0f710 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29df4350 con 0x558d29f03f40 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63c0a37b80 10 client.0 _ll_register_callbacks cb 0x558d29c517a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f04860 0x558d29f0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f0cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f04860 0x558d29f0c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34534/0 (socket says 172.21.15.45:34534) 2023-09-23T09:34:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f03f40 0x558d29f0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.209+0000 7f63add9b700 1 -- 172.21.15.45:0/631584293 learned_addr learned my addr 172.21.15.45:0/631584293 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63aed9d700 1 -- 172.21.15.45:0/631584293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f03f40 msgr2=0x558d29f0f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63aed9d700 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f03f40 0x558d29f0f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63aed9d700 1 -- 172.21.15.45:0/631584293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f04860 msgr2=0x558d29f0c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63aed9d700 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f04860 0x558d29f0c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63aed9d700 1 -- 172.21.15.45:0/631584293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a4009910 con 0x558d29f0f710 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63aed9d700 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f0cb20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x558d29f05f70 tx=0x7f63a8099b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a410e55c924357b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f6395ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:34:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f6395ffb700 1 -- 172.21.15.45:0/631584293 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a809bcc0 con 0x558d29f0f710 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d29ef8090 con 0x558d29f0f710 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f6395ffb700 1 -- 172.21.15.45:0/631584293 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f63a80a38c0 con 0x558d29f0f710 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f6395ffb700 1 -- 172.21.15.45:0/631584293 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a809f710 con 0x558d29f0f710 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63c0a37b80 10 client.25835 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.210+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f63b01a42e0 con 0x558d29f0f710 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f6395ffb700 1 -- 172.21.15.45:0/631584293 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d29ef8090 con 0x558d29f0f710 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f6395ffb700 1 -- 172.21.15.45:0/631584293 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f63a809b920 con 0x558d29f0f710 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63c0a37b80 10 client.25835 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63c0a37b80 10 client.25835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63c0a37b80 20 client.25835 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f1ebe0 0x558d29f21000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63c0a37b80 4 client.25835 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d29e50db0 con 0x558d29f1ebe0 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63ae59c700 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f1ebe0 0x558d29f21000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f63ae59c700 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f1ebe0 0x558d29f21000 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3969fe4555c1cec8 server_cookie=48a1261e7517f16a in_seq=0 out_seq=0 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.211+0000 7f6395ffb700 10 client.25835 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.212+0000 7f6395ffb700 1 -- 172.21.15.45:0/631584293 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558d29e50db0 con 0x558d29f1ebe0 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.212+0000 7f6395ffb700 10 client.25835 handle_command_reply: tid=0 2023-09-23T09:34:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.212+0000 7f63c0a37b80 1 client.25835 shutdown 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 msgr2=0x558d29f0cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f0cb20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x558d29f05f70 tx=0x7f63a8099b70 comp rx=0 tx=0).stop 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 shutdown_connections 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f1ebe0 0x558d29f21000 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f0cb20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f03f40 0x558d29f0f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/631584293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f04860 0x558d29f0c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 >> 172.21.15.45:0/631584293 conn(0x558d29e79200 msgr2=0x558d29efff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f6395ffb700 0 client.25835 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 shutdown_connections 2023-09-23T09:34:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 1 -- 172.21.15.45:0/631584293 wait complete. 2023-09-23T09:34:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.213+0000 7f63c0a37b80 20 client.25835 trim_cache size 0 max 16384 2023-09-23T09:34:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.236+0000 7f63c0a37b80 1 Processor -- start 2023-09-23T09:34:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.236+0000 7f63c0a37b80 1 -- start start 2023-09-23T09:34:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.236+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f0b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.236+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f04860 0x558d29f07770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.236+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f0f710 0x558d29f07cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29cd88c0 con 0x558d29f03f40 2023-09-23T09:34:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29f04d00 con 0x558d29f04860 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29f0d940 con 0x558d29f0f710 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f0b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f04860 0x558d29f07770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f04860 0x558d29f07770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55894/0 (socket says 172.21.15.45:55894) 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f0b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34542/0 (socket says 172.21.15.45:34542) 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f0f710 0x558d29f07cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 -- 172.21.15.45:0/821880833 learned_addr learned my addr 172.21.15.45:0/821880833 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 -- 172.21.15.45:0/821880833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f0f710 msgr2=0x558d29f07cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f0f710 0x558d29f07cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 -- 172.21.15.45:0/821880833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 msgr2=0x558d29f0b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f0b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 -- 172.21.15.45:0/821880833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d29c82d80 con 0x558d29f04860 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63ae59c700 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f63add9b700 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f04860 0x558d29f07770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63a40115d0 tx=0x7f63a4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2fd660f887cc77b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.237+0000 7f639d7fa700 1 -- 172.21.15.45:0/821880833 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4020070 con 0x558d29f04860 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f639d7fa700 1 -- 172.21.15.45:0/821880833 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a4018040 con 0x558d29f04860 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f639d7fa700 1 -- 172.21.15.45:0/821880833 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4005240 con 0x558d29f04860 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 -- 172.21.15.45:0/821880833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f04860 msgr2=0x558d29f07770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f04860 0x558d29f07770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63a40115d0 tx=0x7f63a4009260 comp rx=0 tx=0).stop 2023-09-23T09:34:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 -- 172.21.15.45:0/821880833 shutdown_connections 2023-09-23T09:34:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f0f710 0x558d29f07cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/821880833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f04860 0x558d29f07770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 -- 172.21.15.45:0/821880833 >> 172.21.15.45:0/821880833 conn(0x558d29e79200 msgr2=0x558d29f1b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 -- 172.21.15.45:0/821880833 shutdown_connections 2023-09-23T09:34:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.238+0000 7f63c0a37b80 1 -- 172.21.15.45:0/821880833 wait complete. 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 -- start start 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 Processor -- start 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f14810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29f71f10 con 0x558d29f03f40 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29f72050 con 0x558d29f0f710 2023-09-23T09:34:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29f72190 con 0x558d29f04860 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63c0a37b80 10 client.0 _ll_register_callbacks cb 0x7f63a0004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34548/0 (socket says 172.21.15.45:34548) 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63ae59c700 1 -- 172.21.15.45:0/2247154349 learned_addr learned my addr 172.21.15.45:0/2247154349 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63add9b700 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f14810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63aed9d700 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63ae59c700 1 -- 172.21.15.45:0/2247154349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 msgr2=0x558d29f14810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63ae59c700 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f14810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63ae59c700 1 -- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 msgr2=0x558d29f719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:34:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.239+0000 7f63ae59c700 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63ae59c700 1 -- 172.21.15.45:0/2247154349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a0003d10 con 0x558d29f03f40 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63ae59c700 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f142d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f63a0003a40 tx=0x7f63a0013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747c8bd96792791a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f6395ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f6395ffb700 1 -- 172.21.15.45:0/2247154349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a002e040 con 0x558d29f03f40 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63a809b5c0 con 0x558d29f03f40 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f6395ffb700 1 -- 172.21.15.45:0/2247154349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f63a0017070 con 0x558d29f03f40 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f6395ffb700 1 -- 172.21.15.45:0/2247154349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a0024040 con 0x558d29f03f40 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63c0a37b80 10 client.16152 fetch_fsmap learned FSMap version 32 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558d29c6bc00 con 0x558d29f03f40 2023-09-23T09:34:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f6395ffb700 1 -- 172.21.15.45:0/2247154349 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a003f020 con 0x558d29f03f40 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f6395ffb700 1 -- 172.21.15.45:0/2247154349 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f63a0028bf0 con 0x558d29f03f40 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63c0a37b80 10 client.16152 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63c0a37b80 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.240+0000 7f63c0a37b80 20 client.16152 populate_metadata read hostname 'smithi045' 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.241+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f739c0 0x558d29f75de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.241+0000 7f63c0a37b80 4 client.16152 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.241+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d29e50ad0 con 0x558d29f739c0 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.241+0000 7f63add9b700 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f739c0 0x558d29f75de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:34:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.241+0000 7f63add9b700 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f739c0 0x558d29f75de0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1214807db85f26 server_cookie=7e53bc99ddb91c57 in_seq=0 out_seq=0 2023-09-23T09:34:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.241+0000 7f6395ffb700 10 client.16152 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f6395ffb700 1 -- 172.21.15.45:0/2247154349 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d29e50ad0 con 0x558d29f739c0 2023-09-23T09:34:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f6395ffb700 10 client.16152 handle_command_reply: tid=0 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 client.16152 shutdown 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 msgr2=0x558d29f142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f142d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f63a0003a40 tx=0x7f63a0013070 comp rx=0 tx=0).stop 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 shutdown_connections 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558d29f739c0 0x558d29f75de0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d29f0f710 0x558d29f719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f6395ffb700 0 client.16152 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d29f03f40 0x558d29f142d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 --2- 172.21.15.45:0/2247154349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d29f04860 0x558d29f14810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 >> 172.21.15.45:0/2247154349 conn(0x558d29e79200 msgr2=0x558d29f07770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 shutdown_connections 2023-09-23T09:34:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.263+0000 7f63c0a37b80 1 -- 172.21.15.45:0/2247154349 wait complete. 2023-09-23T09:34:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63c0a37b80 20 client.16152 trim_cache size 0 max 16384 2023-09-23T09:34:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:34:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:34:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:34:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:34:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f63980a17c0 msgr2=0x7f63980a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f63980a17c0 0x7f63980a3c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005950 tx=0x7f63a000b410 comp rx=0 tx=0).stop 2023-09-23T09:34:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01a0ea0 msgr2=0x7f63b01a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01a0ea0 0x7f63b01a3290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63a8015000 tx=0x7f63a8009480 comp rx=0 tx=0).stop 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 shutdown_connections 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01a0ea0 0x7f63b01a3290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f63980a17c0 0x7f63980a3c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b0100160 0x7f63b0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 --2- 172.21.15.45:0/3004478660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0105050 0x7f63b0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 >> 172.21.15.45:0/3004478660 conn(0x7f63b0076840 msgr2=0x7f63b00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 shutdown_connections 2023-09-23T09:34:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:34:55.264+0000 7f63b484d700 1 -- 172.21.15.45:0/3004478660 wait complete. 2023-09-23T09:34:56.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:55 smithi098 ceph-mon[102386]: pgmap v168: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 29 MiB/s wr, 2.01k op/s 2023-09-23T09:34:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:55 smithi045 ceph-mon[98627]: pgmap v168: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 29 MiB/s wr, 2.01k op/s 2023-09-23T09:34:56.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:55 smithi123 ceph-mon[103577]: pgmap v168: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 29 MiB/s wr, 2.01k op/s 2023-09-23T09:34:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 310.0 s, 2448.1 tps, lat 40.579 ms stddev 49.454 2023-09-23T09:34:57.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:56 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:57.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:56 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:57.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:56 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:34:58.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:57 smithi098 ceph-mon[102386]: pgmap v169: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2023-09-23T09:34:58.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:57 smithi045 ceph-mon[98627]: pgmap v169: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2023-09-23T09:34:58.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:57 smithi123 ceph-mon[103577]: pgmap v169: 97 pgs: 97 active+clean; 7.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2023-09-23T09:34:59.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:58 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:34:59.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:58 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:34:59.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:34:58 smithi098 ceph-mon[102386]: pgmap v170: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 2.06k op/s 2023-09-23T09:34:59.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:58 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:34:59.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:58 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:34:59.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:34:58 smithi045 ceph-mon[98627]: pgmap v170: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 2.06k op/s 2023-09-23T09:34:59.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:58 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:34:59.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:58 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:34:59.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:34:58 smithi123 ceph-mon[103577]: pgmap v170: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 2.06k op/s 2023-09-23T09:35:00.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:35:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 -- 172.21.15.45:0/4118687049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c100890 msgr2=0x7f0f0c1048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 --2- 172.21.15.45:0/4118687049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c100890 0x7f0f0c1048d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009990 tx=0x7f0efc009ca0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 -- 172.21.15.45:0/4118687049 shutdown_connections 2023-09-23T09:35:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 --2- 172.21.15.45:0/4118687049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f0c105000 0x7f0f0c10cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 --2- 172.21.15.45:0/4118687049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c0ffee0 0x7f0f0c1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 --2- 172.21.15.45:0/4118687049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c100890 0x7f0f0c1048d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 -- 172.21.15.45:0/4118687049 >> 172.21.15.45:0/4118687049 conn(0x7f0f0c0fb740 msgr2=0x7f0f0c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 -- 172.21.15.45:0/4118687049 shutdown_connections 2023-09-23T09:35:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 -- 172.21.15.45:0/4118687049 wait complete. 2023-09-23T09:35:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 Processor -- start 2023-09-23T09:35:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.348+0000 7f0f11e67700 1 -- start start 2023-09-23T09:35:00.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f11e67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c0ffee0 0x7f0f0c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f11e67700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c105000 0x7f0f0c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f11e67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f0c19b960 0x7f0f0c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f11e67700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c19c3e0 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f11e67700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c19c550 con 0x7f0f0c105000 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f11e67700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c19c710 con 0x7f0f0c19b960 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c0ffee0 0x7f0f0c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c0ffee0 0x7f0f0c196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34562/0 (socket says 172.21.15.45:34562) 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c105000 0x7f0f0c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/564160988 learned_addr learned my addr 172.21.15.45:0/564160988 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0bfff700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f0c19b960 0x7f0f0c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/564160988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f0c19b960 msgr2=0x7f0f0c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f0c19b960 0x7f0f0c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/564160988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c105000 msgr2=0x7f0f0c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c105000 0x7f0f0c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.349+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/564160988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc0095f0 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.350+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c0ffee0 0x7f0f0c196fc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9a0 tx=0x7f0f0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.350+0000 7f0f08ff9700 1 -- 172.21.15.45:0/564160988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00013d00 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.350+0000 7f0f08ff9700 1 -- 172.21.15.45:0/564160988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f00013e60 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.350+0000 7f0f08ff9700 1 -- 172.21.15.45:0/564160988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0001b980 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.351+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c101900 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.351+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c101d90 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.354+0000 7f0f08ff9700 1 -- 172.21.15.45:0/564160988 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f0f0000c4e0 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.355+0000 7f0f08ff9700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0ef40a1860 0x7f0ef40a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.355+0000 7f0f08ff9700 1 -- 172.21.15.45:0/564160988 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0f000d1320 con 0x7f0f0c0ffee0 2023-09-23T09:35:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.355+0000 7f0f0affd700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0ef40a1860 0x7f0ef40a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.355+0000 7f0f0affd700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0ef40a1860 0x7f0ef40a3d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b2e0 tx=0x7f0efc005960 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.355+0000 7f0f1e051b80 1 Processor -- start 2023-09-23T09:35:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.355+0000 7f0f1e051b80 1 -- start start 2023-09-23T09:35:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147400 0x5632cb1477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb152690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb152bd0 0x5632cb154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb0b0460 con 0x5632cb147400 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb0a1c20 con 0x5632cb147d20 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb09f540 con 0x5632cb152bd0 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147400 0x5632cb1477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147400 0x5632cb1477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34570/0 (socket says 172.21.15.45:34570) 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/1015134364 learned_addr learned my addr 172.21.15.45:0/1015134364 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0affd700 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0bfff700 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb152bd0 0x5632cb154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/1015134364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb152bd0 msgr2=0x5632cb154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb152bd0 0x5632cb154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/1015134364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 msgr2=0x5632cb152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/1015134364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632cb0fd8d0 con 0x5632cb147400 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147400 0x5632cb1477e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f000a61f0 tx=0x7f0f0009ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad039710d994112 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0ef1ffb700 1 -- 172.21.15.45:0/1015134364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f000a24e0 con 0x5632cb147400 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.356+0000 7f0ef1ffb700 1 -- 172.21.15.45:0/1015134364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f000a6720 con 0x5632cb147400 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0ef1ffb700 1 -- 172.21.15.45:0/1015134364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0009d880 con 0x5632cb147400 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1015134364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147400 msgr2=0x5632cb1477e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147400 0x5632cb1477e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f000a61f0 tx=0x7f0f0009ddc0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1015134364 shutdown_connections 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb152bd0 0x5632cb154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147400 0x5632cb1477e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1015134364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1015134364 >> 172.21.15.45:0/1015134364 conn(0x5632cb0bc260 msgr2=0x5632cb143790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:00.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1015134364 shutdown_connections 2023-09-23T09:35:00.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.357+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1015134364 wait complete. 2023-09-23T09:35:00.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 -- start start 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 Processor -- start 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb14f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147d20 0x5632cb14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb152bd0 0x5632cb1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb09ae30 con 0x5632cb147d20 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb037710 con 0x5632cb152bd0 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.358+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb0373b0 con 0x5632cb147400 2023-09-23T09:35:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f1e051b80 10 client.0 _ll_register_callbacks cb 0x5632cae94800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb152bd0 0x5632cb1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb152bd0 0x5632cb1503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55926/0 (socket says 172.21.15.45:55926) 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 -- 172.21.15.45:0/1867906191 learned_addr learned my addr 172.21.15.45:0/1867906191 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 -- 172.21.15.45:0/1867906191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 msgr2=0x5632cb14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb14f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 -- 172.21.15.45:0/1867906191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147d20 msgr2=0x5632cb14fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147d20 0x5632cb14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.359+0000 7f0f0bfff700 1 -- 172.21.15.45:0/1867906191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c101d90 con 0x5632cb152bd0 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0f0bfff700 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb152bd0 0x5632cb1503a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f04011f90 tx=0x7f0f04014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44dd604ab0454245 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0eeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0eeaffd700 1 -- 172.21.15.45:0/1867906191 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400e040 con 0x5632cb152bd0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632cb13b250 con 0x5632cb152bd0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0eeaffd700 1 -- 172.21.15.45:0/1867906191 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0f040090d0 con 0x5632cb152bd0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0eeaffd700 1 -- 172.21.15.45:0/1867906191 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f04009290 con 0x5632cb152bd0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0f1e051b80 10 client.25853 fetch_fsmap learned FSMap version 33 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.360+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632caec5de0 con 0x5632cb152bd0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0eeaffd700 1 -- 172.21.15.45:0/1867906191 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632cb13b250 con 0x5632cb152bd0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0eeaffd700 1 -- 172.21.15.45:0/1867906191 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0f0401c020 con 0x5632cb152bd0 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0f1e051b80 10 client.25853 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0f1e051b80 10 client.25853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0f1e051b80 20 client.25853 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb162780 0x5632cb164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0f1e051b80 4 client.25853 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:35:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.361+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632cb093e10 con 0x5632cb162780 2023-09-23T09:35:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.362+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb162780 0x5632cb164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.362+0000 7f0eeaffd700 10 client.25853 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.362+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb162780 0x5632cb164ba0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce360c89441e8ec server_cookie=fb079a6574386578 in_seq=0 out_seq=0 2023-09-23T09:35:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0eeaffd700 1 -- 172.21.15.45:0/1867906191 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5632cb093e10 con 0x5632cb162780 2023-09-23T09:35:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0eeaffd700 10 client.25853 handle_command_reply: tid=0 2023-09-23T09:35:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 client.25853 shutdown 2023-09-23T09:35:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb152bd0 msgr2=0x5632cb1503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb152bd0 0x5632cb1503a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f04011f90 tx=0x7f0f04014600 comp rx=0 tx=0).stop 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 shutdown_connections 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb162780 0x5632cb164ba0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb152bd0 0x5632cb1503a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0eeaffd700 0 client.25853 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/1867906191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb147d20 0x5632cb14fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.363+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 >> 172.21.15.45:0/1867906191 conn(0x5632cb0bc260 msgr2=0x5632cb143320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.364+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 shutdown_connections 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.364+0000 7f0f1e051b80 1 -- 172.21.15.45:0/1867906191 wait complete. 2023-09-23T09:35:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.364+0000 7f0f1e051b80 20 client.25853 trim_cache size 0 max 16384 2023-09-23T09:35:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 Processor -- start 2023-09-23T09:35:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 -- start start 2023-09-23T09:35:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb1501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb149a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb149f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632caf1b920 con 0x5632cb152bd0 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb14a7f0 con 0x5632cb147d20 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb144630 con 0x5632cb147400 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb1501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb1501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36690/0 (socket says 172.21.15.45:36690) 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/4180232493 learned_addr learned my addr 172.21.15.45:0/4180232493 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.386+0000 7f0f0bfff700 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0f0affd700 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/4180232493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 msgr2=0x5632cb149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/4180232493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 msgr2=0x5632cb149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/4180232493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632caeaec60 con 0x5632cb147400 2023-09-23T09:35:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb1501d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f000cefe0 tx=0x7f0f0009e510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da4ea7115a55060 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:00.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0ef27fc700 1 -- 172.21.15.45:0/4180232493 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000cc10 con 0x5632cb147400 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0ef27fc700 1 -- 172.21.15.45:0/4180232493 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f000a2170 con 0x5632cb147400 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.387+0000 7f0ef27fc700 1 -- 172.21.15.45:0/4180232493 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0009e850 con 0x5632cb147400 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 -- 172.21.15.45:0/4180232493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 msgr2=0x5632cb1501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb1501d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f000cefe0 tx=0x7f0f0009e510 comp rx=0 tx=0).stop 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 -- 172.21.15.45:0/4180232493 shutdown_connections 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147400 0x5632cb1501d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/4180232493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147d20 0x5632cb149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 -- 172.21.15.45:0/4180232493 >> 172.21.15.45:0/4180232493 conn(0x5632cb0bc260 msgr2=0x5632cb1492f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 -- 172.21.15.45:0/4180232493 shutdown_connections 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.388+0000 7f0f1e051b80 1 -- 172.21.15.45:0/4180232493 wait complete. 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 -- start start 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 Processor -- start 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147400 0x5632cb158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147d20 0x5632cb1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb157450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb157a70 con 0x5632cb152bd0 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb157be0 con 0x5632cb147400 2023-09-23T09:35:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632cb157da0 con 0x5632cb147d20 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f1e051b80 10 client.0 _ll_register_callbacks cb 0x7f0f040118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147400 0x5632cb158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147400 0x5632cb158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55940/0 (socket says 172.21.15.45:55940) 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f0affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147d20 0x5632cb1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.389+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/3079866770 learned_addr learned my addr 172.21.15.45:0/3079866770 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/3079866770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147d20 msgr2=0x5632cb1594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147d20 0x5632cb1594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/3079866770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 msgr2=0x5632cb157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f0b7fe700 1 -- 172.21.15.45:0/3079866770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632cb0fd8d0 con 0x5632cb147400 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f0b7fe700 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147400 0x5632cb158fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f0009c6e0 tx=0x7f0f0009eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913477a3d265e1e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0eeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0eeaffd700 1 -- 172.21.15.45:0/3079866770 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0009e7f0 con 0x5632cb147400 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f000a6720 con 0x5632cb147400 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0eeaffd700 1 -- 172.21.15.45:0/3079866770 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0f0009d880 con 0x5632cb147400 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0eeaffd700 1 -- 172.21.15.45:0/3079866770 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f0009da40 con 0x5632cb147400 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f1e051b80 10 client.25859 fetch_fsmap learned FSMap version 33 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632caeec250 con 0x5632cb147400 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.390+0000 7f0eeaffd700 1 -- 172.21.15.45:0/3079866770 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f000a6720 con 0x5632cb147400 2023-09-23T09:35:00.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0eeaffd700 1 -- 172.21.15.45:0/3079866770 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0f0009a180 con 0x5632cb147400 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f1e051b80 10 client.25859 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f1e051b80 10 client.25859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f1e051b80 20 client.25859 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb1bf9d0 0x5632cb1c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f1e051b80 4 client.25859 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632cb093b30 con 0x5632cb1bf9d0 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f0affd700 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb1bf9d0 0x5632cb1c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0eeaffd700 10 client.25859 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.391+0000 7f0f0affd700 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb1bf9d0 0x5632cb1c1dc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d815a47789c929 server_cookie=e55cb4583d18e6d7 in_seq=0 out_seq=0 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0eeaffd700 1 -- 172.21.15.45:0/3079866770 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5632cb093b30 con 0x5632cb1bf9d0 2023-09-23T09:35:00.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0eeaffd700 10 client.25859 handle_command_reply: tid=0 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 client.25859 shutdown 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147400 msgr2=0x5632cb158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147400 0x5632cb158fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f0009c6e0 tx=0x7f0f0009eb90 comp rx=0 tx=0).stop 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 shutdown_connections 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5632cb1bf9d0 0x5632cb1c1dc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632cb152bd0 0x5632cb157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632cb147400 0x5632cb158fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 --2- 172.21.15.45:0/3079866770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632cb147d20 0x5632cb1594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0eeaffd700 0 client.25859 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 >> 172.21.15.45:0/3079866770 conn(0x5632cb0bc260 msgr2=0x5632cb149b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:00.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 shutdown_connections 2023-09-23T09:35:00.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 1 -- 172.21.15.45:0/3079866770 wait complete. 2023-09-23T09:35:00.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.392+0000 7f0f1e051b80 20 client.25859 trim_cache size 0 max 16384 2023-09-23T09:35:00.401 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:35:00.401 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:35:00.401 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:35:00.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:35:00.401 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:35:00.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:35:00.402 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:35:00.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:35:00.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:35:00.402 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:35:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.394+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0ef40a1860 msgr2=0x7f0ef40a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.394+0000 7f0f11e67700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0ef40a1860 0x7f0ef40a3d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b2e0 tx=0x7f0efc005960 comp rx=0 tx=0).stop 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.394+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c0ffee0 msgr2=0x7f0f0c196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.394+0000 7f0f11e67700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c0ffee0 0x7f0f0c196fc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9a0 tx=0x7f0f0000ecb0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.394+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 shutdown_connections 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.395+0000 7f0f11e67700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f0c19b960 0x7f0f0c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.395+0000 7f0f11e67700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0ef40a1860 0x7f0ef40a3d20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.395+0000 7f0f11e67700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f0c0ffee0 0x7f0f0c196fc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.395+0000 7f0f11e67700 1 --2- 172.21.15.45:0/564160988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c105000 0x7f0f0c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.395+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 >> 172.21.15.45:0/564160988 conn(0x7f0f0c0fb740 msgr2=0x7f0f0c0691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.395+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 shutdown_connections 2023-09-23T09:35:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:00.395+0000 7f0f11e67700 1 -- 172.21.15.45:0/564160988 wait complete. 2023-09-23T09:35:00.418 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:35:01.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:01 smithi098 ceph-mon[102386]: pgmap v171: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 15 MiB/s wr, 968 op/s 2023-09-23T09:35:01.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:01 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:01.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:01 smithi045 ceph-mon[98627]: pgmap v171: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 15 MiB/s wr, 968 op/s 2023-09-23T09:35:01.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:01 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:01.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:01 smithi123 ceph-mon[103577]: pgmap v171: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 15 MiB/s wr, 968 op/s 2023-09-23T09:35:01.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:01 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 315.0 s, 2068.8 tps, lat 46.997 ms stddev 59.313 2023-09-23T09:35:03.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:03 smithi098 ceph-mon[102386]: pgmap v172: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 15 MiB/s wr, 968 op/s 2023-09-23T09:35:03.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:03 smithi045 ceph-mon[98627]: pgmap v172: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 15 MiB/s wr, 968 op/s 2023-09-23T09:35:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:03 smithi123 ceph-mon[103577]: pgmap v172: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 15 MiB/s wr, 968 op/s 2023-09-23T09:35:05.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:05 smithi098 ceph-mon[102386]: pgmap v173: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-09-23T09:35:06.037 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:05 smithi123 ceph-mon[103577]: pgmap v173: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-09-23T09:35:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:05 smithi045 ceph-mon[98627]: pgmap v173: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-09-23T09:35:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 320.0 s, 2180.4 tps, lat 47.200 ms stddev 101.038 2023-09-23T09:35:07.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:07 smithi098 ceph-mon[102386]: pgmap v174: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-09-23T09:35:07.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:07 smithi045 ceph-mon[98627]: pgmap v174: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-09-23T09:35:07.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:07 smithi123 ceph-mon[103577]: pgmap v174: 97 pgs: 97 active+clean; 7.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-09-23T09:35:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.419+0000 7f132aab2700 1 -- 172.21.15.45:0/2769852328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132410f9e0 msgr2=0x7f1324108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.419+0000 7f132aab2700 1 --2- 172.21.15.45:0/2769852328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132410f9e0 0x7f1324108360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f131001a9a0 tx=0x7f131001acb0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.419+0000 7f132aab2700 1 -- 172.21.15.45:0/2769852328 shutdown_connections 2023-09-23T09:35:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.419+0000 7f132aab2700 1 --2- 172.21.15.45:0/2769852328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f132411a600 0x7f132411c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.419+0000 7f132aab2700 1 --2- 172.21.15.45:0/2769852328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132410f570 0x7f1324107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.419+0000 7f132aab2700 1 --2- 172.21.15.45:0/2769852328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132410f9e0 0x7f1324108360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.419+0000 7f132aab2700 1 -- 172.21.15.45:0/2769852328 >> 172.21.15.45:0/2769852328 conn(0x7f132406cee0 msgr2=0x7f132406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.421+0000 7f132aab2700 1 -- 172.21.15.45:0/2769852328 shutdown_connections 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.421+0000 7f132aab2700 1 -- 172.21.15.45:0/2769852328 wait complete. 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 Processor -- start 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 -- start start 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f132410f570 0x7f1324115b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132410f9e0 0x7f1324116040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132411a600 0x7f13241a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13241a28e0 con 0x7f132410f570 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13241a2a20 con 0x7f132411a600 2023-09-23T09:35:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.422+0000 7f132aab2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13241a2be0 con 0x7f132410f9e0 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132411a600 0x7f13241a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132411a600 0x7f13241a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55956/0 (socket says 172.21.15.45:55956) 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 -- 172.21.15.45:0/1796836193 learned_addr learned my addr 172.21.15.45:0/1796836193 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132884e700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f132410f570 0x7f1324115b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 -- 172.21.15.45:0/1796836193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132410f9e0 msgr2=0x7f1324116040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132410f9e0 0x7f1324116040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 -- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f132410f570 msgr2=0x7f1324115b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f132410f570 0x7f1324115b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.423+0000 7f132904f700 1 -- 172.21.15.45:0/1796836193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131001a600 con 0x7f132411a600 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.424+0000 7f132904f700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132411a600 0x7f13241a2280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f131c00b9e0 tx=0x7f131c00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.424+0000 7f1321ffb700 1 -- 172.21.15.45:0/1796836193 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00c9b0 con 0x7f132411a600 2023-09-23T09:35:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.424+0000 7f1321ffb700 1 -- 172.21.15.45:0/1796836193 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131c014440 con 0x7f132411a600 2023-09-23T09:35:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.424+0000 7f1321ffb700 1 -- 172.21.15.45:0/1796836193 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c0125a0 con 0x7f132411a600 2023-09-23T09:35:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.424+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13241a2ec0 con 0x7f132411a600 2023-09-23T09:35:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.424+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13241aa640 con 0x7f132411a600 2023-09-23T09:35:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.427+0000 7f1321ffb700 1 -- 172.21.15.45:0/1796836193 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f131c0145b0 con 0x7f132411a600 2023-09-23T09:35:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.428+0000 7f1321ffb700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f13080a1750 0x7f13080a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.428+0000 7f132884e700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f13080a1750 0x7f13080a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.428+0000 7f1321ffb700 1 -- 172.21.15.45:0/1796836193 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f131c006e40 con 0x7f132411a600 2023-09-23T09:35:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.429+0000 7f132884e700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f13080a1750 0x7f13080a3c10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1314006fd0 tx=0x7f1314008040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 Processor -- start 2023-09-23T09:35:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 -- start start 2023-09-23T09:35:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682967f40 0x559682968320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682968860 0x5596829731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 0x559682975b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596828d1400 con 0x559682968860 2023-09-23T09:35:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596828c2bc0 con 0x559682967f40 2023-09-23T09:35:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.431+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596828c04e0 con 0x559682973710 2023-09-23T09:35:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682968860 0x5596829731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682968860 0x5596829731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34614/0 (socket says 172.21.15.45:34614) 2023-09-23T09:35:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 -- 172.21.15.45:0/2393021682 learned_addr learned my addr 172.21.15.45:0/2393021682 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f132884e700 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682967f40 0x559682968320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 -- 172.21.15.45:0/2393021682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 msgr2=0x559682975b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:35:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 0x559682975b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 -- 172.21.15.45:0/2393021682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682967f40 msgr2=0x559682968320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682967f40 0x559682968320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 -- 172.21.15.45:0/2393021682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55968291e870 con 0x559682968860 2023-09-23T09:35:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.432+0000 7f1323fff700 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682968860 0x5596829731d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1310000c00 tx=0x7f131001c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff74ef9caea387d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.433+0000 7f131affd700 1 -- 172.21.15.45:0/2393021682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310005220 con 0x559682968860 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.433+0000 7f131affd700 1 -- 172.21.15.45:0/2393021682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131001c990 con 0x559682968860 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.433+0000 7f131affd700 1 -- 172.21.15.45:0/2393021682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310003d00 con 0x559682968860 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2393021682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682968860 msgr2=0x5596829731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682968860 0x5596829731d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1310000c00 tx=0x7f131001c550 comp rx=0 tx=0).stop 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2393021682 shutdown_connections 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 0x559682975b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682967f40 0x559682968320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2393021682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682968860 0x5596829731d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2393021682 >> 172.21.15.45:0/2393021682 conn(0x5596828dd200 msgr2=0x5596829643c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2393021682 shutdown_connections 2023-09-23T09:35:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2393021682 wait complete. 2023-09-23T09:35:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.434+0000 7f1336c9cb80 1 -- start start 2023-09-23T09:35:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 1 Processor -- start 2023-09-23T09:35:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x559682970600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682973710 0x559682970b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x55968297f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596828bbdd0 con 0x559682973710 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596828586b0 con 0x559682971080 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682858350 con 0x559682967f40 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1336c9cb80 10 client.0 _ll_register_callbacks cb 0x5596826b57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1323fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682973710 0x559682970b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f132884e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x559682970600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f132904f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x55968297f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1323fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682973710 0x559682970b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34618/0 (socket says 172.21.15.45:34618) 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.435+0000 7f1323fff700 1 -- 172.21.15.45:0/2281243360 learned_addr learned my addr 172.21.15.45:0/2281243360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1323fff700 1 -- 172.21.15.45:0/2281243360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 msgr2=0x559682970600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1323fff700 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x559682970600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1323fff700 1 -- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 msgr2=0x55968297f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1323fff700 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x55968297f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1323fff700 1 -- 172.21.15.45:0/2281243360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132411d320 con 0x559682973710 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f132904f700 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x55968297f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1323fff700 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682973710 0x559682970b40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1310006010 tx=0x7f131001cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3051a687bfd2702d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1307fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1307fff700 1 -- 172.21.15.45:0/2281243360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310005da0 con 0x559682973710 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55968295c090 con 0x559682973710 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1307fff700 1 -- 172.21.15.45:0/2281243360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1310005630 con 0x559682973710 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1307fff700 1 -- 172.21.15.45:0/2281243360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1310003960 con 0x559682973710 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1336c9cb80 10 client.16176 fetch_fsmap learned FSMap version 34 2023-09-23T09:35:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.436+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5596826e6d80 con 0x559682973710 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1307fff700 1 -- 172.21.15.45:0/2281243360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55968295c090 con 0x559682973710 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1307fff700 1 -- 172.21.15.45:0/2281243360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f131005cbd0 con 0x559682973710 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1336c9cb80 10 client.16176 fetch_fsmap finished waiting for FSMap version 34 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1336c9cb80 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1336c9cb80 20 client.16176 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1336c9cb80 4 client.16176 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596828b4db0 con 0x559682983380 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.437+0000 7f132884e700 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1307fff700 10 client.16176 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f132884e700 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829857a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf05bd2727aa407 server_cookie=e0e8dcc52c4c58c8 in_seq=0 out_seq=0 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1307fff700 1 -- 172.21.15.45:0/2281243360 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596828b4db0 con 0x559682983380 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1307fff700 10 client.16176 handle_command_reply: tid=0 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1336c9cb80 1 client.16176 shutdown 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682973710 msgr2=0x559682970b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682973710 0x559682970b40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1310006010 tx=0x7f131001cdb0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 shutdown_connections 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829857a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x55968297f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1307fff700 0 client.16176 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.438+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x559682970600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.439+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/2281243360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682973710 0x559682970b40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.439+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 >> 172.21.15.45:0/2281243360 conn(0x5596828dd200 msgr2=0x559682963fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.439+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 shutdown_connections 2023-09-23T09:35:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.439+0000 7f1336c9cb80 1 -- 172.21.15.45:0/2281243360 wait complete. 2023-09-23T09:35:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.439+0000 7f1336c9cb80 20 client.16176 trim_cache size 0 max 16384 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.453+0000 7f1336c9cb80 1 Processor -- start 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.453+0000 7f1336c9cb80 1 -- start start 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x55968297f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682971080 0x55968297a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682973710 0x559682972620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968273c8c0 con 0x559682971080 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682973420 con 0x559682973710 2023-09-23T09:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682968c40 con 0x559682967f40 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682971080 0x55968297a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682971080 0x55968297a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34626/0 (socket says 172.21.15.45:34626) 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f132884e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x55968297f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 -- 172.21.15.45:0/1608127929 learned_addr learned my addr 172.21.15.45:0/1608127929 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f132904f700 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682973710 0x559682972620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 -- 172.21.15.45:0/1608127929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 msgr2=0x55968297f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x55968297f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 -- 172.21.15.45:0/1608127929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682973710 msgr2=0x559682972620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682973710 0x559682972620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 -- 172.21.15.45:0/1608127929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596826cfc00 con 0x559682971080 2023-09-23T09:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f132884e700 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x55968297f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f1323fff700 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682971080 0x55968297a230 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f131005cf60 tx=0x7f131001a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356537debad9fd88 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.454+0000 7f131b7fe700 1 -- 172.21.15.45:0/1608127929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131001ced0 con 0x559682971080 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f131b7fe700 1 -- 172.21.15.45:0/1608127929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1310005cb0 con 0x559682971080 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f131b7fe700 1 -- 172.21.15.45:0/1608127929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131005b3b0 con 0x559682971080 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 -- 172.21.15.45:0/1608127929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682971080 msgr2=0x55968297a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682971080 0x55968297a230 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f131005cf60 tx=0x7f131001a110 comp rx=0 tx=0).stop 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 -- 172.21.15.45:0/1608127929 shutdown_connections 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682973710 0x559682972620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682967f40 0x55968297f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/1608127929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682971080 0x55968297a230 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 -- 172.21.15.45:0/1608127929 >> 172.21.15.45:0/1608127929 conn(0x5596828dd200 msgr2=0x559682971f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 -- 172.21.15.45:0/1608127929 shutdown_connections 2023-09-23T09:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 -- 172.21.15.45:0/1608127929 wait complete. 2023-09-23T09:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.455+0000 7f1336c9cb80 1 -- start start 2023-09-23T09:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 1 Processor -- start 2023-09-23T09:35:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682967f40 0x5596829cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x5596829d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 0x559682979920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682979f00 con 0x559682967f40 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968297a070 con 0x559682971080 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968297a230 con 0x559682973710 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1336c9cb80 10 client.0 _ll_register_callbacks cb 0x7f13140091a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682967f40 0x5596829cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682967f40 0x5596829cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34630/0 (socket says 172.21.15.45:34630) 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 -- 172.21.15.45:0/835896391 learned_addr learned my addr 172.21.15.45:0/835896391 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132904f700 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 0x559682979920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f1323fff700 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x5596829d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 -- 172.21.15.45:0/835896391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 msgr2=0x559682979920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 0x559682979920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 -- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 msgr2=0x5596829d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x5596829d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.456+0000 7f132884e700 1 -- 172.21.15.45:0/835896391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131400e040 con 0x559682967f40 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f132884e700 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682967f40 0x5596829cfe00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f131400d440 tx=0x7f1314010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddc0d32bc5f5736 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1307fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1307fff700 1 -- 172.21.15.45:0/835896391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314015c40 con 0x559682967f40 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1307fff700 1 -- 172.21.15.45:0/835896391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1314015da0 con 0x559682967f40 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f131001c990 con 0x559682967f40 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1307fff700 1 -- 172.21.15.45:0/835896391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1314030440 con 0x559682967f40 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 10 client.16182 fetch_fsmap learned FSMap version 34 2023-09-23T09:35:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5596826e6d80 con 0x559682967f40 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1307fff700 1 -- 172.21.15.45:0/835896391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131403c080 con 0x559682967f40 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1307fff700 1 -- 172.21.15.45:0/835896391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f13140385d0 con 0x559682967f40 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 34 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 20 client.16182 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 4 client.16182 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596828b4ad0 con 0x559682983380 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.457+0000 7f1323fff700 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.458+0000 7f1307fff700 10 client.16182 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.458+0000 7f1323fff700 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829d9dd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399673b0a2b4d66 server_cookie=486e17df460b545a in_seq=0 out_seq=0 2023-09-23T09:35:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.501+0000 7f1307fff700 1 -- 172.21.15.45:0/835896391 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596828b4ad0 con 0x559682983380 2023-09-23T09:35:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.501+0000 7f1307fff700 10 client.16182 handle_command_reply: tid=0 2023-09-23T09:35:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 client.16182 shutdown 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682967f40 msgr2=0x5596829cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682967f40 0x5596829cfe00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f131400d440 tx=0x7f1314010000 comp rx=0 tx=0).stop 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 shutdown_connections 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559682983380 0x5596829d9dd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559682973710 0x559682979920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559682967f40 0x5596829cfe00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 --2- 172.21.15.45:0/835896391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559682971080 0x5596829d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 >> 172.21.15.45:0/835896391 conn(0x5596828dd200 msgr2=0x55968297f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.502+0000 7f1307fff700 0 client.16182 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.503+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 shutdown_connections 2023-09-23T09:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.503+0000 7f1336c9cb80 1 -- 172.21.15.45:0/835896391 wait complete. 2023-09-23T09:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.503+0000 7f1336c9cb80 20 client.16182 trim_cache size 0 max 16384 2023-09-23T09:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f13080a1750 msgr2=0x7f13080a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f13080a1750 0x7f13080a3c10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1314006fd0 tx=0x7f1314008040 comp rx=0 tx=0).stop 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132411a600 msgr2=0x7f13241a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132411a600 0x7f13241a2280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f131c00b9e0 tx=0x7f131c00bda0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 shutdown_connections 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f132411a600 0x7f13241a2280 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f13080a1750 0x7f13080a3c10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f132410f570 0x7f1324115b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 --2- 172.21.15.45:0/1796836193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132410f9e0 0x7f1324116040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 >> 172.21.15.45:0/1796836193 conn(0x7f132406cee0 msgr2=0x7f13240717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 shutdown_connections 2023-09-23T09:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:08.504+0000 7f132aab2700 1 -- 172.21.15.45:0/1796836193 wait complete. 2023-09-23T09:35:09.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:09 smithi098 ceph-mon[102386]: pgmap v175: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 1.87k op/s 2023-09-23T09:35:09.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:09 smithi045 ceph-mon[98627]: pgmap v175: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 1.87k op/s 2023-09-23T09:35:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:09 smithi123 ceph-mon[103577]: pgmap v175: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 1.87k op/s 2023-09-23T09:35:10.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:10 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:10.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:10 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:10.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:10 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:11.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:11 smithi098 ceph-mon[102386]: pgmap v176: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-23T09:35:11.835 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:11 smithi045 ceph-mon[98627]: pgmap v176: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-23T09:35:11.842 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:11 smithi123 ceph-mon[103577]: pgmap v176: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-23T09:35:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 325.0 s, 2660.8 tps, lat 37.405 ms stddev 44.102 2023-09-23T09:35:13.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:13 smithi098 ceph-mon[102386]: pgmap v177: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-23T09:35:13.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:13 smithi045 ceph-mon[98627]: pgmap v177: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-23T09:35:13.842 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:13 smithi123 ceph-mon[103577]: pgmap v177: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-23T09:35:15.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:15 smithi098 ceph-mon[102386]: pgmap v178: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 26 MiB/s wr, 1.84k op/s 2023-09-23T09:35:15.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:15 smithi045 ceph-mon[98627]: pgmap v178: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 26 MiB/s wr, 1.84k op/s 2023-09-23T09:35:15.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:15 smithi123 ceph-mon[103577]: pgmap v178: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 26 MiB/s wr, 1.84k op/s 2023-09-23T09:35:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 330.0 s, 2175.2 tps, lat 46.132 ms stddev 51.693 2023-09-23T09:35:17.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:17 smithi098 ceph-mon[102386]: pgmap v179: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 873 op/s 2023-09-23T09:35:17.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:17 smithi045 ceph-mon[98627]: pgmap v179: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 873 op/s 2023-09-23T09:35:17.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:17 smithi123 ceph-mon[103577]: pgmap v179: 97 pgs: 97 active+clean; 7.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 873 op/s 2023-09-23T09:35:19.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:19 smithi098 ceph-mon[102386]: pgmap v180: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 1.15k op/s 2023-09-23T09:35:19.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:19 smithi045 ceph-mon[98627]: pgmap v180: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 1.15k op/s 2023-09-23T09:35:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:19 smithi123 ceph-mon[103577]: pgmap v180: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 1.15k op/s 2023-09-23T09:35:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.714+0000 7f216be24700 1 -- 172.21.15.45:0/3187609029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f216410b7e0 msgr2=0x7f216410bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.714+0000 7f216be24700 1 --2- 172.21.15.45:0/3187609029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f216410b7e0 0x7f216410bc60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f215c00b0a0 tx=0x7f215c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.714+0000 7f216be24700 1 -- 172.21.15.45:0/3187609029 shutdown_connections 2023-09-23T09:35:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.714+0000 7f216be24700 1 --2- 172.21.15.45:0/3187609029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f216410c1a0 0x7f2164118660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.714+0000 7f216be24700 1 --2- 172.21.15.45:0/3187609029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21641117a0 0x7f2164111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.714+0000 7f216be24700 1 --2- 172.21.15.45:0/3187609029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f216410b7e0 0x7f216410bc60 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f215c00b0a0 tx=0x7f215c00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.714+0000 7f216be24700 1 -- 172.21.15.45:0/3187609029 >> 172.21.15.45:0/3187609029 conn(0x7f216406bbb0 msgr2=0x7f2164109e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.735+0000 7f216be24700 1 -- 172.21.15.45:0/3187609029 shutdown_connections 2023-09-23T09:35:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.737+0000 7f216be24700 1 -- 172.21.15.45:0/3187609029 wait complete. 2023-09-23T09:35:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.738+0000 7f216be24700 1 Processor -- start 2023-09-23T09:35:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.739+0000 7f216be24700 1 -- start start 2023-09-23T09:35:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.739+0000 7f216be24700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f216410c1a0 0x7f2164116e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.739+0000 7f216be24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21641117a0 0x7f2164117340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.739+0000 7f216be24700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2164117880 0x7f21641bacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.739+0000 7f216be24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21641bb230 con 0x7f21641117a0 2023-09-23T09:35:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.739+0000 7f216be24700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21641bb370 con 0x7f2164117880 2023-09-23T09:35:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.739+0000 7f216be24700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21641bb4b0 con 0x7f216410c1a0 2023-09-23T09:35:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2164117880 0x7f21641bacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2164117880 0x7f21641bacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:55992/0 (socket says 172.21.15.45:55992) 2023-09-23T09:35:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 -- 172.21.15.45:0/1290737618 learned_addr learned my addr 172.21.15.45:0/1290737618 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 -- 172.21.15.45:0/1290737618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f216410c1a0 msgr2=0x7f2164116e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:35:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f216410c1a0 0x7f2164116e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 -- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21641117a0 msgr2=0x7f2164117340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21641117a0 0x7f2164117340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.740+0000 7f216a3c1700 1 -- 172.21.15.45:0/1290737618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c009d00 con 0x7f2164117880 2023-09-23T09:35:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.741+0000 7f216a3c1700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2164117880 0x7f21641bacf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f216000b550 tx=0x7f216000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.741+0000 7f215affd700 1 -- 172.21.15.45:0/1290737618 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011830 con 0x7f2164117880 2023-09-23T09:35:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.741+0000 7f215affd700 1 -- 172.21.15.45:0/1290737618 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2160011e50 con 0x7f2164117880 2023-09-23T09:35:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.741+0000 7f215affd700 1 -- 172.21.15.45:0/1290737618 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160010e40 con 0x7f2164117880 2023-09-23T09:35:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.741+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21641bb790 con 0x7f2164117880 2023-09-23T09:35:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.741+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21641bbc50 con 0x7f2164117880 2023-09-23T09:35:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.754+0000 7f217800eb80 1 Processor -- start 2023-09-23T09:35:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.755+0000 7f215affd700 1 -- 172.21.15.45:0/1290737618 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f2160011990 con 0x7f2164117880 2023-09-23T09:35:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.755+0000 7f217800eb80 1 -- start start 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.756+0000 7f217800eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd391320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.756+0000 7f217800eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.756+0000 7f217800eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd39c710 0x5615cd39eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.756+0000 7f217800eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd2fa400 con 0x5615cd39c710 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.756+0000 7f217800eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd2ebbc0 con 0x5615cd391860 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.756+0000 7f217800eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd2e94e0 con 0x5615cd390f40 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.756+0000 7f215affd700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f21500a17d0 0x7f21500a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.757+0000 7f215affd700 1 -- 172.21.15.45:0/1290737618 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f21600ccc80 con 0x7f2164117880 2023-09-23T09:35:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd391320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd391320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36752/0 (socket says 172.21.15.45:36752) 2023-09-23T09:35:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 -- 172.21.15.45:0/1565109368 learned_addr learned my addr 172.21.15.45:0/1565109368 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd39c710 0x5615cd39eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 -- 172.21.15.45:0/1565109368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 msgr2=0x5615cd39c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 -- 172.21.15.45:0/1565109368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd39c710 msgr2=0x5615cd39eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd39c710 0x5615cd39eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 -- 172.21.15.45:0/1565109368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615cd347870 con 0x5615cd390f40 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.758+0000 7f21693bf700 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd391320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f215c007fd0 tx=0x7f215c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9beca7f72c926c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.759+0000 7f214ffff700 1 -- 172.21.15.45:0/1565109368 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00e040 con 0x5615cd390f40 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.759+0000 7f214ffff700 1 -- 172.21.15.45:0/1565109368 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f215c024020 con 0x5615cd390f40 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.759+0000 7f214ffff700 1 -- 172.21.15.45:0/1565109368 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c005110 con 0x5615cd390f40 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.760+0000 7f217800eb80 1 -- 172.21.15.45:0/1565109368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 msgr2=0x5615cd391320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.760+0000 7f217800eb80 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd391320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f215c007fd0 tx=0x7f215c0099e0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f216a3c1700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f21500a17d0 0x7f21500a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f217800eb80 1 -- 172.21.15.45:0/1565109368 shutdown_connections 2023-09-23T09:35:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f217800eb80 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd39c710 0x5615cd39eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f217800eb80 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd391320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f217800eb80 1 --2- 172.21.15.45:0/1565109368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f217800eb80 1 -- 172.21.15.45:0/1565109368 >> 172.21.15.45:0/1565109368 conn(0x5615cd306200 msgr2=0x5615cd38d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f216a3c1700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f21500a17d0 0x7f21500a3c90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21600961f0 tx=0x7f2160004750 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f217800eb80 1 -- 172.21.15.45:0/1565109368 shutdown_connections 2023-09-23T09:35:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.762+0000 7f217800eb80 1 -- 172.21.15.45:0/1565109368 wait complete. 2023-09-23T09:35:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 -- start start 2023-09-23T09:35:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 Processor -- start 2023-09-23T09:35:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd39c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd391860 0x5615cd3992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd399890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd2e4dd0 con 0x5615cd391860 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd2816b0 con 0x5615cd39c710 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd281350 con 0x5615cd390f40 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.763+0000 7f217800eb80 10 client.0 _ll_register_callbacks cb 0x5615cd0de7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd39c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd39c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36760/0 (socket says 172.21.15.45:36760) 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 -- 172.21.15.45:0/1598743378 learned_addr learned my addr 172.21.15.45:0/1598743378 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd399890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 -- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 msgr2=0x5615cd399890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd399890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 -- 172.21.15.45:0/1598743378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd391860 msgr2=0x5615cd3992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd391860 0x5615cd3992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.764+0000 7f21693bf700 1 -- 172.21.15.45:0/1598743378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2164118f60 con 0x5615cd390f40 2023-09-23T09:35:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f21693bf700 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd39c050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f215c009710 tx=0x7f215c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955b4302dfae4c9d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f214d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:35:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f214d7fa700 1 -- 172.21.15.45:0/1598743378 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00e070 con 0x5615cd390f40 2023-09-23T09:35:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f214d7fa700 1 -- 172.21.15.45:0/1598743378 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f215c025070 con 0x5615cd390f40 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615cd385090 con 0x5615cd390f40 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f214d7fa700 1 -- 172.21.15.45:0/1598743378 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f215c004f00 con 0x5615cd390f40 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f217800eb80 10 client.25873 fetch_fsmap learned FSMap version 35 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5615cd10fd80 con 0x5615cd390f40 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f214d7fa700 1 -- 172.21.15.45:0/1598743378 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f215c03a020 con 0x5615cd390f40 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.765+0000 7f214d7fa700 1 -- 172.21.15.45:0/1598743378 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 35) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f215c00b4d0 con 0x5615cd390f40 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f217800eb80 10 client.25873 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f217800eb80 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f217800eb80 20 client.25873 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f217800eb80 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd3abe90 0x5615cd3ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f217800eb80 4 client.25873 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:35:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615cd2dddb0 con 0x5615cd3abe90 2023-09-23T09:35:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f2169bc0700 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd3abe90 0x5615cd3ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f214d7fa700 10 client.25873 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.766+0000 7f2169bc0700 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd3abe90 0x5615cd3ae2b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b958d498088d849 server_cookie=c5eeead0793bed4a in_seq=0 out_seq=0 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.767+0000 7f214d7fa700 1 -- 172.21.15.45:0/1598743378 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5615cd2dddb0 con 0x5615cd3abe90 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.767+0000 7f214d7fa700 10 client.25873 handle_command_reply: tid=0 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 client.25873 shutdown 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 msgr2=0x5615cd39c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd39c050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f215c009710 tx=0x7f215c0096a0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 shutdown_connections 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd3abe90 0x5615cd3ae2b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd399890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd390f40 0x5615cd39c050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 --2- 172.21.15.45:0/1598743378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd391860 0x5615cd3992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f214d7fa700 0 client.25873 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 >> 172.21.15.45:0/1598743378 conn(0x5615cd306200 msgr2=0x5615cd38cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 shutdown_connections 2023-09-23T09:35:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.768+0000 7f217800eb80 1 -- 172.21.15.45:0/1598743378 wait complete. 2023-09-23T09:35:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.769+0000 7f217800eb80 20 client.25873 trim_cache size 0 max 16384 2023-09-23T09:35:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.793+0000 7f217800eb80 1 Processor -- start 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f217800eb80 1 -- start start 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f217800eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd3a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f217800eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f217800eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd39c710 0x5615cd394630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f217800eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd1658c0 con 0x5615cd390f40 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f217800eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd39a8b0 con 0x5615cd391860 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f217800eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd389ba0 con 0x5615cd39c710 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f21693bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd3a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f21693bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd3a31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34662/0 (socket says 172.21.15.45:34662) 2023-09-23T09:35:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f21693bf700 1 -- 172.21.15.45:0/3045124534 learned_addr learned my addr 172.21.15.45:0/3045124534 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f2169bc0700 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f21693bf700 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd39c710 0x5615cd394630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.794+0000 7f2169bc0700 1 -- 172.21.15.45:0/3045124534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd39c710 msgr2=0x5615cd394630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2169bc0700 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd39c710 0x5615cd394630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2169bc0700 1 -- 172.21.15.45:0/3045124534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 msgr2=0x5615cd3a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2169bc0700 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd3a31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2169bc0700 1 -- 172.21.15.45:0/3045124534 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615cd0f8c00 con 0x5615cd391860 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2169bc0700 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39bbf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5615cd3ac270 tx=0x7f2154014f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a911bc7cf6009d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2158ff9700 1 -- 172.21.15.45:0/3045124534 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154009da0 con 0x5615cd391860 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2158ff9700 1 -- 172.21.15.45:0/3045124534 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2154014530 con 0x5615cd391860 2023-09-23T09:35:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.795+0000 7f2158ff9700 1 -- 172.21.15.45:0/3045124534 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154013cd0 con 0x5615cd391860 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 -- 172.21.15.45:0/3045124534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 msgr2=0x5615cd39bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39bbf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5615cd3ac270 tx=0x7f2154014f20 comp rx=0 tx=0).stop 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 -- 172.21.15.45:0/3045124534 shutdown_connections 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd39c710 0x5615cd394630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd3a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 --2- 172.21.15.45:0/3045124534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd391860 0x5615cd39bbf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 -- 172.21.15.45:0/3045124534 >> 172.21.15.45:0/3045124534 conn(0x5615cd306200 msgr2=0x5615cd3a8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 -- 172.21.15.45:0/3045124534 shutdown_connections 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.796+0000 7f217800eb80 1 -- 172.21.15.45:0/3045124534 wait complete. 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 -- start start 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 Processor -- start 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd396790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd391860 0x5615cd396cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd395550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd395a90 con 0x5615cd390f40 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd395bd0 con 0x5615cd39c710 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615cd395d90 con 0x5615cd391860 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.797+0000 7f217800eb80 10 client.0 _ll_register_callbacks cb 0x7f215c007a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd391860 0x5615cd396cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd391860 0x5615cd396cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36774/0 (socket says 172.21.15.45:36774) 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 -- 172.21.15.45:0/746966024 learned_addr learned my addr 172.21.15.45:0/746966024 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f21693bf700 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd395550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 -- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 msgr2=0x5615cd395550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd395550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 -- 172.21.15.45:0/746966024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 msgr2=0x5615cd396790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd396790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 -- 172.21.15.45:0/746966024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154014a00 con 0x5615cd391860 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f2169bc0700 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd391860 0x5615cd396cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604e0574c359dc34 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f214d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f214d7fa700 1 -- 172.21.15.45:0/746966024 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154013610 con 0x5615cd391860 2023-09-23T09:35:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f214d7fa700 1 -- 172.21.15.45:0/746966024 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2154025420 con 0x5615cd391860 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.798+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f215c024020 con 0x5615cd391860 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f214d7fa700 1 -- 172.21.15.45:0/746966024 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2154005a90 con 0x5615cd391860 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 10 client.25879 fetch_fsmap learned FSMap version 35 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5615cd1361f0 con 0x5615cd391860 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f214d7fa700 1 -- 172.21.15.45:0/746966024 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f215c024020 con 0x5615cd391860 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f214d7fa700 1 -- 172.21.15.45:0/746966024 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 35) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2154006020 con 0x5615cd391860 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 10 client.25879 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 20 client.25879 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd400be0 0x5615cd403000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 4 client.25879 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:35:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615cd2ddad0 con 0x5615cd400be0 2023-09-23T09:35:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.799+0000 7f21693bf700 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd400be0 0x5615cd403000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.800+0000 7f214d7fa700 10 client.25879 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.800+0000 7f21693bf700 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd400be0 0x5615cd403000 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cbccc840b3c61f0 server_cookie=df832ce0bbe591cc in_seq=0 out_seq=0 2023-09-23T09:35:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.825+0000 7f214d7fa700 1 -- 172.21.15.45:0/746966024 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615cd2ddad0 con 0x5615cd400be0 2023-09-23T09:35:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.825+0000 7f214d7fa700 10 client.25879 handle_command_reply: tid=0 2023-09-23T09:35:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.825+0000 7f217800eb80 1 client.25879 shutdown 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.825+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd391860 msgr2=0x5615cd396cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.825+0000 7f217800eb80 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd391860 0x5615cd396cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154009c30 comp rx=0 tx=0).stop 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 shutdown_connections 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5615cd400be0 0x5615cd403000 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615cd39c710 0x5615cd395550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f214d7fa700 0 client.25879 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5615cd390f40 0x5615cd396790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 --2- 172.21.15.45:0/746966024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615cd391860 0x5615cd396cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 >> 172.21.15.45:0/746966024 conn(0x5615cd306200 msgr2=0x5615cd394630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 shutdown_connections 2023-09-23T09:35:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 1 -- 172.21.15.45:0/746966024 wait complete. 2023-09-23T09:35:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.826+0000 7f217800eb80 20 client.25879 trim_cache size 0 max 16384 2023-09-23T09:35:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:35:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:35:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:35:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f21500a17d0 msgr2=0x7f21500a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f21500a17d0 0x7f21500a3c90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21600961f0 tx=0x7f2160004750 comp rx=0 tx=0).stop 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2164117880 msgr2=0x7f21641bacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2164117880 0x7f21641bacf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f216000b550 tx=0x7f216000b910 comp rx=0 tx=0).stop 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 shutdown_connections 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2164117880 0x7f21641bacf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f216410c1a0 0x7f2164116e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f21500a17d0 0x7f21500a3c90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 --2- 172.21.15.45:0/1290737618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21641117a0 0x7f2164117340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.827+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 >> 172.21.15.45:0/1290737618 conn(0x7f216406bbb0 msgr2=0x7f216410f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:21.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:21 smithi045 ceph-mon[98627]: pgmap v181: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 589 op/s 2023-09-23T09:35:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.831+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 shutdown_connections 2023-09-23T09:35:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:21.831+0000 7f216be24700 1 -- 172.21.15.45:0/1290737618 wait complete. 2023-09-23T09:35:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:21 smithi123 ceph-mon[103577]: pgmap v181: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 589 op/s 2023-09-23T09:35:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 335.0 s, 2013.6 tps, lat 49.195 ms stddev 58.272 2023-09-23T09:35:22.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:21 smithi098 ceph-mon[102386]: pgmap v181: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 589 op/s 2023-09-23T09:35:23.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:23 smithi123 ceph-mon[103577]: pgmap v182: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 589 op/s 2023-09-23T09:35:24.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:23 smithi098 ceph-mon[102386]: pgmap v182: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 589 op/s 2023-09-23T09:35:24.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:23 smithi045 ceph-mon[98627]: pgmap v182: 97 pgs: 97 active+clean; 7.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 589 op/s 2023-09-23T09:35:25.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:25 smithi123 ceph-mon[103577]: pgmap v183: 97 pgs: 97 active+clean; 7.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2023-09-23T09:35:25.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:25 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:26.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:25 smithi098 ceph-mon[102386]: pgmap v183: 97 pgs: 97 active+clean; 7.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2023-09-23T09:35:26.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:25 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:26.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:25 smithi045 ceph-mon[98627]: pgmap v183: 97 pgs: 97 active+clean; 7.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2023-09-23T09:35:26.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:25 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 340.0 s, 1278.0 tps, lat 78.477 ms stddev 143.402 2023-09-23T09:35:28.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:27 smithi098 ceph-mon[102386]: pgmap v184: 97 pgs: 97 active+clean; 7.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2023-09-23T09:35:28.088 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:27 smithi045 ceph-mon[98627]: pgmap v184: 97 pgs: 97 active+clean; 7.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2023-09-23T09:35:28.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:27 smithi123 ceph-mon[103577]: pgmap v184: 97 pgs: 97 active+clean; 7.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2023-09-23T09:35:30.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:29 smithi098 ceph-mon[102386]: pgmap v185: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 31 MiB/s wr, 2.07k op/s 2023-09-23T09:35:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:29 smithi045 ceph-mon[98627]: pgmap v185: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 31 MiB/s wr, 2.07k op/s 2023-09-23T09:35:30.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:29 smithi123 ceph-mon[103577]: pgmap v185: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 31 MiB/s wr, 2.07k op/s 2023-09-23T09:35:30.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:35:30.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.643+0000 7f519e9d3700 1 -- 172.21.15.45:0/819864749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 msgr2=0x7f519810d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.643+0000 7f519e9d3700 1 --2- 172.21.15.45:0/819864749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f519810d060 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f519000b0a0 tx=0x7f519000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.644+0000 7f519e9d3700 1 -- 172.21.15.45:0/819864749 shutdown_connections 2023-09-23T09:35:30.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.644+0000 7f519e9d3700 1 --2- 172.21.15.45:0/819864749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 0x7f519811c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.644+0000 7f519e9d3700 1 --2- 172.21.15.45:0/819864749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f519810d060 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.644+0000 7f519e9d3700 1 --2- 172.21.15.45:0/819864749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f519810f940 0x7f519810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.644+0000 7f519e9d3700 1 -- 172.21.15.45:0/819864749 >> 172.21.15.45:0/819864749 conn(0x7f519806d0e0 msgr2=0x7f519806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.644+0000 7f519e9d3700 1 -- 172.21.15.45:0/819864749 shutdown_connections 2023-09-23T09:35:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.644+0000 7f519e9d3700 1 -- 172.21.15.45:0/819864749 wait complete. 2023-09-23T09:35:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 Processor -- start 2023-09-23T09:35:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 -- start start 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f51981a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f519810f940 0x7f51981a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 0x7f51981aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981aae30 con 0x7f519810f4d0 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981aafa0 con 0x7f519811a600 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519e9d3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981ab160 con 0x7f519810f940 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f5197fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f51981a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519cf70700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 0x7f51981aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f5197fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f51981a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34682/0 (socket says 172.21.15.45:34682) 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f5197fff700 1 -- 172.21.15.45:0/51640862 learned_addr learned my addr 172.21.15.45:0/51640862 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.645+0000 7f519cf70700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 0x7f51981aa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56034/0 (socket says 172.21.15.45:56034) 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.646+0000 7f519cf70700 1 -- 172.21.15.45:0/51640862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f519810f940 msgr2=0x7f51981a6390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.646+0000 7f519cf70700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f519810f940 0x7f51981a6390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.646+0000 7f519cf70700 1 -- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 msgr2=0x7f51981a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.646+0000 7f519cf70700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f51981a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.646+0000 7f519cf70700 1 -- 172.21.15.45:0/51640862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190009d00 con 0x7f519811a600 2023-09-23T09:35:30.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.646+0000 7f5197fff700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f51981a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:30.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.660+0000 7f519cf70700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 0x7f51981aa7f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f518000e940 tx=0x7f518000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:30.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.664+0000 7f51957fa700 1 -- 172.21.15.45:0/51640862 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180004220 con 0x7f519811a600 2023-09-23T09:35:30.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.664+0000 7f51957fa700 1 -- 172.21.15.45:0/51640862 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51800095c0 con 0x7f519811a600 2023-09-23T09:35:30.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.664+0000 7f51957fa700 1 -- 172.21.15.45:0/51640862 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000cda0 con 0x7f519811a600 2023-09-23T09:35:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.667+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51981ab440 con 0x7f519811a600 2023-09-23T09:35:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.667+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51981b2c80 con 0x7f519811a600 2023-09-23T09:35:30.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.673+0000 7f51aabbdb80 1 Processor -- start 2023-09-23T09:35:30.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.676+0000 7f51957fa700 1 -- 172.21.15.45:0/51640862 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f5180004750 con 0x7f519811a600 2023-09-23T09:35:30.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.677+0000 7f51957fa700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51840a1860 0x7f51840a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.677+0000 7f51957fa700 1 -- 172.21.15.45:0/51640862 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5180014070 con 0x7f519811a600 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51aabbdb80 1 -- start start 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51977fe700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51840a1860 0x7f51840a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4ac297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29d20 0x559f4ac456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 0x559f4ac47fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab92460 con 0x559f4ac45be0 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab83c20 con 0x559f4ac29400 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab81540 con 0x559f4ac29d20 2023-09-23T09:35:30.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f5197fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4ac297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f5197fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4ac297e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56038/0 (socket says 172.21.15.45:56038) 2023-09-23T09:35:30.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.678+0000 7f5197fff700 1 -- 172.21.15.45:0/345289281 learned_addr learned my addr 172.21.15.45:0/345289281 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:30.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.679+0000 7f5197fff700 1 -- 172.21.15.45:0/345289281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29d20 msgr2=0x559f4ac456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.679+0000 7f5197fff700 1 --2- 172.21.15.45:0/345289281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29d20 0x559f4ac456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.679+0000 7f5197fff700 1 -- 172.21.15.45:0/345289281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 msgr2=0x559f4ac47fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.679+0000 7f5197fff700 1 --2- 172.21.15.45:0/345289281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 0x559f4ac47fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.679+0000 7f5197fff700 1 -- 172.21.15.45:0/345289281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f4abdf8d0 con 0x559f4ac29400 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.679+0000 7f5197fff700 1 --2- 172.21.15.45:0/345289281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4ac297e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5190009c60 tx=0x7f5190009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24dd709ff8a8ae8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.680+0000 7f518e7fc700 1 -- 172.21.15.45:0/345289281 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000e040 con 0x559f4ac29400 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.680+0000 7f51977fe700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51840a1860 0x7f51840a3d20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f51981a7460 tx=0x7f5188009670 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.681+0000 7f518e7fc700 1 -- 172.21.15.45:0/345289281 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519000bcb0 con 0x559f4ac29400 2023-09-23T09:35:30.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.681+0000 7f518e7fc700 1 -- 172.21.15.45:0/345289281 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190005830 con 0x559f4ac29400 2023-09-23T09:35:30.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.681+0000 7f51aabbdb80 1 -- 172.21.15.45:0/345289281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 msgr2=0x559f4ac297e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.681+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/345289281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4ac297e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5190009c60 tx=0x7f5190009c90 comp rx=0 tx=0).stop 2023-09-23T09:35:30.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.682+0000 7f51aabbdb80 1 -- 172.21.15.45:0/345289281 shutdown_connections 2023-09-23T09:35:30.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.682+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/345289281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 0x559f4ac47fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.682+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/345289281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4ac297e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.682+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/345289281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29d20 0x559f4ac456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.682+0000 7f51aabbdb80 1 -- 172.21.15.45:0/345289281 >> 172.21.15.45:0/345289281 conn(0x559f4ab9e260 msgr2=0x559f4ac25790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:30.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.686+0000 7f51aabbdb80 1 -- 172.21.15.45:0/345289281 shutdown_connections 2023-09-23T09:35:30.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.686+0000 7f51aabbdb80 1 -- 172.21.15.45:0/345289281 wait complete. 2023-09-23T09:35:30.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.686+0000 7f51aabbdb80 1 -- start start 2023-09-23T09:35:30.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 1 Processor -- start 2023-09-23T09:35:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac2f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29d20 0x559f4ac2f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 0x559f4ac2fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab7ce30 con 0x559f4ac45be0 2023-09-23T09:35:30.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab19710 con 0x559f4ac29d20 2023-09-23T09:35:30.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab193b0 con 0x559f4ac29400 2023-09-23T09:35:30.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51aabbdb80 10 client.0 _ll_register_callbacks cb 0x559f4a976800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:30.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.687+0000 7f51977fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29d20 0x559f4ac2f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29d20 0x559f4ac2f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56046/0 (socket says 172.21.15.45:56046) 2023-09-23T09:35:30.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 -- 172.21.15.45:0/4052010522 learned_addr learned my addr 172.21.15.45:0/4052010522 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:30.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f5197fff700 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac2f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 -- 172.21.15.45:0/4052010522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 msgr2=0x559f4ac2f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:30.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac2f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 -- 172.21.15.45:0/4052010522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 msgr2=0x559f4ac2fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:30.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 0x559f4ac2fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 -- 172.21.15.45:0/4052010522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519811d2f0 con 0x559f4ac29d20 2023-09-23T09:35:30.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f5197fff700 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac2f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51977fe700 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29d20 0x559f4ac2f920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f51880124d0 tx=0x7f51880047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc69208e42298cf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51777fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51777fe700 1 -- 172.21.15.45:0/4052010522 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518801f800 con 0x559f4ac29d20 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51777fe700 1 -- 172.21.15.45:0/4052010522 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f518801a040 con 0x559f4ac29d20 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f4ac1d250 con 0x559f4ac29d20 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51777fe700 1 -- 172.21.15.45:0/4052010522 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5188005c60 con 0x559f4ac29d20 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51aabbdb80 10 client.25904 fetch_fsmap learned FSMap version 36 2023-09-23T09:35:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.688+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559f4a9a7de0 con 0x559f4ac29d20 2023-09-23T09:35:30.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51777fe700 1 -- 172.21.15.45:0/4052010522 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f4ac1d250 con 0x559f4ac29d20 2023-09-23T09:35:30.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51777fe700 1 -- 172.21.15.45:0/4052010522 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5188011990 con 0x559f4ac29d20 2023-09-23T09:35:30.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51aabbdb80 10 client.25904 fetch_fsmap finished waiting for FSMap version 36 2023-09-23T09:35:30.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51aabbdb80 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:30.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51aabbdb80 20 client.25904 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:30.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4ac556f0 0x559f4ac57b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51aabbdb80 4 client.25904 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:35:30.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f4ab75e10 con 0x559f4ac556f0 2023-09-23T09:35:30.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f5197fff700 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4ac556f0 0x559f4ac57b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.691+0000 7f51777fe700 10 client.25904 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:30.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.692+0000 7f5197fff700 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4ac556f0 0x559f4ac57b30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1145838c90693f7b server_cookie=d9a8b3fa3ee0a5aa in_seq=0 out_seq=0 2023-09-23T09:35:30.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.692+0000 7f51777fe700 1 -- 172.21.15.45:0/4052010522 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559f4ab75e10 con 0x559f4ac556f0 2023-09-23T09:35:30.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.692+0000 7f51777fe700 10 client.25904 handle_command_reply: tid=0 2023-09-23T09:35:30.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.692+0000 7f51aabbdb80 1 client.25904 shutdown 2023-09-23T09:35:30.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29d20 msgr2=0x559f4ac2f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29d20 0x559f4ac2f920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f51880124d0 tx=0x7f51880047c0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 shutdown_connections 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4ac556f0 0x559f4ac57b30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac45be0 0x559f4ac2fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51777fe700 0 client.25904 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac2f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/4052010522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29d20 0x559f4ac2f920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 >> 172.21.15.45:0/4052010522 conn(0x559f4ab9e260 msgr2=0x559f4ac25320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 shutdown_connections 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 1 -- 172.21.15.45:0/4052010522 wait complete. 2023-09-23T09:35:30.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.693+0000 7f51aabbdb80 20 client.25904 trim_cache size 0 max 16384 2023-09-23T09:35:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 Processor -- start 2023-09-23T09:35:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 -- start start 2023-09-23T09:35:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac4c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4ac47540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac45be0 0x559f4ac47a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4a9fd920 con 0x559f4ac29d20 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ac51fe0 con 0x559f4ac45be0 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ac4cc10 con 0x559f4ac29400 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51977fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4ac47540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f5197fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac4c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51977fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4ac47540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34702/0 (socket says 172.21.15.45:34702) 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f5197fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac4c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36808/0 (socket says 172.21.15.45:36808) 2023-09-23T09:35:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f51977fe700 1 -- 172.21.15.45:0/3654622524 learned_addr learned my addr 172.21.15.45:0/3654622524 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.718+0000 7f519cf70700 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac45be0 0x559f4ac47a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f519cf70700 1 -- 172.21.15.45:0/3654622524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 msgr2=0x559f4ac4c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f519cf70700 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac4c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f519cf70700 1 -- 172.21.15.45:0/3654622524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 msgr2=0x559f4ac47540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f519cf70700 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4ac47540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f519cf70700 1 -- 172.21.15.45:0/3654622524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f4a990c60 con 0x559f4ac45be0 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f5197fff700 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac4c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f519cf70700 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac45be0 0x559f4ac47a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51800caca0 tx=0x7f5180096910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b5cb5d059da34a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f518effd700 1 -- 172.21.15.45:0/3654622524 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180004380 con 0x559f4ac45be0 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f518effd700 1 -- 172.21.15.45:0/3654622524 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518009abc0 con 0x559f4ac45be0 2023-09-23T09:35:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.719+0000 7f518effd700 1 -- 172.21.15.45:0/3654622524 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800999b0 con 0x559f4ac45be0 2023-09-23T09:35:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 -- 172.21.15.45:0/3654622524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac45be0 msgr2=0x559f4ac47a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac45be0 0x559f4ac47a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51800caca0 tx=0x7f5180096910 comp rx=0 tx=0).stop 2023-09-23T09:35:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 -- 172.21.15.45:0/3654622524 shutdown_connections 2023-09-23T09:35:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac45be0 0x559f4ac47a80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac29400 0x559f4ac4c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/3654622524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4ac47540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 -- 172.21.15.45:0/3654622524 >> 172.21.15.45:0/3654622524 conn(0x559f4ab9e260 msgr2=0x559f4ac465c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 -- 172.21.15.45:0/3654622524 shutdown_connections 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 -- 172.21.15.45:0/3654622524 wait complete. 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.720+0000 7f51aabbdb80 1 -- start start 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.721+0000 7f51aabbdb80 1 Processor -- start 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4aca2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4aca2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51aabbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac45be0 0x559f4ac9d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ac9daa0 con 0x559f4ac29d20 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ac9dc10 con 0x559f4ac29400 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51aabbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ac9ddd0 con 0x559f4ac45be0 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51aabbdb80 10 client.0 _ll_register_callbacks cb 0x7f5188010c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51977fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4aca2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51977fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4aca2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34706/0 (socket says 172.21.15.45:34706) 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51977fe700 1 -- 172.21.15.45:0/439887773 learned_addr learned my addr 172.21.15.45:0/439887773 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.722+0000 7f51977fe700 1 -- 172.21.15.45:0/439887773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac45be0 msgr2=0x559f4ac9d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51977fe700 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac45be0 0x559f4ac9d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51977fe700 1 -- 172.21.15.45:0/439887773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 msgr2=0x559f4aca2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51977fe700 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4aca2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51977fe700 1 -- 172.21.15.45:0/439887773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f4abdf8d0 con 0x559f4ac29d20 2023-09-23T09:35:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51977fe700 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4aca2fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f51880124d0 tx=0x7f5188012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc7684ab205026d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51777fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51777fe700 1 -- 172.21.15.45:0/439887773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188027470 con 0x559f4ac29d20 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51777fe700 1 -- 172.21.15.45:0/439887773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f51880115f0 con 0x559f4ac29d20 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f519000bcb0 con 0x559f4ac29d20 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51777fe700 1 -- 172.21.15.45:0/439887773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51880117b0 con 0x559f4ac29d20 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51aabbdb80 10 client.16200 fetch_fsmap learned FSMap version 36 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f51981b2c80 con 0x559f4ac29d20 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.723+0000 7f51777fe700 1 -- 172.21.15.45:0/439887773 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519000bcb0 con 0x559f4ac29d20 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.724+0000 7f51777fe700 1 -- 172.21.15.45:0/439887773 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f51880050b0 con 0x559f4ac29d20 2023-09-23T09:35:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.724+0000 7f51aabbdb80 10 client.16200 fetch_fsmap finished waiting for FSMap version 36 2023-09-23T09:35:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.724+0000 7f51aabbdb80 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.724+0000 7f51aabbdb80 20 client.16200 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.724+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4acaa8e0 0x559f4acacda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.724+0000 7f51aabbdb80 4 client.16200 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:35:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.724+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f4ab75b30 con 0x559f4acaa8e0 2023-09-23T09:35:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.725+0000 7f5197fff700 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4acaa8e0 0x559f4acacda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.725+0000 7f51777fe700 10 client.16200 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.725+0000 7f5197fff700 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4acaa8e0 0x559f4acacda0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56af9b0aeb48772 server_cookie=f90b99a9ed003abc in_seq=0 out_seq=0 2023-09-23T09:35:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.725+0000 7f51777fe700 1 -- 172.21.15.45:0/439887773 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559f4ab75b30 con 0x559f4acaa8e0 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51777fe700 10 client.16200 handle_command_reply: tid=0 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 client.16200 shutdown 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 msgr2=0x559f4aca2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4aca2fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f51880124d0 tx=0x7f5188012f40 comp rx=0 tx=0).stop 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 shutdown_connections 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559f4acaa8e0 0x559f4acacda0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f4ac45be0 0x559f4ac9d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f4ac29400 0x559f4aca2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 --2- 172.21.15.45:0/439887773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f4ac29d20 0x559f4aca2fa0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51777fe700 0 client.16200 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 >> 172.21.15.45:0/439887773 conn(0x559f4ab9e260 msgr2=0x559f4ac47540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 shutdown_connections 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.726+0000 7f51aabbdb80 1 -- 172.21.15.45:0/439887773 wait complete. 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.727+0000 7f51aabbdb80 20 client.16200 trim_cache size 0 max 16384 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:35:30.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:35:30.734 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:35:30.734 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:35:30.734 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:35:30.734 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51840a1860 msgr2=0x7f51840a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51840a1860 0x7f51840a3d20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f51981a7460 tx=0x7f5188009670 comp rx=0 tx=0).stop 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 msgr2=0x7f51981aa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 0x7f51981aa7f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f518000e940 tx=0x7f518000ec50 comp rx=0 tx=0).stop 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 shutdown_connections 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f519811a600 0x7f51981aa7f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51840a1860 0x7f51840a3d20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f519810f4d0 0x7f51981a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 --2- 172.21.15.45:0/51640862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f519810f940 0x7f51981a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 >> 172.21.15.45:0/51640862 conn(0x7f519806d0e0 msgr2=0x7f51981185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 shutdown_connections 2023-09-23T09:35:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:35:30.730+0000 7f519e9d3700 1 -- 172.21.15.45:0/51640862 wait complete. 2023-09-23T09:35:30.750 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:35:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 345.0 s, 1099.8 tps, lat 81.028 ms stddev 250.883 2023-09-23T09:35:32.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:31 smithi098 ceph-mon[102386]: pgmap v186: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-09-23T09:35:32.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:31 smithi045 ceph-mon[98627]: pgmap v186: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-09-23T09:35:32.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:31 smithi123 ceph-mon[103577]: pgmap v186: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-09-23T09:35:34.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:33 smithi098 ceph-mon[102386]: pgmap v187: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-09-23T09:35:34.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:33 smithi045 ceph-mon[98627]: pgmap v187: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-09-23T09:35:34.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:33 smithi123 ceph-mon[103577]: pgmap v187: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-09-23T09:35:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 -- 172.21.15.45:0/1446236617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51780fff40 msgr2=0x7f5178100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 --2- 172.21.15.45:0/1446236617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51780fff40 0x7f5178100320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5174009990 tx=0x7f5174009ca0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 -- 172.21.15.45:0/1446236617 shutdown_connections 2023-09-23T09:35:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 --2- 172.21.15.45:0/1446236617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5178105060 0x7f517810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 --2- 172.21.15.45:0/1446236617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51780fff40 0x7f5178100320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 --2- 172.21.15.45:0/1446236617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51781008f0 0x7f5178104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 -- 172.21.15.45:0/1446236617 >> 172.21.15.45:0/1446236617 conn(0x7f51780fb760 msgr2=0x7f51780fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 -- 172.21.15.45:0/1446236617 shutdown_connections 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.000+0000 7f517f5cf700 1 -- 172.21.15.45:0/1446236617 wait complete. 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 Processor -- start 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 -- start start 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51780fff40 0x7f5178199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5178105060 0x7f51781996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f517819dd10 0x7f517819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819e790 con 0x7f517819dd10 2023-09-23T09:35:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819e900 con 0x7f5178105060 2023-09-23T09:35:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517f5cf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819eac0 con 0x7f51780fff40 2023-09-23T09:35:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.001+0000 7f517cb6a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5178105060 0x7f51781996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517cb6a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5178105060 0x7f51781996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56068/0 (socket says 172.21.15.45:56068) 2023-09-23T09:35:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517cb6a700 1 -- 172.21.15.45:0/3584800068 learned_addr learned my addr 172.21.15.45:0/3584800068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517d36b700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51780fff40 0x7f5178199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517cb6a700 1 -- 172.21.15.45:0/3584800068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51780fff40 msgr2=0x7f5178199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517cb6a700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51780fff40 0x7f5178199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517cb6a700 1 -- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f517819dd10 msgr2=0x7f517819e190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517cb6a700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f517819dd10 0x7f517819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.002+0000 7f517cb6a700 1 -- 172.21.15.45:0/3584800068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170009c40 con 0x7f5178105060 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.003+0000 7f517cb6a700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5178105060 0x7f51781996a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f516800b500 tx=0x7f516800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.003+0000 7f516e7fc700 1 -- 172.21.15.45:0/3584800068 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168011830 con 0x7f5178105060 2023-09-23T09:35:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.003+0000 7f516e7fc700 1 -- 172.21.15.45:0/3584800068 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5168011e50 con 0x7f5178105060 2023-09-23T09:35:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.003+0000 7f516e7fc700 1 -- 172.21.15.45:0/3584800068 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168010e50 con 0x7f5178105060 2023-09-23T09:35:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.003+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51740095f0 con 0x7f5178105060 2023-09-23T09:35:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.003+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51781a6200 con 0x7f5178105060 2023-09-23T09:35:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.005+0000 7f516e7fc700 1 -- 172.21.15.45:0/3584800068 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f51680104f0 con 0x7f5178105060 2023-09-23T09:35:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.006+0000 7f516e7fc700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51640a1790 0x7f51640a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.006+0000 7f517d36b700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51640a1790 0x7f51640a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.006+0000 7f516e7fc700 1 -- 172.21.15.45:0/3584800068 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f51680cccb0 con 0x7f5178105060 2023-09-23T09:35:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.006+0000 7f517d36b700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51640a1790 0x7f51640a3c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5174006010 tx=0x7f517400b260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 Processor -- start 2023-09-23T09:35:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 -- start start 2023-09-23T09:35:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d8187320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d81921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d8194b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d80f0400 con 0x5611d8187860 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d80e1bc0 con 0x5611d8186f40 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d80df4e0 con 0x5611d8192710 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f517d36b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d8187320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f517cb6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d81921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f517db6c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d8194b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f517cb6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d81921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34726/0 (socket says 172.21.15.45:34726) 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f517d36b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d8187320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56076/0 (socket says 172.21.15.45:56076) 2023-09-23T09:35:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.008+0000 7f517cb6a700 1 -- 172.21.15.45:0/2511245162 learned_addr learned my addr 172.21.15.45:0/2511245162 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.009+0000 7f517db6c700 1 -- 172.21.15.45:0/2511245162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 msgr2=0x5611d8187320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.009+0000 7f517db6c700 1 --2- 172.21.15.45:0/2511245162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d8187320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.009+0000 7f517db6c700 1 -- 172.21.15.45:0/2511245162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 msgr2=0x5611d81921d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.009+0000 7f517db6c700 1 --2- 172.21.15.45:0/2511245162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d81921d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.009+0000 7f517db6c700 1 -- 172.21.15.45:0/2511245162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611d813d870 con 0x5611d8192710 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.009+0000 7f517db6c700 1 --2- 172.21.15.45:0/2511245162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d8194b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f517000e460 tx=0x7f517000e770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263deb34bd428533 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.009+0000 7f51637fe700 1 -- 172.21.15.45:0/2511245162 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700092d0 con 0x5611d8192710 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.010+0000 7f51637fe700 1 -- 172.21.15.45:0/2511245162 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f517000f040 con 0x5611d8192710 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.010+0000 7f51637fe700 1 -- 172.21.15.45:0/2511245162 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170007500 con 0x5611d8192710 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.010+0000 7f518b7b9b80 1 -- 172.21.15.45:0/2511245162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 msgr2=0x5611d8194b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.010+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/2511245162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d8194b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f517000e460 tx=0x7f517000e770 comp rx=0 tx=0).stop 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 -- 172.21.15.45:0/2511245162 shutdown_connections 2023-09-23T09:35:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/2511245162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d8194b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/2511245162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d8187320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/2511245162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d81921d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 -- 172.21.15.45:0/2511245162 >> 172.21.15.45:0/2511245162 conn(0x5611d80fc200 msgr2=0x5611d81833c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 -- 172.21.15.45:0/2511245162 shutdown_connections 2023-09-23T09:35:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 -- 172.21.15.45:0/2511245162 wait complete. 2023-09-23T09:35:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 -- start start 2023-09-23T09:35:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.011+0000 7f518b7b9b80 1 Processor -- start 2023-09-23T09:35:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8186f40 0x5611d81921c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d818f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8192710 0x5611d818fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d80dadd0 con 0x5611d8187860 2023-09-23T09:35:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d80776b0 con 0x5611d8192710 2023-09-23T09:35:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d8077350 con 0x5611d8186f40 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f518b7b9b80 10 client.0 _ll_register_callbacks cb 0x5611d7ed47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8192710 0x5611d818fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8192710 0x5611d818fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56082/0 (socket says 172.21.15.45:56082) 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 -- 172.21.15.45:0/3356464852 learned_addr learned my addr 172.21.15.45:0/3356464852 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517cb6a700 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d818f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517d36b700 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8186f40 0x5611d81921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 -- 172.21.15.45:0/3356464852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8186f40 msgr2=0x5611d81921c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8186f40 0x5611d81921c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 -- 172.21.15.45:0/3356464852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 msgr2=0x5611d818f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d818f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.012+0000 7f517db6c700 1 -- 172.21.15.45:0/3356464852 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517000e140 con 0x5611d8192710 2023-09-23T09:35:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f517db6c700 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8192710 0x5611d818fb80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f517000f010 tx=0x7f5170004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=574df3209ff3490b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f5160ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f5160ff9700 1 -- 172.21.15.45:0/3356464852 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170004540 con 0x5611d8192710 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611d817b090 con 0x5611d8192710 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f5160ff9700 1 -- 172.21.15.45:0/3356464852 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f51700046a0 con 0x5611d8192710 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f5160ff9700 1 -- 172.21.15.45:0/3356464852 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5170004860 con 0x5611d8192710 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f518b7b9b80 10 client.25919 fetch_fsmap learned FSMap version 37 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5611d7f05d80 con 0x5611d8192710 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.013+0000 7f5160ff9700 1 -- 172.21.15.45:0/3356464852 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611d817b090 con 0x5611d8192710 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f5160ff9700 1 -- 172.21.15.45:0/3356464852 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 37) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f517000e490 con 0x5611d8192710 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f518b7b9b80 10 client.25919 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T09:35:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f518b7b9b80 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f518b7b9b80 20 client.25919 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81aa990 0x5611d81acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f518b7b9b80 4 client.25919 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:35:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611d80d3db0 con 0x5611d81aa990 2023-09-23T09:35:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f517d36b700 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81aa990 0x5611d81acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f517d36b700 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81aa990 0x5611d81acdb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165ba3335b00dff9 server_cookie=8eb9c1e74fede15d in_seq=0 out_seq=0 2023-09-23T09:35:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.014+0000 7f5160ff9700 10 client.25919 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f5160ff9700 1 -- 172.21.15.45:0/3356464852 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611d80d3db0 con 0x5611d81aa990 2023-09-23T09:35:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f5160ff9700 10 client.25919 handle_command_reply: tid=0 2023-09-23T09:35:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 client.25919 shutdown 2023-09-23T09:35:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8192710 msgr2=0x5611d818fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8192710 0x5611d818fb80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f517000f010 tx=0x7f5170004230 comp rx=0 tx=0).stop 2023-09-23T09:35:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 shutdown_connections 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81aa990 0x5611d81acdb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8192710 0x5611d818fb80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8186f40 0x5611d81921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f5160ff9700 0 client.25919 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3356464852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8187860 0x5611d818f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 >> 172.21.15.45:0/3356464852 conn(0x5611d80fc200 msgr2=0x5611d8182f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 shutdown_connections 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3356464852 wait complete. 2023-09-23T09:35:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.015+0000 7f518b7b9b80 20 client.25919 trim_cache size 0 max 16384 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 Processor -- start 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 -- start start 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8186f40 0x5611d8189de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8187860 0x5611d818ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d7f5b8c0 con 0x5611d8186f40 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d817fba0 con 0x5611d8187860 2023-09-23T09:35:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.035+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d8190370 con 0x5611d8192710 2023-09-23T09:35:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8186f40 0x5611d8189de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8186f40 0x5611d8189de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34740/0 (socket says 172.21.15.45:34740) 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 -- 172.21.15.45:0/3050616220 learned_addr learned my addr 172.21.15.45:0/3050616220 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517db6c700 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517cb6a700 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8187860 0x5611d818ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 -- 172.21.15.45:0/3050616220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 msgr2=0x5611d819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 -- 172.21.15.45:0/3050616220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8187860 msgr2=0x5611d818ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8187860 0x5611d818ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 -- 172.21.15.45:0/3050616220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611d7eeec00 con 0x5611d8186f40 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.036+0000 7f517d36b700 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8186f40 0x5611d8189de0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5a54ea8b934944 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.037+0000 7f5163fff700 1 -- 172.21.15.45:0/3050616220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174005670 con 0x5611d8186f40 2023-09-23T09:35:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.037+0000 7f5163fff700 1 -- 172.21.15.45:0/3050616220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f517402c040 con 0x5611d8186f40 2023-09-23T09:35:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.037+0000 7f5163fff700 1 -- 172.21.15.45:0/3050616220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174004760 con 0x5611d8186f40 2023-09-23T09:35:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.037+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3050616220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8186f40 msgr2=0x5611d8189de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.037+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8186f40 0x5611d8189de0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174022000 comp rx=0 tx=0).stop 2023-09-23T09:35:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3050616220 shutdown_connections 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8192710 0x5611d819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8186f40 0x5611d8189de0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/3050616220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8187860 0x5611d818ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3050616220 >> 172.21.15.45:0/3050616220 conn(0x5611d80fc200 msgr2=0x5611d81a6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3050616220 shutdown_connections 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 -- 172.21.15.45:0/3050616220 wait complete. 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 -- start start 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 Processor -- start 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.038+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8187860 0x5611d8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f518b7b9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8192710 0x5611d8198890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d8198ea0 con 0x5611d8192710 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d8199010 con 0x5611d8186f40 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f518b7b9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d81991d0 con 0x5611d8187860 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f518b7b9b80 10 client.0 _ll_register_callbacks cb 0x5611d7ed47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8187860 0x5611d8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517db6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8192710 0x5611d8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517d36b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8187860 0x5611d8197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36854/0 (socket says 172.21.15.45:36854) 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 -- 172.21.15.45:0/158300982 learned_addr learned my addr 172.21.15.45:0/158300982 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517db6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8192710 0x5611d8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34750/0 (socket says 172.21.15.45:34750) 2023-09-23T09:35:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 -- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 msgr2=0x5611d81975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d81975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 -- 172.21.15.45:0/158300982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8192710 msgr2=0x5611d8198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8192710 0x5611d8198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 -- 172.21.15.45:0/158300982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51780690b0 con 0x5611d8187860 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517d36b700 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d81975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517db6c700 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8192710 0x5611d8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f517cb6a700 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8187860 0x5611d8197b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f516800bfd0 tx=0x7f5168004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62900a5f32f349ce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f5160ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.039+0000 7f5160ff9700 1 -- 172.21.15.45:0/158300982 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680b2290 con 0x5611d8187860 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f5160ff9700 1 -- 172.21.15.45:0/158300982 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f51680b1070 con 0x5611d8187860 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f517000f040 con 0x5611d8187860 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f5160ff9700 1 -- 172.21.15.45:0/158300982 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51680b23f0 con 0x5611d8187860 2023-09-23T09:35:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 10 client.25906 fetch_fsmap learned FSMap version 37 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5611d7f2c1f0 con 0x5611d8187860 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f5160ff9700 1 -- 172.21.15.45:0/158300982 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f517000f040 con 0x5611d8187860 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f5160ff9700 1 -- 172.21.15.45:0/158300982 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f51680992a0 con 0x5611d8187860 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 20 client.25906 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81ff9f0 0x5611d8201e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 4 client.25906 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611d80d3ad0 con 0x5611d81ff9f0 2023-09-23T09:35:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.040+0000 7f517d36b700 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81ff9f0 0x5611d8201e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.041+0000 7f517d36b700 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81ff9f0 0x5611d8201e10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf63f40d78e5330 server_cookie=749463efa20a318e in_seq=0 out_seq=0 2023-09-23T09:35:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.041+0000 7f5160ff9700 10 client.25906 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.059+0000 7f5160ff9700 1 -- 172.21.15.45:0/158300982 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611d80d3ad0 con 0x5611d81ff9f0 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.059+0000 7f5160ff9700 10 client.25906 handle_command_reply: tid=0 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.059+0000 7f518b7b9b80 1 client.25906 shutdown 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.059+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8187860 msgr2=0x5611d8197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.059+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8187860 0x5611d8197b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f516800bfd0 tx=0x7f5168004860 comp rx=0 tx=0).stop 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.059+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 shutdown_connections 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.059+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5611d81ff9f0 0x5611d8201e10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611d8192710 0x5611d8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611d8186f40 0x5611d81975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f518b7b9b80 1 --2- 172.21.15.45:0/158300982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611d8187860 0x5611d8197b00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 >> 172.21.15.45:0/158300982 conn(0x5611d80fc200 msgr2=0x5611d819d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f5160ff9700 0 client.25906 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 shutdown_connections 2023-09-23T09:35:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f518b7b9b80 1 -- 172.21.15.45:0/158300982 wait complete. 2023-09-23T09:35:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.060+0000 7f518b7b9b80 20 client.25906 trim_cache size 0 max 16384 2023-09-23T09:35:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:35:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:35:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:35:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51640a1790 msgr2=0x7f51640a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51640a1790 0x7f51640a3c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5174006010 tx=0x7f517400b260 comp rx=0 tx=0).stop 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5178105060 msgr2=0x7f51781996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5178105060 0x7f51781996a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f516800b500 tx=0x7f516800b8c0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 shutdown_connections 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f51640a1790 0x7f51640a3c50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f517819dd10 0x7f517819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51780fff40 0x7f5178199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 --2- 172.21.15.45:0/3584800068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5178105060 0x7f51781996a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.061+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 >> 172.21.15.45:0/3584800068 conn(0x7f51780fb760 msgr2=0x7f51780fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.062+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 shutdown_connections 2023-09-23T09:35:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:35.062+0000 7f517f5cf700 1 -- 172.21.15.45:0/3584800068 wait complete. 2023-09-23T09:35:36.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:35 smithi098 ceph-mon[102386]: pgmap v188: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 37 MiB/s wr, 2.49k op/s 2023-09-23T09:35:36.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:35 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:36.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:35 smithi045 ceph-mon[98627]: pgmap v188: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 37 MiB/s wr, 2.49k op/s 2023-09-23T09:35:36.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:35 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:36.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:35 smithi123 ceph-mon[103577]: pgmap v188: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 37 MiB/s wr, 2.49k op/s 2023-09-23T09:35:36.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:35 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:36.897 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 350.0 s, 871.3 tps, lat 128.089 ms stddev 511.943 2023-09-23T09:35:38.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:37 smithi098 ceph-mon[102386]: pgmap v189: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-23T09:35:38.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:37 smithi045 ceph-mon[98627]: pgmap v189: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-23T09:35:38.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:37 smithi123 ceph-mon[103577]: pgmap v189: 97 pgs: 97 active+clean; 7.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-23T09:35:39.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:38 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:35:39.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:38 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:35:39.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:38 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:35:40.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:39 smithi098 ceph-mon[102386]: pgmap v190: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-09-23T09:35:40.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:39 smithi045 ceph-mon[98627]: pgmap v190: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-09-23T09:35:40.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:39 smithi123 ceph-mon[103577]: pgmap v190: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-09-23T09:35:41.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:40 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:41.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:40 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:41.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:40 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:35:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 355.0 s, 1377.9 tps, lat 72.652 ms stddev 191.054 2023-09-23T09:35:42.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:41 smithi098 ceph-mon[102386]: pgmap v191: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-09-23T09:35:42.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:41 smithi045 ceph-mon[98627]: pgmap v191: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-09-23T09:35:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:41 smithi123 ceph-mon[103577]: pgmap v191: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-09-23T09:35:43.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:42 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:35:43.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:42 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:35:43.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:42 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:35:43.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:42 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:35:43.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:42 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:35:43.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:42 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:35:43.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:42 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:35:43.204 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:42 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:35:43.204 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:42 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:35:44.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:43 smithi098 ceph-mon[102386]: pgmap v192: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 807 op/s 2023-09-23T09:35:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:43 smithi045 ceph-mon[98627]: pgmap v192: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 807 op/s 2023-09-23T09:35:44.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:43 smithi123 ceph-mon[103577]: pgmap v192: 97 pgs: 97 active+clean; 7.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 807 op/s 2023-09-23T09:35:46.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:45 smithi098 ceph-mon[102386]: pgmap v193: 97 pgs: 97 active+clean; 7.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-09-23T09:35:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:45 smithi045 ceph-mon[98627]: pgmap v193: 97 pgs: 97 active+clean; 7.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-09-23T09:35:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:45 smithi123 ceph-mon[103577]: pgmap v193: 97 pgs: 97 active+clean; 7.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-09-23T09:35:46.886 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 360.0 s, 680.5 tps, lat 47.276 ms stddev 56.070 2023-09-23T09:35:48.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:47 smithi098 ceph-mon[102386]: pgmap v194: 97 pgs: 97 active+clean; 7.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 944 op/s 2023-09-23T09:35:48.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:47 smithi045 ceph-mon[98627]: pgmap v194: 97 pgs: 97 active+clean; 7.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 944 op/s 2023-09-23T09:35:48.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:47 smithi123 ceph-mon[103577]: pgmap v194: 97 pgs: 97 active+clean; 7.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 944 op/s 2023-09-23T09:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 -- 172.21.15.45:0/159785442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2268108090 msgr2=0x7f2268108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 --2- 172.21.15.45:0/159785442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2268108090 0x7f2268108470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f225c007510 tx=0x7f225c00af90 comp rx=0 tx=0).stop 2023-09-23T09:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 -- 172.21.15.45:0/159785442 shutdown_connections 2023-09-23T09:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 --2- 172.21.15.45:0/159785442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 0x7f2268114730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 --2- 172.21.15.45:0/159785442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2268108090 0x7f2268108470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 --2- 172.21.15.45:0/159785442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2268108a40 0x7f226810c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 -- 172.21.15.45:0/159785442 >> 172.21.15.45:0/159785442 conn(0x7f226806d320 msgr2=0x7f226806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 -- 172.21.15.45:0/159785442 shutdown_connections 2023-09-23T09:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.251+0000 7f226d9ac700 1 -- 172.21.15.45:0/159785442 wait complete. 2023-09-23T09:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.252+0000 7f226d9ac700 1 Processor -- start 2023-09-23T09:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.252+0000 7f226d9ac700 1 -- start start 2023-09-23T09:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f226d9ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2268108090 0x7f226819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f226d9ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 0x7f226819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f226d9ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226819dd20 0x7f22681b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f226d9ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22681b25d0 con 0x7f226810cb20 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f226d9ac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22681b2710 con 0x7f2268108090 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f226d9ac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22681b2880 con 0x7f226819dd20 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f22667fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 0x7f226819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f22677fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226819dd20 0x7f22681b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f22667fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 0x7f226819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34762/0 (socket says 172.21.15.45:34762) 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f22667fc700 1 -- 172.21.15.45:0/1730896057 learned_addr learned my addr 172.21.15.45:0/1730896057 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.253+0000 7f22667fc700 1 -- 172.21.15.45:0/1730896057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226819dd20 msgr2=0x7f22681b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.254+0000 7f22667fc700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226819dd20 0x7f22681b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.254+0000 7f22667fc700 1 -- 172.21.15.45:0/1730896057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2268108090 msgr2=0x7f226819d2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.254+0000 7f22667fc700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2268108090 0x7f226819d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.254+0000 7f22667fc700 1 -- 172.21.15.45:0/1730896057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c0071c0 con 0x7f226810cb20 2023-09-23T09:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.254+0000 7f22677fe700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226819dd20 0x7f22681b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.259+0000 7f22667fc700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 0x7f226819d7e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f225800b550 tx=0x7f225800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.259+0000 7f226c9aa700 1 -- 172.21.15.45:0/1730896057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258011830 con 0x7f226810cb20 2023-09-23T09:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.259+0000 7f226c9aa700 1 -- 172.21.15.45:0/1730896057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2258011e50 con 0x7f226810cb20 2023-09-23T09:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.259+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22681b2b60 con 0x7f226810cb20 2023-09-23T09:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.259+0000 7f226c9aa700 1 -- 172.21.15.45:0/1730896057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258010e70 con 0x7f226810cb20 2023-09-23T09:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.259+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22681b30c0 con 0x7f226810cb20 2023-09-23T09:35:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.263+0000 7f226c9aa700 1 -- 172.21.15.45:0/1730896057 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f2258010510 con 0x7f226810cb20 2023-09-23T09:35:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.264+0000 7f2279b96b80 1 Processor -- start 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.264+0000 7f2279b96b80 1 -- start start 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f226c9aa700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22500a1810 0x7f22500a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bd5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd5860 0x55e3d8be01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8be2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2279b96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8b3e400 con 0x55e3d8bd4f40 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2279b96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8b2fbc0 con 0x55e3d8bd5860 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2279b96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8b2d4e0 con 0x55e3d8be0710 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f226c9aa700 1 -- 172.21.15.45:0/1730896057 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f22580ccc60 con 0x7f226810cb20 2023-09-23T09:35:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bd5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bd5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34766/0 (socket says 172.21.15.45:34766) 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 -- 172.21.15.45:0/3545571972 learned_addr learned my addr 172.21.15.45:0/3545571972 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f22677fe700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22500a1810 0x7f22500a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f22677fe700 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8be2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 -- 172.21.15.45:0/3545571972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 msgr2=0x55e3d8be2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8be2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 -- 172.21.15.45:0/3545571972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd5860 msgr2=0x55e3d8be01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd5860 0x55e3d8be01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.265+0000 7f2266ffd700 1 -- 172.21.15.45:0/3545571972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3d8b8b870 con 0x55e3d8bd4f40 2023-09-23T09:35:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.266+0000 7f22677fe700 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8be2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:35:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.266+0000 7f2266ffd700 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bd5320 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f225c000c00 tx=0x7f225c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3c058f0c359b26 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.266+0000 7f22677fe700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22500a1810 0x7f22500a3cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2260007810 tx=0x7f226000b040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.266+0000 7f224dffb700 1 -- 172.21.15.45:0/3545571972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00f070 con 0x55e3d8bd4f40 2023-09-23T09:35:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.266+0000 7f224dffb700 1 -- 172.21.15.45:0/3545571972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f225c003680 con 0x55e3d8bd4f40 2023-09-23T09:35:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.266+0000 7f224dffb700 1 -- 172.21.15.45:0/3545571972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c003c60 con 0x55e3d8bd4f40 2023-09-23T09:35:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 -- 172.21.15.45:0/3545571972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bd4f40 msgr2=0x55e3d8bd5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bd5320 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f225c000c00 tx=0x7f225c005c30 comp rx=0 tx=0).stop 2023-09-23T09:35:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 -- 172.21.15.45:0/3545571972 shutdown_connections 2023-09-23T09:35:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8be2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bd5320 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3545571972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd5860 0x55e3d8be01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 -- 172.21.15.45:0/3545571972 >> 172.21.15.45:0/3545571972 conn(0x55e3d8b4a200 msgr2=0x55e3d8bd13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.267+0000 7f2279b96b80 1 -- 172.21.15.45:0/3545571972 shutdown_connections 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.268+0000 7f2279b96b80 1 -- 172.21.15.45:0/3545571972 wait complete. 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.268+0000 7f2279b96b80 1 -- start start 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.268+0000 7f2279b96b80 1 Processor -- start 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8bdda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f2279b96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8b28dd0 con 0x55e3d8bddfb0 2023-09-23T09:35:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f2279b96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8ac56b0 con 0x55e3d8bd4f40 2023-09-23T09:35:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f2279b96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8ac5350 con 0x55e3d8be0710 2023-09-23T09:35:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f2279b96b80 10 client.0 _ll_register_callbacks cb 0x55e3d89227a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f22677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f22677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34774/0 (socket says 172.21.15.45:34774) 2023-09-23T09:35:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.269+0000 7f22677fe700 1 -- 172.21.15.45:0/2580148913 learned_addr learned my addr 172.21.15.45:0/2580148913 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f2266ffd700 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f2266ffd700 1 -- 172.21.15.45:0/2580148913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 msgr2=0x55e3d8bdda70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f2266ffd700 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8bdda70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f2266ffd700 1 -- 172.21.15.45:0/2580148913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 msgr2=0x55e3d8bec280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f2266ffd700 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f2266ffd700 1 -- 172.21.15.45:0/2580148913 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2268114f40 con 0x55e3d8bd4f40 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f22677fe700 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.270+0000 7f2266ffd700 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdd530 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c370 tx=0x7f225c00c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e76587f7d808ce8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.271+0000 7f2246ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.271+0000 7f2246ffd700 1 -- 172.21.15.45:0/2580148913 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00f040 con 0x55e3d8bd4f40 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.271+0000 7f2246ffd700 1 -- 172.21.15.45:0/2580148913 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f225c00ce20 con 0x55e3d8bd4f40 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.271+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3d8bc9090 con 0x55e3d8bd4f40 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.271+0000 7f2246ffd700 1 -- 172.21.15.45:0/2580148913 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f225c007e00 con 0x55e3d8bd4f40 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.271+0000 7f2279b96b80 10 client.25931 fetch_fsmap learned FSMap version 38 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.271+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e3d8953d80 con 0x55e3d8bd4f40 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.272+0000 7f2246ffd700 1 -- 172.21.15.45:0/2580148913 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3d8bc9090 con 0x55e3d8bd4f40 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2246ffd700 1 -- 172.21.15.45:0/2580148913 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 38) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f225c00a5b0 con 0x55e3d8bd4f40 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2279b96b80 10 client.25931 fetch_fsmap finished waiting for FSMap version 38 2023-09-23T09:35:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2279b96b80 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2279b96b80 20 client.25931 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2279b96b80 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8bf0290 0x55e3d8bf26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2279b96b80 4 client.25931 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:35:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3d8b21db0 con 0x55e3d8bf0290 2023-09-23T09:35:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f22667fc700 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8bf0290 0x55e3d8bf26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f2246ffd700 10 client.25931 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.273+0000 7f22667fc700 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8bf0290 0x55e3d8bf26b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916a88c68ccb7d95 server_cookie=8f03e4a755242d29 in_seq=0 out_seq=0 2023-09-23T09:35:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.274+0000 7f2246ffd700 1 -- 172.21.15.45:0/2580148913 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3d8b21db0 con 0x55e3d8bf0290 2023-09-23T09:35:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.274+0000 7f2246ffd700 10 client.25931 handle_command_reply: tid=0 2023-09-23T09:35:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.274+0000 7f2279b96b80 1 client.25931 shutdown 2023-09-23T09:35:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.275+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 msgr2=0x55e3d8bdd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.275+0000 7f2279b96b80 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdd530 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c370 tx=0x7f225c00c980 comp rx=0 tx=0).stop 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 shutdown_connections 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8bf0290 0x55e3d8bf26b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdd530 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 --2- 172.21.15.45:0/2580148913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8bdda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 >> 172.21.15.45:0/2580148913 conn(0x55e3d8b4a200 msgr2=0x55e3d8bd0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2246ffd700 0 client.25931 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 shutdown_connections 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 1 -- 172.21.15.45:0/2580148913 wait complete. 2023-09-23T09:35:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.276+0000 7f2279b96b80 20 client.25931 trim_cache size 0 max 16384 2023-09-23T09:35:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.297+0000 7f2279b96b80 1 Processor -- start 2023-09-23T09:35:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.297+0000 7f2279b96b80 1 -- start start 2023-09-23T09:35:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8bda7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2279b96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d89a98c0 con 0x55e3d8bddfb0 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2279b96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8bcdba0 con 0x55e3d8bd4f40 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2279b96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8bc8f60 con 0x55e3d8be0710 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2266ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2266ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56130/0 (socket says 172.21.15.45:56130) 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f2266ffd700 1 -- 172.21.15.45:0/3783116050 learned_addr learned my addr 172.21.15.45:0/3783116050 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f22677fe700 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8bda7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.298+0000 7f22667fc700 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f22667fc700 1 -- 172.21.15.45:0/3783116050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 msgr2=0x55e3d8bda7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f22667fc700 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8bda7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f22667fc700 1 -- 172.21.15.45:0/3783116050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 msgr2=0x55e3d8bdccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f22667fc700 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f22667fc700 1 -- 172.21.15.45:0/3783116050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3d893cc00 con 0x55e3d8bddfb0 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f2266ffd700 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f22667fc700 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec030 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f22580ca420 tx=0x7f2258010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8049f9436ff1a580 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f224e7fc700 1 -- 172.21.15.45:0/3783116050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225809a370 con 0x55e3d8bddfb0 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f224e7fc700 1 -- 172.21.15.45:0/3783116050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2258099c70 con 0x55e3d8bddfb0 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.299+0000 7f224e7fc700 1 -- 172.21.15.45:0/3783116050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258099ed0 con 0x55e3d8bddfb0 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 -- 172.21.15.45:0/3783116050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 msgr2=0x55e3d8bec030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec030 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f22580ca420 tx=0x7f2258010fd0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 -- 172.21.15.45:0/3783116050 shutdown_connections 2023-09-23T09:35:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8be0710 0x55e3d8bda7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8bd4f40 0x55e3d8bdccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 --2- 172.21.15.45:0/3783116050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8bec030 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 -- 172.21.15.45:0/3783116050 >> 172.21.15.45:0/3783116050 conn(0x55e3d8b4a200 msgr2=0x55e3d8beb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 -- 172.21.15.45:0/3783116050 shutdown_connections 2023-09-23T09:35:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.300+0000 7f2279b96b80 1 -- 172.21.15.45:0/3783116050 wait complete. 2023-09-23T09:35:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.301+0000 7f2279b96b80 1 -- start start 2023-09-23T09:35:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.301+0000 7f2279b96b80 1 Processor -- start 2023-09-23T09:35:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8bd4f40 0x55e3d8be7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8be44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2279b96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8be0710 0x55e3d8be49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2279b96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8be50d0 con 0x55e3d8bddfb0 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2279b96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8be5240 con 0x55e3d8be0710 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2279b96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d8c430c0 con 0x55e3d8bd4f40 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2279b96b80 10 client.0 _ll_register_callbacks cb 0x7f225c0073e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8bd4f40 0x55e3d8be7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8bd4f40 0x55e3d8be7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36892/0 (socket says 172.21.15.45:36892) 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 -- 172.21.15.45:0/1180671243 learned_addr learned my addr 172.21.15.45:0/1180671243 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f22667fc700 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8be44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f22677fe700 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8be0710 0x55e3d8be49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 -- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8be0710 msgr2=0x55e3d8be49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8be0710 0x55e3d8be49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 -- 172.21.15.45:0/1180671243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 msgr2=0x55e3d8be44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8be44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.302+0000 7f2266ffd700 1 -- 172.21.15.45:0/1180671243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3d8953d80 con 0x55e3d8bd4f40 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.303+0000 7f2266ffd700 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8bd4f40 0x55e3d8be7200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f225c00cac0 tx=0x7f225c008580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825a8b14835a02ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.304+0000 7f2246ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.304+0000 7f2246ffd700 1 -- 172.21.15.45:0/1180671243 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00f040 con 0x55e3d8bd4f40 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.304+0000 7f2246ffd700 1 -- 172.21.15.45:0/1180671243 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f225c000e40 con 0x55e3d8bd4f40 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.304+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f225c003680 con 0x55e3d8bd4f40 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.304+0000 7f2246ffd700 1 -- 172.21.15.45:0/1180671243 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f225c00cbe0 con 0x55e3d8bd4f40 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 10 client.25915 fetch_fsmap learned FSMap version 38 2023-09-23T09:35:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f22681b30c0 con 0x55e3d8bd4f40 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2246ffd700 1 -- 172.21.15.45:0/1180671243 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f225c003680 con 0x55e3d8bd4f40 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2246ffd700 1 -- 172.21.15.45:0/1180671243 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 38) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f225c005340 con 0x55e3d8bd4f40 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 10 client.25915 fetch_fsmap finished waiting for FSMap version 38 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 20 client.25915 populate_metadata read hostname 'smithi045' 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8c443c0 0x55e3d8c467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 4 client.25915 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3d8b21ad0 con 0x55e3d8c443c0 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.305+0000 7f22667fc700 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8c443c0 0x55e3d8c467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.306+0000 7f2246ffd700 10 client.25915 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.306+0000 7f22667fc700 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8c443c0 0x55e3d8c467e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65509047831ffb56 server_cookie=da3cdd1dbac1b505 in_seq=0 out_seq=0 2023-09-23T09:35:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2246ffd700 1 -- 172.21.15.45:0/1180671243 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3d8b21ad0 con 0x55e3d8c443c0 2023-09-23T09:35:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2246ffd700 10 client.25915 handle_command_reply: tid=0 2023-09-23T09:35:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 client.25915 shutdown 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8bd4f40 msgr2=0x55e3d8be7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8bd4f40 0x55e3d8be7200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f225c00cac0 tx=0x7f225c008580 comp rx=0 tx=0).stop 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 shutdown_connections 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e3d8c443c0 0x55e3d8c467e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e3d8be0710 0x55e3d8be49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2246ffd700 0 client.25915 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3d8bd4f40 0x55e3d8be7200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 --2- 172.21.15.45:0/1180671243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3d8bddfb0 0x55e3d8be44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.321+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 >> 172.21.15.45:0/1180671243 conn(0x55e3d8b4a200 msgr2=0x55e3d8bec1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 shutdown_connections 2023-09-23T09:35:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f2279b96b80 1 -- 172.21.15.45:0/1180671243 wait complete. 2023-09-23T09:35:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f2279b96b80 20 client.25915 trim_cache size 0 max 16384 2023-09-23T09:35:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:35:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:35:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:35:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:35:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22500a1810 msgr2=0x7f22500a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22500a1810 0x7f22500a3cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2260007810 tx=0x7f226000b040 comp rx=0 tx=0).stop 2023-09-23T09:35:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 msgr2=0x7f226819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:35:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 0x7f226819d7e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f225800b550 tx=0x7f225800b860 comp rx=0 tx=0).stop 2023-09-23T09:35:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 shutdown_connections 2023-09-23T09:35:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f226819dd20 0x7f22681b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2268108090 0x7f226819d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22500a1810 0x7f22500a3cd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 --2- 172.21.15.45:0/1730896057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f226810cb20 0x7f226819d7e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:35:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.322+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 >> 172.21.15.45:0/1730896057 conn(0x7f226806d320 msgr2=0x7f2268070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:35:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.323+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 shutdown_connections 2023-09-23T09:35:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:35:48.323+0000 7f226d9ac700 1 -- 172.21.15.45:0/1730896057 wait complete. 2023-09-23T09:35:50.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:49 smithi098 ceph-mon[102386]: pgmap v195: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-09-23T09:35:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:49 smithi045 ceph-mon[98627]: pgmap v195: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-09-23T09:35:50.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:49 smithi123 ceph-mon[103577]: pgmap v195: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-09-23T09:35:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 365.0 s, 1748.6 tps, lat 96.093 ms stddev 412.010 2023-09-23T09:35:52.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:51 smithi098 ceph-mon[102386]: pgmap v196: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-09-23T09:35:52.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:51 smithi045 ceph-mon[98627]: pgmap v196: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-09-23T09:35:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:51 smithi123 ceph-mon[103577]: pgmap v196: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-09-23T09:35:54.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:53 smithi098 ceph-mon[102386]: pgmap v197: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 863 op/s 2023-09-23T09:35:54.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:53 smithi045 ceph-mon[98627]: pgmap v197: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 863 op/s 2023-09-23T09:35:54.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:53 smithi123 ceph-mon[103577]: pgmap v197: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 863 op/s 2023-09-23T09:35:56.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:55 smithi098 ceph-mon[102386]: pgmap v198: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-23T09:35:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:55 smithi045 ceph-mon[98627]: pgmap v198: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-23T09:35:56.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:55 smithi123 ceph-mon[103577]: pgmap v198: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-23T09:35:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 370.0 s, 2405.4 tps, lat 41.376 ms stddev 49.454 2023-09-23T09:35:57.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:56 smithi098 ceph-mon[102386]: pgmap v199: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 836 op/s 2023-09-23T09:35:57.087 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:56 smithi045 ceph-mon[98627]: pgmap v199: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 836 op/s 2023-09-23T09:35:57.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:56 smithi123 ceph-mon[103577]: pgmap v199: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 836 op/s 2023-09-23T09:35:59.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:35:58 smithi098 ceph-mon[102386]: pgmap v200: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-09-23T09:35:59.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:35:58 smithi045 ceph-mon[98627]: pgmap v200: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-09-23T09:35:59.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:35:58 smithi123 ceph-mon[103577]: pgmap v200: 97 pgs: 97 active+clean; 7.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-09-23T09:36:00.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:36:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.928+0000 7fae91bc9700 1 -- 172.21.15.45:0/1355394296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c1008a0 msgr2=0x7fae8c1048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.928+0000 7fae91bc9700 1 --2- 172.21.15.45:0/1355394296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c1008a0 0x7fae8c1048e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fae80009990 tx=0x7fae80009ca0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.928+0000 7fae91bc9700 1 -- 172.21.15.45:0/1355394296 shutdown_connections 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 --2- 172.21.15.45:0/1355394296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c105010 0x7fae8c10cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 --2- 172.21.15.45:0/1355394296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 0x7fae8c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 --2- 172.21.15.45:0/1355394296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c1008a0 0x7fae8c1048e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 -- 172.21.15.45:0/1355394296 >> 172.21.15.45:0/1355394296 conn(0x7fae8c0fb770 msgr2=0x7fae8c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 -- 172.21.15.45:0/1355394296 shutdown_connections 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 -- 172.21.15.45:0/1355394296 wait complete. 2023-09-23T09:36:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 Processor -- start 2023-09-23T09:36:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.929+0000 7fae91bc9700 1 -- start start 2023-09-23T09:36:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae91bc9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 0x7fae8c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae91bc9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c105010 0x7fae8c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae91bc9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c19b870 0x7fae8c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae91bc9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c19c2f0 con 0x7fae8c19b870 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae91bc9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c19c460 con 0x7fae8c105010 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae91bc9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c19c620 con 0x7fae8c0ffef0 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 0x7fae8c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c105010 0x7fae8c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 0x7fae8c196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36906/0 (socket says 172.21.15.45:36906) 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 -- 172.21.15.45:0/2998553767 learned_addr learned my addr 172.21.15.45:0/2998553767 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8bfff700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c19b870 0x7fae8c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 -- 172.21.15.45:0/2998553767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c105010 msgr2=0x7fae8c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c105010 0x7fae8c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 -- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c19b870 msgr2=0x7fae8c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c19b870 0x7fae8c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 -- 172.21.15.45:0/2998553767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae800095f0 con 0x7fae8c0ffef0 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8affd700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c105010 0x7fae8c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8bfff700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c19b870 0x7fae8c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.930+0000 7fae8b7fe700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 0x7fae8c196ed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e9a0 tx=0x7fae7c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.931+0000 7fae88ff9700 1 -- 172.21.15.45:0/2998553767 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009790 con 0x7fae8c0ffef0 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.931+0000 7fae88ff9700 1 -- 172.21.15.45:0/2998553767 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae7c010430 con 0x7fae8c0ffef0 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.931+0000 7fae88ff9700 1 -- 172.21.15.45:0/2998553767 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c018680 con 0x7fae8c0ffef0 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.931+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c1a3b80 con 0x7fae8c0ffef0 2023-09-23T09:36:00.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.931+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c1a4060 con 0x7fae8c0ffef0 2023-09-23T09:36:00.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.935+0000 7fae9ddb3b80 1 Processor -- start 2023-09-23T09:36:00.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae9ddb3b80 1 -- start start 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6400 0x55596a3c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6d20 0x55596a3d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a3dc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a32f460 con 0x55596a3da3e0 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a320c20 con 0x55596a3c6400 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a31e540 con 0x55596a3c6d20 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6400 0x55596a3c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6400 0x55596a3c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56156/0 (socket says 172.21.15.45:56156) 2023-09-23T09:36:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.936+0000 7fae8bfff700 1 -- 172.21.15.45:0/601049698 learned_addr learned my addr 172.21.15.45:0/601049698 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae88ff9700 1 -- 172.21.15.45:0/2998553767 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fae7c0105a0 con 0x7fae8c0ffef0 2023-09-23T09:36:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae8bfff700 1 -- 172.21.15.45:0/601049698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6d20 msgr2=0x55596a3d9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae8bfff700 1 --2- 172.21.15.45:0/601049698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6d20 0x55596a3d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae8bfff700 1 -- 172.21.15.45:0/601049698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 msgr2=0x55596a3dc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae8bfff700 1 --2- 172.21.15.45:0/601049698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a3dc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae8bfff700 1 -- 172.21.15.45:0/601049698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596a37c8d0 con 0x55596a3c6400 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae8bfff700 1 --2- 172.21.15.45:0/601049698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6400 0x55596a3c67e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fae84003eb0 tx=0x7fae84009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d52d7f0d672e0d8f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae88ff9700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae740a1860 0x7fae740a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae71ffb700 1 -- 172.21.15.45:0/601049698 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840092d0 con 0x55596a3c6400 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae71ffb700 1 -- 172.21.15.45:0/601049698 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae8400f040 con 0x55596a3c6400 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae71ffb700 1 -- 172.21.15.45:0/601049698 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400e910 con 0x55596a3c6400 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.937+0000 7fae88ff9700 1 -- 172.21.15.45:0/2998553767 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae7c014070 con 0x7fae8c0ffef0 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/601049698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6400 msgr2=0x55596a3c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/601049698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6400 0x55596a3c67e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fae84003eb0 tx=0x7fae84009920 comp rx=0 tx=0).stop 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/601049698 shutdown_connections 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/601049698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a3dc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/601049698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6400 0x55596a3c67e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/601049698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6d20 0x55596a3d9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/601049698 >> 172.21.15.45:0/601049698 conn(0x55596a33b260 msgr2=0x55596a3c2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/601049698 shutdown_connections 2023-09-23T09:36:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.938+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/601049698 wait complete. 2023-09-23T09:36:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.939+0000 7fae9ddb3b80 1 -- start start 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.939+0000 7fae9ddb3b80 1 Processor -- start 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6d20 0x55596a3d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3da3e0 0x55596a3d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a319e30 con 0x55596a3c6d20 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a2b6710 con 0x55596a3da3e0 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a2b63b0 con 0x55596a3c6400 2023-09-23T09:36:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae9ddb3b80 10 client.0 _ll_register_callbacks cb 0x55596a113800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3d9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:36920/0 (socket says 172.21.15.45:36920) 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 -- 172.21.15.45:0/860837884 learned_addr learned my addr 172.21.15.45:0/860837884 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3da3e0 0x55596a3d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8b7fe700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae740a1860 0x7fae740a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 -- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3da3e0 msgr2=0x55596a3d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3da3e0 0x55596a3d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 -- 172.21.15.45:0/860837884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6d20 msgr2=0x55596a3d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6d20 0x55596a3d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 -- 172.21.15.45:0/860837884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae84003b90 con 0x55596a3c6400 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae8bfff700 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3d9ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae84009fd0 tx=0x7fae8400e4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e4e7b91ebeddc21 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.940+0000 7fae6affd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.941+0000 7fae6affd700 1 -- 172.21.15.45:0/860837884 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400e590 con 0x55596a3c6400 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.941+0000 7fae6affd700 1 -- 172.21.15.45:0/860837884 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae8400e6f0 con 0x55596a3c6400 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.941+0000 7fae8b7fe700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae740a1860 0x7fae740a3d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae7c09a080 tx=0x7fae7c0cd520 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55596a3ba250 con 0x55596a3c6400 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae6affd700 1 -- 172.21.15.45:0/860837884 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae8400e8b0 con 0x55596a3c6400 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 10 client.25924 fetch_fsmap learned FSMap version 38 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55596a144de0 con 0x55596a3c6400 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae6affd700 1 -- 172.21.15.45:0/860837884 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55596a3ba250 con 0x55596a3c6400 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae6affd700 1 -- 172.21.15.45:0/860837884 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 38) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fae8402c070 con 0x55596a3c6400 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 10 client.25924 fetch_fsmap finished waiting for FSMap version 38 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 10 client.25924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 20 client.25924 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a3cc2c0 0x55596a3ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 4 client.25924 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:36:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55596a312e10 con 0x55596a3cc2c0 2023-09-23T09:36:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.942+0000 7fae8affd700 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a3cc2c0 0x55596a3ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.943+0000 7fae6affd700 10 client.25924 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.943+0000 7fae8affd700 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a3cc2c0 0x55596a3ce6e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acf1f1f7cccdd3a server_cookie=bb16c638bc074493 in_seq=0 out_seq=0 2023-09-23T09:36:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae6affd700 1 -- 172.21.15.45:0/860837884 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55596a312e10 con 0x55596a3cc2c0 2023-09-23T09:36:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae6affd700 10 client.25924 handle_command_reply: tid=0 2023-09-23T09:36:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 client.25924 shutdown 2023-09-23T09:36:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 msgr2=0x55596a3d9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3d9ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae84009fd0 tx=0x7fae8400e4f0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 shutdown_connections 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a3cc2c0 0x55596a3ce6e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3da3e0 0x55596a3d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3d9ea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae6affd700 0 client.25924 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.944+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/860837884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6d20 0x55596a3d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.945+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 >> 172.21.15.45:0/860837884 conn(0x55596a33b260 msgr2=0x55596a3c2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.945+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 shutdown_connections 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.945+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/860837884 wait complete. 2023-09-23T09:36:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.945+0000 7fae9ddb3b80 20 client.25924 trim_cache size 0 max 16384 2023-09-23T09:36:00.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 Processor -- start 2023-09-23T09:36:00.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 -- start start 2023-09-23T09:36:00.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6400 0x55596a3d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3da3e0 0x55596a3d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a19a920 con 0x55596a3c6400 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a3bed60 con 0x55596a3c6d20 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a3c75f0 con 0x55596a3da3e0 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6400 0x55596a3d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6400 0x55596a3d6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34822/0 (socket says 172.21.15.45:34822) 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae8bfff700 1 -- 172.21.15.45:0/876760421 learned_addr learned my addr 172.21.15.45:0/876760421 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae8bfff700 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3da3e0 0x55596a3d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae8bfff700 1 -- 172.21.15.45:0/876760421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3da3e0 msgr2=0x55596a3d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.966+0000 7fae8bfff700 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3da3e0 0x55596a3d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.967+0000 7fae8bfff700 1 -- 172.21.15.45:0/876760421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 msgr2=0x55596a3d9a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:36:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.967+0000 7fae8bfff700 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3d9a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.967+0000 7fae8bfff700 1 -- 172.21.15.45:0/876760421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596a12dc60 con 0x55596a3c6400 2023-09-23T09:36:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.967+0000 7fae8bfff700 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6400 0x55596a3d6900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fae84000f80 tx=0x7fae840021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacc91b80db810d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.967+0000 7fae727fc700 1 -- 172.21.15.45:0/876760421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84009270 con 0x55596a3c6400 2023-09-23T09:36:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.967+0000 7fae727fc700 1 -- 172.21.15.45:0/876760421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae84015450 con 0x55596a3c6400 2023-09-23T09:36:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.967+0000 7fae727fc700 1 -- 172.21.15.45:0/876760421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84005290 con 0x55596a3c6400 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/876760421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6400 msgr2=0x55596a3d6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6400 0x55596a3d6900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fae84000f80 tx=0x7fae840021d0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/876760421 shutdown_connections 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3da3e0 0x55596a3d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3c6400 0x55596a3d6900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/876760421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3d9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/876760421 >> 172.21.15.45:0/876760421 conn(0x55596a33b260 msgr2=0x55596a3d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/876760421 shutdown_connections 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/876760421 wait complete. 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.968+0000 7fae9ddb3b80 1 -- start start 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 1 Processor -- start 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3df640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a4450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a445630 con 0x55596a3da3e0 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a4457a0 con 0x55596a3c6d20 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596a445960 con 0x55596a3c6400 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae9ddb3b80 10 client.0 _ll_register_callbacks cb 0x7fae84007930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a4450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae8affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3df640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a4450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34832/0 (socket says 172.21.15.45:34832) 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae8bfff700 1 -- 172.21.15.45:0/2380100627 learned_addr learned my addr 172.21.15.45:0/2380100627 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae8bfff700 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae8bfff700 1 -- 172.21.15.45:0/2380100627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 msgr2=0x55596a3df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.969+0000 7fae8bfff700 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3df100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae8bfff700 1 -- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 msgr2=0x55596a3df640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae8bfff700 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3df640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae8bfff700 1 -- 172.21.15.45:0/2380100627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55596a144de0 con 0x55596a3da3e0 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae8affd700 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3df640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae8bfff700 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a4450f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fae84009960 tx=0x7fae8403a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56801006893954b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae6affd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:36:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae6affd700 1 -- 172.21.15.45:0/2380100627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8403d030 con 0x55596a3da3e0 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae6affd700 1 -- 172.21.15.45:0/2380100627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fae8403e070 con 0x55596a3da3e0 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae8400f040 con 0x55596a3da3e0 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae6affd700 1 -- 172.21.15.45:0/2380100627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae840093f0 con 0x55596a3da3e0 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae9ddb3b80 10 client.16242 fetch_fsmap learned FSMap version 38 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55596a16b250 con 0x55596a3da3e0 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.970+0000 7fae6affd700 1 -- 172.21.15.45:0/2380100627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae8400f040 con 0x55596a3da3e0 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae6affd700 1 -- 172.21.15.45:0/2380100627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 38) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fae8402c040 con 0x55596a3da3e0 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae9ddb3b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 38 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae9ddb3b80 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae9ddb3b80 20 client.16242 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a446cc0 0x55596a4490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae9ddb3b80 4 client.16242 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:36:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55596a312b30 con 0x55596a446cc0 2023-09-23T09:36:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae8affd700 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a446cc0 0x55596a4490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae6affd700 10 client.16242 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.971+0000 7fae8affd700 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a446cc0 0x55596a4490e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f5736827044d89 server_cookie=c67791908e106156 in_seq=0 out_seq=0 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae6affd700 1 -- 172.21.15.45:0/2380100627 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55596a312b30 con 0x55596a446cc0 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae6affd700 10 client.16242 handle_command_reply: tid=0 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 client.16242 shutdown 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 msgr2=0x55596a4450f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a4450f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fae84009960 tx=0x7fae8403a040 comp rx=0 tx=0).stop 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 shutdown_connections 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55596a446cc0 0x55596a4490e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55596a3da3e0 0x55596a4450f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55596a3c6400 0x55596a3df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 --2- 172.21.15.45:0/2380100627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55596a3c6d20 0x55596a3df640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 >> 172.21.15.45:0/2380100627 conn(0x55596a33b260 msgr2=0x55596a3d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae6affd700 0 client.16242 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 shutdown_connections 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 1 -- 172.21.15.45:0/2380100627 wait complete. 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.972+0000 7fae9ddb3b80 20 client.16242 trim_cache size 0 max 16384 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:36:00.979 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:36:00.980 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae740a1860 msgr2=0x7fae740a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae740a1860 0x7fae740a3d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae7c09a080 tx=0x7fae7c0cd520 comp rx=0 tx=0).stop 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 msgr2=0x7fae8c196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 0x7fae8c196ed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e9a0 tx=0x7fae7c00ecb0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 shutdown_connections 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fae740a1860 0x7fae740a3d20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae8c19b870 0x7fae8c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c0ffef0 0x7fae8c196ed0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 --2- 172.21.15.45:0/2998553767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c105010 0x7fae8c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 >> 172.21.15.45:0/2998553767 conn(0x7fae8c0fb770 msgr2=0x7fae8c0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 shutdown_connections 2023-09-23T09:36:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:00.973+0000 7fae91bc9700 1 -- 172.21.15.45:0/2998553767 wait complete. 2023-09-23T09:36:00.990 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:36:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 -- 172.21.15.45:0/1494842909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0410f9e0 msgr2=0x7f3f04108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 --2- 172.21.15.45:0/1494842909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0410f9e0 0x7f3f04108360 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3ef401a9a0 tx=0x7f3ef401acb0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 -- 172.21.15.45:0/1494842909 shutdown_connections 2023-09-23T09:36:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 --2- 172.21.15.45:0/1494842909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0411a600 0x7f3f0411c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 --2- 172.21.15.45:0/1494842909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0410f570 0x7f3f04107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 --2- 172.21.15.45:0/1494842909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0410f9e0 0x7f3f04108360 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 -- 172.21.15.45:0/1494842909 >> 172.21.15.45:0/1494842909 conn(0x7f3f0406cee0 msgr2=0x7f3f0406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 -- 172.21.15.45:0/1494842909 shutdown_connections 2023-09-23T09:36:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.557+0000 7f3f0b566700 1 -- 172.21.15.45:0/1494842909 wait complete. 2023-09-23T09:36:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.558+0000 7f3f0b566700 1 Processor -- start 2023-09-23T09:36:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f0b566700 1 -- start start 2023-09-23T09:36:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f0b566700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0410f570 0x7f3f04119850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f0b566700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0410f9e0 0x7f3f04119d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f0b566700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0411a600 0x7f3f041156f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f0b566700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04115e80 con 0x7f3f0411a600 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f0b566700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04115ff0 con 0x7f3f0410f570 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f0b566700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f041161b0 con 0x7f3f0410f9e0 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f09b03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0411a600 0x7f3f041156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f09302700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0410f570 0x7f3f04119850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f09302700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0410f570 0x7f3f04119850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56192/0 (socket says 172.21.15.45:56192) 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f09b03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0411a600 0x7f3f041156f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34846/0 (socket says 172.21.15.45:34846) 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.559+0000 7f3f09302700 1 -- 172.21.15.45:0/2903985887 learned_addr learned my addr 172.21.15.45:0/2903985887 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.560+0000 7f3f09b03700 1 -- 172.21.15.45:0/2903985887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0410f9e0 msgr2=0x7f3f04119d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.560+0000 7f3f09b03700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0410f9e0 0x7f3f04119d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.560+0000 7f3f09b03700 1 -- 172.21.15.45:0/2903985887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0410f570 msgr2=0x7f3f04119850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.560+0000 7f3f09b03700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0410f570 0x7f3f04119850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.560+0000 7f3f09b03700 1 -- 172.21.15.45:0/2903985887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef401a600 con 0x7f3f0411a600 2023-09-23T09:36:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.560+0000 7f3f09b03700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0411a600 0x7f3f041156f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3efc007ee0 tx=0x7f3efc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.565+0000 7f3efa7fc700 1 -- 172.21.15.45:0/2903985887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00ddd0 con 0x7f3f0411a600 2023-09-23T09:36:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.565+0000 7f3efa7fc700 1 -- 172.21.15.45:0/2903985887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3efc00f040 con 0x7f3f0411a600 2023-09-23T09:36:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.565+0000 7f3efa7fc700 1 -- 172.21.15.45:0/2903985887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc005a40 con 0x7f3f0411a600 2023-09-23T09:36:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.565+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f04116490 con 0x7f3f0411a600 2023-09-23T09:36:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.565+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f041cc580 con 0x7f3f0411a600 2023-09-23T09:36:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.573+0000 7f3f17750b80 1 Processor -- start 2023-09-23T09:36:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.578+0000 7f3f17750b80 1 -- start start 2023-09-23T09:36:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.578+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5df40 0x5555b9c5e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.578+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5e860 0x5555b9c691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.578+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.578+0000 7f3f17750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9bc7400 con 0x5555b9c5df40 2023-09-23T09:36:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.578+0000 7f3f17750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9bb8bc0 con 0x5555b9c5e860 2023-09-23T09:36:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.578+0000 7f3f17750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9bb64e0 con 0x5555b9c69710 2023-09-23T09:36:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.579+0000 7f3f09302700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5e860 0x5555b9c691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.579+0000 7f3f09302700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5e860 0x5555b9c691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56202/0 (socket says 172.21.15.45:56202) 2023-09-23T09:36:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.579+0000 7f3f09302700 1 -- 172.21.15.45:0/401940410 learned_addr learned my addr 172.21.15.45:0/401940410 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.582+0000 7f3f09302700 1 -- 172.21.15.45:0/401940410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 msgr2=0x5555b9c6bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.582+0000 7f3f09302700 1 --2- 172.21.15.45:0/401940410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.582+0000 7f3f09302700 1 -- 172.21.15.45:0/401940410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5df40 msgr2=0x5555b9c5e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.582+0000 7f3f09302700 1 --2- 172.21.15.45:0/401940410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5df40 0x5555b9c5e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.582+0000 7f3f09302700 1 -- 172.21.15.45:0/401940410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b9c14870 con 0x5555b9c5e860 2023-09-23T09:36:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.586+0000 7f3f09302700 1 --2- 172.21.15.45:0/401940410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5e860 0x5555b9c691d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009a80 tx=0x7f3f00009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459313275f51cc3c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.588+0000 7f3eef7fe700 1 -- 172.21.15.45:0/401940410 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000c070 con 0x5555b9c5e860 2023-09-23T09:36:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.588+0000 7f3eef7fe700 1 -- 172.21.15.45:0/401940410 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0000ed20 con 0x5555b9c5e860 2023-09-23T09:36:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.588+0000 7f3eef7fe700 1 -- 172.21.15.45:0/401940410 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00013d50 con 0x5555b9c5e860 2023-09-23T09:36:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.589+0000 7f3f17750b80 1 -- 172.21.15.45:0/401940410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5e860 msgr2=0x5555b9c691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.589+0000 7f3f17750b80 1 --2- 172.21.15.45:0/401940410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5e860 0x5555b9c691d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009a80 tx=0x7f3f00009d90 comp rx=0 tx=0).stop 2023-09-23T09:36:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.603+0000 7f3efa7fc700 1 -- 172.21.15.45:0/2903985887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f3efc004ad0 con 0x7f3f0411a600 2023-09-23T09:36:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.604+0000 7f3efa7fc700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3ef00a18e0 0x7f3ef00a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.604+0000 7f3efa7fc700 1 -- 172.21.15.45:0/2903985887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3efc0cca60 con 0x7f3f0411a600 2023-09-23T09:36:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.609+0000 7f3f08b01700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3ef00a18e0 0x7f3ef00a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.611+0000 7f3f17750b80 1 -- 172.21.15.45:0/401940410 shutdown_connections 2023-09-23T09:36:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.611+0000 7f3f17750b80 1 --2- 172.21.15.45:0/401940410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.611+0000 7f3f17750b80 1 --2- 172.21.15.45:0/401940410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5df40 0x5555b9c5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.611+0000 7f3f17750b80 1 --2- 172.21.15.45:0/401940410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5e860 0x5555b9c691d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.611+0000 7f3f17750b80 1 -- 172.21.15.45:0/401940410 >> 172.21.15.45:0/401940410 conn(0x5555b9bd3200 msgr2=0x5555b9c5a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.633+0000 7f3f17750b80 1 -- 172.21.15.45:0/401940410 shutdown_connections 2023-09-23T09:36:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.642+0000 7f3f17750b80 1 -- 172.21.15.45:0/401940410 wait complete. 2023-09-23T09:36:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.642+0000 7f3f17750b80 1 -- start start 2023-09-23T09:36:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.642+0000 7f3f17750b80 1 Processor -- start 2023-09-23T09:36:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.646+0000 7f3f08b01700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3ef00a18e0 0x7f3ef00a3da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ef401c5d0 tx=0x7f3ef4005a90 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.649+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 0x5555b9c690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.649+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c66350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.649+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.649+0000 7f3f17750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9bb1dd0 con 0x5555b9c5e860 2023-09-23T09:36:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.649+0000 7f3f17750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9b4e6b0 con 0x5555b9c69710 2023-09-23T09:36:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.649+0000 7f3f17750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9b4e350 con 0x5555b9c5df40 2023-09-23T09:36:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.649+0000 7f3f17750b80 10 client.0 _ll_register_callbacks cb 0x5555b99ab7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c66350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c66350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34860/0 (socket says 172.21.15.45:34860) 2023-09-23T09:36:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 -- 172.21.15.45:0/3724676791 learned_addr learned my addr 172.21.15.45:0/3724676791 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 -- 172.21.15.45:0/3724676791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 msgr2=0x5555b9c690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 0x5555b9c690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 -- 172.21.15.45:0/3724676791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 msgr2=0x5555b9c669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.653+0000 7f3f08b01700 1 -- 172.21.15.45:0/3724676791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f000096e0 con 0x5555b9c5e860 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.654+0000 7f3f08b01700 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c66350 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21426baf1a4e5300 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.654+0000 7f3eecff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.654+0000 7f3eecff9700 1 -- 172.21.15.45:0/3724676791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef401cd90 con 0x5555b9c5e860 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.654+0000 7f3eecff9700 1 -- 172.21.15.45:0/3724676791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3ef4013030 con 0x5555b9c5e860 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.655+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555b9c52090 con 0x5555b9c5e860 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.655+0000 7f3eecff9700 1 -- 172.21.15.45:0/3724676791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef400e430 con 0x5555b9c5e860 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.655+0000 7f3f17750b80 10 client.16251 fetch_fsmap learned FSMap version 38 2023-09-23T09:36:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.655+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5555b99dcd80 con 0x5555b9c5e860 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.655+0000 7f3eecff9700 1 -- 172.21.15.45:0/3724676791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555b9c52090 con 0x5555b9c5e860 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.655+0000 7f3eecff9700 1 -- 172.21.15.45:0/3724676791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 38) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3ef4028550 con 0x5555b9c5e860 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.656+0000 7f3f17750b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 38 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.656+0000 7f3f17750b80 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.656+0000 7f3f17750b80 20 client.16251 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.656+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9c79170 0x5555b9c7b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.656+0000 7f3f17750b80 4 client.16251 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:36:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.656+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555b9baadb0 con 0x5555b9c79170 2023-09-23T09:36:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.664+0000 7f3f09302700 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9c79170 0x5555b9c7b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.686+0000 7f3eecff9700 10 client.16251 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.686+0000 7f3f09302700 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9c79170 0x5555b9c7b5d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16bee52442a82a7 server_cookie=48ffd1db82574d6d in_seq=0 out_seq=0 2023-09-23T09:36:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.699+0000 7f3eecff9700 1 -- 172.21.15.45:0/3724676791 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555b9baadb0 con 0x5555b9c79170 2023-09-23T09:36:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.699+0000 7f3eecff9700 10 client.16251 handle_command_reply: tid=0 2023-09-23T09:36:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.700+0000 7f3f17750b80 1 client.16251 shutdown 2023-09-23T09:36:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.701+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 msgr2=0x5555b9c66350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.701+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c66350 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef401cc70 comp rx=0 tx=0).stop 2023-09-23T09:36:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.701+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 shutdown_connections 2023-09-23T09:36:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.701+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9c79170 0x5555b9c7b5d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.702+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.702+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 0x5555b9c690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.702+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3724676791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c66350 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.702+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 >> 172.21.15.45:0/3724676791 conn(0x5555b9bd3200 msgr2=0x5555b9c59fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.710+0000 7f3eecff9700 0 client.16251 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.722+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 shutdown_connections 2023-09-23T09:36:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.729+0000 7f3f17750b80 1 -- 172.21.15.45:0/3724676791 wait complete. 2023-09-23T09:36:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.731+0000 7f3f17750b80 20 client.16251 trim_cache size 0 max 16384 2023-09-23T09:36:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.758+0000 7f3f17750b80 1 Processor -- start 2023-09-23T09:36:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.758+0000 7f3f17750b80 1 -- start start 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 0x5555b9c702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c65b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c7b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f17750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9a328c0 con 0x5555b9c5e860 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f17750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9c56ba0 con 0x5555b9c69710 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f17750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9c707f0 con 0x5555b9c5df40 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f09b03700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c7b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f09b03700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c7b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56216/0 (socket says 172.21.15.45:56216) 2023-09-23T09:36:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.759+0000 7f3f09b03700 1 -- 172.21.15.45:0/1007387742 learned_addr learned my addr 172.21.15.45:0/1007387742 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.762+0000 7f3f09302700 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 0x5555b9c702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.763+0000 7f3f08b01700 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c65b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.764+0000 7f3f08b01700 1 -- 172.21.15.45:0/1007387742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 msgr2=0x5555b9c702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.764+0000 7f3f08b01700 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 0x5555b9c702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.764+0000 7f3f08b01700 1 -- 172.21.15.45:0/1007387742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 msgr2=0x5555b9c7b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.764+0000 7f3f08b01700 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c7b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.764+0000 7f3f08b01700 1 -- 172.21.15.45:0/1007387742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b99c5c00 con 0x5555b9c5e860 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.764+0000 7f3f08b01700 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c65b10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005b40 tx=0x7f3ef4003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59005101e38cd278 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.764+0000 7f3eeffff700 1 -- 172.21.15.45:0/1007387742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef401cd90 con 0x5555b9c5e860 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.765+0000 7f3eeffff700 1 -- 172.21.15.45:0/1007387742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef4028aa0 con 0x5555b9c5e860 2023-09-23T09:36:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.765+0000 7f3eeffff700 1 -- 172.21.15.45:0/1007387742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef403bb60 con 0x5555b9c5e860 2023-09-23T09:36:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.766+0000 7f3f17750b80 1 -- 172.21.15.45:0/1007387742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 msgr2=0x5555b9c65b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.766+0000 7f3f17750b80 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c65b10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005b40 tx=0x7f3ef4003cc0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.784+0000 7f3f17750b80 1 -- 172.21.15.45:0/1007387742 shutdown_connections 2023-09-23T09:36:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.784+0000 7f3f17750b80 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c69710 0x5555b9c7b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.784+0000 7f3f17750b80 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c5df40 0x5555b9c702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.784+0000 7f3f17750b80 1 --2- 172.21.15.45:0/1007387742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c65b10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.784+0000 7f3f17750b80 1 -- 172.21.15.45:0/1007387742 >> 172.21.15.45:0/1007387742 conn(0x5555b9bd3200 msgr2=0x5555b9c74a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:01.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:01 smithi098 ceph-mon[102386]: pgmap v201: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 15 MiB/s wr, 904 op/s 2023-09-23T09:36:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.796+0000 7f3f17750b80 1 -- 172.21.15.45:0/1007387742 shutdown_connections 2023-09-23T09:36:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.799+0000 7f3f17750b80 1 -- 172.21.15.45:0/1007387742 wait complete. 2023-09-23T09:36:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 -- start start 2023-09-23T09:36:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 Processor -- start 2023-09-23T09:36:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5df40 0x5555b9c6c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c6c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9c70600 con 0x5555b9c5e860 2023-09-23T09:36:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9ccc010 con 0x5555b9c5df40 2023-09-23T09:36:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b9ccc150 con 0x5555b9c69710 2023-09-23T09:36:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.800+0000 7f3f17750b80 10 client.0 _ll_register_callbacks cb 0x5555b99ab7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.802+0000 7f3f08b01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c6c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.802+0000 7f3f08b01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c6c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34874/0 (socket says 172.21.15.45:34874) 2023-09-23T09:36:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.802+0000 7f3f08b01700 1 -- 172.21.15.45:0/3941931881 learned_addr learned my addr 172.21.15.45:0/3941931881 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.802+0000 7f3f09b03700 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.803+0000 7f3f09302700 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5df40 0x5555b9c6c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.803+0000 7f3f09b03700 1 -- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5df40 msgr2=0x5555b9c6c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.803+0000 7f3f09b03700 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5df40 0x5555b9c6c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.803+0000 7f3f09b03700 1 -- 172.21.15.45:0/3941931881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 msgr2=0x5555b9c6c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.803+0000 7f3f09b03700 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c6c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.803+0000 7f3f09b03700 1 -- 172.21.15.45:0/3941931881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555b99dcd80 con 0x5555b9c69710 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.803+0000 7f3f09b03700 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6cd10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009220 tx=0x7f3efc09a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21d893571c00c22a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.804+0000 7f3eecff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:36:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.810+0000 7f3eecff9700 1 -- 172.21.15.45:0/3941931881 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0b1070 con 0x5555b9c69710 2023-09-23T09:36:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.810+0000 7f3eecff9700 1 -- 172.21.15.45:0/3941931881 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3efc0a1cf0 con 0x5555b9c69710 2023-09-23T09:36:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.810+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f0000ed20 con 0x5555b9c69710 2023-09-23T09:36:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.814+0000 7f3eecff9700 1 -- 172.21.15.45:0/3941931881 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3efc099bc0 con 0x5555b9c69710 2023-09-23T09:36:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.819+0000 7f3f17750b80 10 client.25936 fetch_fsmap learned FSMap version 38 2023-09-23T09:36:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.819+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5555b9a031f0 con 0x5555b9c69710 2023-09-23T09:36:01.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:01 smithi045 ceph-mon[98627]: pgmap v201: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 15 MiB/s wr, 904 op/s 2023-09-23T09:36:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3eecff9700 1 -- 172.21.15.45:0/3941931881 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0000ed20 con 0x5555b9c69710 2023-09-23T09:36:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3eecff9700 1 -- 172.21.15.45:0/3941931881 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 38) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3efc0a1970 con 0x5555b9c69710 2023-09-23T09:36:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3f17750b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 38 2023-09-23T09:36:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3f17750b80 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3f17750b80 20 client.25936 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9ccd9c0 0x5555b9ccfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3f17750b80 4 client.25936 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:36:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.833+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555b9baaad0 con 0x5555b9ccd9c0 2023-09-23T09:36:01.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:01 smithi123 ceph-mon[103577]: pgmap v201: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 15 MiB/s wr, 904 op/s 2023-09-23T09:36:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.838+0000 7f3f09302700 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9ccd9c0 0x5555b9ccfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.841+0000 7f3f09302700 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9ccd9c0 0x5555b9ccfde0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d769c80265f085ac server_cookie=d937ab0846a03942 in_seq=0 out_seq=0 2023-09-23T09:36:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.849+0000 7f3eecff9700 10 client.25936 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 375.0 s, 1997.2 tps, lat 44.886 ms stddev 58.487 2023-09-23T09:36:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.999+0000 7f3eecff9700 1 -- 172.21.15.45:0/3941931881 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555b9baaad0 con 0x5555b9ccd9c0 2023-09-23T09:36:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:01.999+0000 7f3eecff9700 10 client.25936 handle_command_reply: tid=0 2023-09-23T09:36:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.000+0000 7f3f17750b80 1 client.25936 shutdown 2023-09-23T09:36:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.010+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 msgr2=0x5555b9c6cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.010+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6cd10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009220 tx=0x7f3efc09a110 comp rx=0 tx=0).stop 2023-09-23T09:36:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.015+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 shutdown_connections 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.015+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5555b9ccd9c0 0x5555b9ccfde0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.015+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555b9c69710 0x5555b9c6cd10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.015+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555b9c5df40 0x5555b9c6c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.015+0000 7f3f17750b80 1 --2- 172.21.15.45:0/3941931881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555b9c5e860 0x5555b9c6c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.015+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 >> 172.21.15.45:0/3941931881 conn(0x5555b9bd3200 msgr2=0x5555b9c7b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.015+0000 7f3eecff9700 0 client.25936 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.017+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 shutdown_connections 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.017+0000 7f3f17750b80 1 -- 172.21.15.45:0/3941931881 wait complete. 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.017+0000 7f3f17750b80 20 client.25936 trim_cache size 0 max 16384 2023-09-23T09:36:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:36:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:36:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:36:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:36:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3ef00a18e0 msgr2=0x7f3ef00a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3ef00a18e0 0x7f3ef00a3da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ef401c5d0 tx=0x7f3ef4005a90 comp rx=0 tx=0).stop 2023-09-23T09:36:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0411a600 msgr2=0x7f3f041156f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0411a600 0x7f3f041156f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3efc007ee0 tx=0x7f3efc00d3b0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 shutdown_connections 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3ef00a18e0 0x7f3ef00a3da0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ef401c5d0 tx=0x7f3ef4005a90 comp rx=0 tx=0).stop 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f0411a600 0x7f3f041156f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0410f570 0x7f3f04119850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 --2- 172.21.15.45:0/2903985887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0410f9e0 0x7f3f04119d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.020+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 >> 172.21.15.45:0/2903985887 conn(0x7f3f0406cee0 msgr2=0x7f3f0411bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.026+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 shutdown_connections 2023-09-23T09:36:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:02.026+0000 7f3f0b566700 1 -- 172.21.15.45:0/2903985887 wait complete. 2023-09-23T09:36:03.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:03 smithi098 ceph-mon[102386]: pgmap v202: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 14 MiB/s wr, 809 op/s 2023-09-23T09:36:03.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:03 smithi045 ceph-mon[98627]: pgmap v202: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 14 MiB/s wr, 809 op/s 2023-09-23T09:36:03.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:03 smithi123 ceph-mon[103577]: pgmap v202: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 14 MiB/s wr, 809 op/s 2023-09-23T09:36:05.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:05 smithi098 ceph-mon[102386]: pgmap v203: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 29 MiB/s wr, 2.05k op/s 2023-09-23T09:36:05.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:05 smithi098 ceph-mon[102386]: osdmap e88: 12 total, 12 up, 12 in 2023-09-23T09:36:05.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:05 smithi045 ceph-mon[98627]: pgmap v203: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 29 MiB/s wr, 2.05k op/s 2023-09-23T09:36:05.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:05 smithi045 ceph-mon[98627]: osdmap e88: 12 total, 12 up, 12 in 2023-09-23T09:36:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:05 smithi123 ceph-mon[103577]: pgmap v203: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 29 MiB/s wr, 2.05k op/s 2023-09-23T09:36:05.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:05 smithi123 ceph-mon[103577]: osdmap e88: 12 total, 12 up, 12 in 2023-09-23T09:36:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 380.0 s, 1922.0 tps, lat 57.262 ms stddev 103.707 2023-09-23T09:36:07.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:07 smithi098 ceph-mon[102386]: pgmap v205: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 1.93k op/s 2023-09-23T09:36:07.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:07 smithi045 ceph-mon[98627]: pgmap v205: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 1.93k op/s 2023-09-23T09:36:07.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:07 smithi123 ceph-mon[103577]: pgmap v205: 97 pgs: 97 active+clean; 8.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 1.93k op/s 2023-09-23T09:36:09.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:09 smithi098 ceph-mon[102386]: pgmap v206: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-09-23T09:36:09.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:09 smithi045 ceph-mon[98627]: pgmap v206: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-09-23T09:36:09.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:09 smithi123 ceph-mon[103577]: pgmap v206: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-09-23T09:36:11.802 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:11 smithi098 ceph-mon[102386]: pgmap v207: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-09-23T09:36:11.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:11 smithi045 ceph-mon[98627]: pgmap v207: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-09-23T09:36:11.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:11 smithi123 ceph-mon[103577]: pgmap v207: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-09-23T09:36:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 385.0 s, 1908.6 tps, lat 52.792 ms stddev 59.828 2023-09-23T09:36:13.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:13 smithi098 ceph-mon[102386]: pgmap v208: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-09-23T09:36:13.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:13 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:13.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:13 smithi123 ceph-mon[103577]: pgmap v208: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-09-23T09:36:13.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:13 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:13.845 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:13 smithi045 ceph-mon[98627]: pgmap v208: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-09-23T09:36:13.845 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:13 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 -- 172.21.15.45:0/2763438476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 msgr2=0x7fb820100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 --2- 172.21.15.45:0/2763438476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb820100320 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb810009990 tx=0x7fb810009ca0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 -- 172.21.15.45:0/2763438476 shutdown_connections 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 --2- 172.21.15.45:0/2763438476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb820105060 0x7fb82010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 --2- 172.21.15.45:0/2763438476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb820100320 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 --2- 172.21.15.45:0/2763438476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8201008f0 0x7fb820104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 -- 172.21.15.45:0/2763438476 >> 172.21.15.45:0/2763438476 conn(0x7fb8200fb7a0 msgr2=0x7fb8200fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 -- 172.21.15.45:0/2763438476 shutdown_connections 2023-09-23T09:36:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.192+0000 7fb827500700 1 -- 172.21.15.45:0/2763438476 wait complete. 2023-09-23T09:36:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 Processor -- start 2023-09-23T09:36:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 -- start start 2023-09-23T09:36:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb82019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb820105060 0x7fb82019e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8201a20a0 0x7fb8201a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a2b50 con 0x7fb8200fff40 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a2cc0 con 0x7fb820105060 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb827500700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a2e80 con 0x7fb8201a20a0 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb82529c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb82019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb82529c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb82019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34884/0 (socket says 172.21.15.45:34884) 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb825a9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8201a20a0 0x7fb8201a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb82529c700 1 -- 172.21.15.45:0/3629358536 learned_addr learned my addr 172.21.15.45:0/3629358536 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb82529c700 1 -- 172.21.15.45:0/3629358536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8201a20a0 msgr2=0x7fb8201a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.193+0000 7fb82529c700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8201a20a0 0x7fb8201a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb82529c700 1 -- 172.21.15.45:0/3629358536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb820105060 msgr2=0x7fb82019e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb82529c700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb820105060 0x7fb82019e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb82529c700 1 -- 172.21.15.45:0/3629358536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100095f0 con 0x7fb8200fff40 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8201a20a0 0x7fb8201a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb82529c700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb82019dca0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb810004730 tx=0x7fb810004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb8167fc700 1 -- 172.21.15.45:0/3629358536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81001d070 con 0x7fb8200fff40 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb8167fc700 1 -- 172.21.15.45:0/3629358536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb810005a00 con 0x7fb8200fff40 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8201aa350 con 0x7fb8200fff40 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb8167fc700 1 -- 172.21.15.45:0/3629358536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810005450 con 0x7fb8200fff40 2023-09-23T09:36:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.194+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8201aa800 con 0x7fb8200fff40 2023-09-23T09:36:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.198+0000 7fb8336eab80 1 Processor -- start 2023-09-23T09:36:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.198+0000 7fb8167fc700 1 -- 172.21.15.45:0/3629358536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fb8100047d0 con 0x7fb8200fff40 2023-09-23T09:36:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb8336eab80 1 -- start start 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae63320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae63860 0x55fcdae7f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae81b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb8336eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdadcc400 con 0x55fcdae63860 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb8336eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdadbdbc0 con 0x55fcdae7f720 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb8336eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdadbb4e0 con 0x55fcdae62f40 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb824a9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae81b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb824a9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae81b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56242/0 (socket says 172.21.15.45:56242) 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb824a9b700 1 -- 172.21.15.45:0/3261698786 learned_addr learned my addr 172.21.15.45:0/3261698786 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.199+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae63320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.200+0000 7fb8167fc700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb80c0a1790 0x7fb80c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.200+0000 7fb825a9d700 1 -- 172.21.15.45:0/3261698786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 msgr2=0x55fcdae81b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.200+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae81b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.200+0000 7fb825a9d700 1 -- 172.21.15.45:0/3261698786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae63860 msgr2=0x55fcdae7f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.200+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae63860 0x55fcdae7f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.200+0000 7fb825a9d700 1 -- 172.21.15.45:0/3261698786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcdae19870 con 0x55fcdae62f40 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.200+0000 7fb8167fc700 1 -- 172.21.15.45:0/3629358536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb8100ce520 con 0x7fb8200fff40 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.201+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb80c0a1790 0x7fb80c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.201+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae63320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb818003ec0 tx=0x7fb818004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ed424697819d1b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.201+0000 7fb80b7fe700 1 -- 172.21.15.45:0/3261698786 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818007720 con 0x55fcdae62f40 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.202+0000 7fb80b7fe700 1 -- 172.21.15.45:0/3261698786 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81800f040 con 0x55fcdae62f40 2023-09-23T09:36:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.202+0000 7fb80b7fe700 1 -- 172.21.15.45:0/3261698786 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818017470 con 0x55fcdae62f40 2023-09-23T09:36:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.202+0000 7fb8336eab80 1 -- 172.21.15.45:0/3261698786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 msgr2=0x55fcdae63320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.202+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb80c0a1790 0x7fb80c0a3c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb818020000 tx=0x7fb818021000 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.202+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae63320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb818003ec0 tx=0x7fb818004320 comp rx=0 tx=0).stop 2023-09-23T09:36:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 -- 172.21.15.45:0/3261698786 shutdown_connections 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae81b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae63320 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3261698786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae63860 0x55fcdae7f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 -- 172.21.15.45:0/3261698786 >> 172.21.15.45:0/3261698786 conn(0x55fcdadd8200 msgr2=0x55fcdae5f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 -- 172.21.15.45:0/3261698786 shutdown_connections 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 -- 172.21.15.45:0/3261698786 wait complete. 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 -- start start 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.203+0000 7fb8336eab80 1 Processor -- start 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae62f40 0x55fcdae72880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae63860 0x55fcdae72dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae6db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb8336eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdadb6dd0 con 0x55fcdae62f40 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb8336eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdad536b0 con 0x55fcdae7f720 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb8336eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdad53350 con 0x55fcdae63860 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb8336eab80 10 client.0 _ll_register_callbacks cb 0x55fcdabb07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae63860 0x55fcdae72dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb82529c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae62f40 0x55fcdae72880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae63860 0x55fcdae72dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37004/0 (socket says 172.21.15.45:37004) 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb825a9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae6db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 -- 172.21.15.45:0/1690112991 learned_addr learned my addr 172.21.15.45:0/1690112991 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 -- 172.21.15.45:0/1690112991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 msgr2=0x55fcdae6db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae6db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 -- 172.21.15.45:0/1690112991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae62f40 msgr2=0x55fcdae72880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae62f40 0x55fcdae72880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 -- 172.21.15.45:0/1690112991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb818003ab0 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb824a9b700 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae63860 0x55fcdae72dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009f10 tx=0x7fb81c0326c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccdc4371ee323794 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb808ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.204+0000 7fb808ff9700 1 -- 172.21.15.45:0/1690112991 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c009820 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb808ff9700 1 -- 172.21.15.45:0/1690112991 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb81c039070 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcdae57090 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb808ff9700 1 -- 172.21.15.45:0/1690112991 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb81c032e60 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 10 client.25948 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fcdabe1d80 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb808ff9700 1 -- 172.21.15.45:0/1690112991 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81c04a080 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb808ff9700 1 -- 172.21.15.45:0/1690112991 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb81c033970 con 0x55fcdae63860 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 10 client.25948 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 10 client.25948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 20 client.25948 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdae7de80 0x55fcdae7e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 4 client.25948 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:36:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcdadafdb0 con 0x55fcdae7de80 2023-09-23T09:36:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.205+0000 7fb82529c700 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdae7de80 0x55fcdae7e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.206+0000 7fb808ff9700 10 client.25948 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.206+0000 7fb82529c700 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdae7de80 0x55fcdae7e290 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7b5bbfd19d8c68 server_cookie=1e503a63d114394 in_seq=0 out_seq=0 2023-09-23T09:36:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.208+0000 7fb808ff9700 1 -- 172.21.15.45:0/1690112991 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fcdadafdb0 con 0x55fcdae7de80 2023-09-23T09:36:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.208+0000 7fb808ff9700 10 client.25948 handle_command_reply: tid=0 2023-09-23T09:36:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.208+0000 7fb8336eab80 1 client.25948 shutdown 2023-09-23T09:36:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.208+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae63860 msgr2=0x55fcdae72dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.208+0000 7fb8336eab80 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae63860 0x55fcdae72dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009f10 tx=0x7fb81c0326c0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.209+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 shutdown_connections 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.209+0000 7fb8336eab80 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdae7de80 0x55fcdae7e290 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.209+0000 7fb8336eab80 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae6db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.209+0000 7fb8336eab80 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae62f40 0x55fcdae72880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.209+0000 7fb808ff9700 0 client.25948 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.209+0000 7fb8336eab80 1 --2- 172.21.15.45:0/1690112991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae63860 0x55fcdae72dc0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009f10 tx=0x7fb81c0326c0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.209+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 >> 172.21.15.45:0/1690112991 conn(0x55fcdadd8200 msgr2=0x55fcdae5ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.210+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 shutdown_connections 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.210+0000 7fb8336eab80 1 -- 172.21.15.45:0/1690112991 wait complete. 2023-09-23T09:36:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.210+0000 7fb8336eab80 20 client.25948 trim_cache size 0 max 16384 2023-09-23T09:36:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 Processor -- start 2023-09-23T09:36:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 -- start start 2023-09-23T09:36:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae68370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae7f720 0x55fcdae72670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae65de0 0x55fcdae85b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdac378c0 con 0x55fcdae7f720 2023-09-23T09:36:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdae86070 con 0x55fcdae65de0 2023-09-23T09:36:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.234+0000 7fb8336eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdae6c0d0 con 0x55fcdae62f40 2023-09-23T09:36:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.239+0000 7fb82529c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae68370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.239+0000 7fb82529c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae68370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37012/0 (socket says 172.21.15.45:37012) 2023-09-23T09:36:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.239+0000 7fb82529c700 1 -- 172.21.15.45:0/3006729999 learned_addr learned my addr 172.21.15.45:0/3006729999 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.240+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae65de0 0x55fcdae85b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.241+0000 7fb825a9d700 1 -- 172.21.15.45:0/3006729999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 msgr2=0x55fcdae68370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.241+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae68370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.241+0000 7fb825a9d700 1 -- 172.21.15.45:0/3006729999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae7f720 msgr2=0x55fcdae72670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.241+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae7f720 0x55fcdae72670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.241+0000 7fb825a9d700 1 -- 172.21.15.45:0/3006729999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcdabcac00 con 0x55fcdae65de0 2023-09-23T09:36:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.249+0000 7fb825a9d700 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae65de0 0x55fcdae85b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb818004c90 tx=0x7fb818013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8e0ae424a284f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.250+0000 7fb80bfff700 1 -- 172.21.15.45:0/3006729999 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818033070 con 0x55fcdae65de0 2023-09-23T09:36:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.250+0000 7fb80bfff700 1 -- 172.21.15.45:0/3006729999 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb818004a30 con 0x55fcdae65de0 2023-09-23T09:36:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.250+0000 7fb80bfff700 1 -- 172.21.15.45:0/3006729999 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818007720 con 0x55fcdae65de0 2023-09-23T09:36:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.251+0000 7fb8336eab80 1 -- 172.21.15.45:0/3006729999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae65de0 msgr2=0x55fcdae85b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.251+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae65de0 0x55fcdae85b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb818004c90 tx=0x7fb818013000 comp rx=0 tx=0).stop 2023-09-23T09:36:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.252+0000 7fb8336eab80 1 -- 172.21.15.45:0/3006729999 shutdown_connections 2023-09-23T09:36:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.252+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae65de0 0x55fcdae85b30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.252+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.252+0000 7fb8336eab80 1 --2- 172.21.15.45:0/3006729999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae7f720 0x55fcdae72670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.252+0000 7fb8336eab80 1 -- 172.21.15.45:0/3006729999 >> 172.21.15.45:0/3006729999 conn(0x55fcdadd8200 msgr2=0x55fcdae5daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.253+0000 7fb8336eab80 1 -- 172.21.15.45:0/3006729999 shutdown_connections 2023-09-23T09:36:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.253+0000 7fb8336eab80 1 -- 172.21.15.45:0/3006729999 wait complete. 2023-09-23T09:36:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.253+0000 7fb8336eab80 1 -- start start 2023-09-23T09:36:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.253+0000 7fb8336eab80 1 Processor -- start 2023-09-23T09:36:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.254+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae7fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.254+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae65de0 0x55fcdae80040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.254+0000 7fb8336eab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae84b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.254+0000 7fb8336eab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdae807d0 con 0x55fcdae65de0 2023-09-23T09:36:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.254+0000 7fb8336eab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdae80940 con 0x55fcdae7f720 2023-09-23T09:36:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.254+0000 7fb8336eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcdae85090 con 0x55fcdae62f40 2023-09-23T09:36:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.254+0000 7fb8336eab80 10 client.0 _ll_register_callbacks cb 0x7fb81c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.255+0000 7fb824a9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae65de0 0x55fcdae80040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.255+0000 7fb824a9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae65de0 0x55fcdae80040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34914/0 (socket says 172.21.15.45:34914) 2023-09-23T09:36:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.255+0000 7fb824a9b700 1 -- 172.21.15.45:0/770151369 learned_addr learned my addr 172.21.15.45:0/770151369 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.255+0000 7fb824a9b700 1 -- 172.21.15.45:0/770151369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 msgr2=0x55fcdae7fb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:36:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.255+0000 7fb824a9b700 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae7fb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.256+0000 7fb824a9b700 1 -- 172.21.15.45:0/770151369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 msgr2=0x55fcdae84b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:36:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.256+0000 7fb824a9b700 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae84b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.256+0000 7fb824a9b700 1 -- 172.21.15.45:0/770151369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81802b040 con 0x55fcdae65de0 2023-09-23T09:36:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb824a9b700 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae65de0 0x55fcdae80040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb81c033d20 tx=0x7fb81c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32d322dcbc184d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb808ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb808ff9700 1 -- 172.21.15.45:0/770151369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c009800 con 0x55fcdae65de0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb808ff9700 1 -- 172.21.15.45:0/770151369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb81c039070 con 0x55fcdae65de0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb81800f040 con 0x55fcdae65de0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb808ff9700 1 -- 172.21.15.45:0/770151369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb81c032e60 con 0x55fcdae65de0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb8336eab80 10 client.16269 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.257+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fcdac081f0 con 0x55fcdae65de0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb808ff9700 1 -- 172.21.15.45:0/770151369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81800f040 con 0x55fcdae65de0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb808ff9700 1 -- 172.21.15.45:0/770151369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb81c005040 con 0x55fcdae65de0 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb8336eab80 10 client.16269 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb8336eab80 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb8336eab80 20 client.16269 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb8336eab80 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdaed3900 0x55fcdaed5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb8336eab80 4 client.16269 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:36:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fcdadafad0 con 0x55fcdaed3900 2023-09-23T09:36:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb82529c700 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdaed3900 0x55fcdaed5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.258+0000 7fb808ff9700 10 client.16269 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.259+0000 7fb82529c700 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdaed3900 0x55fcdaed5d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1be9d9c8be40a1 server_cookie=8707f0d5188af29f in_seq=0 out_seq=0 2023-09-23T09:36:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb808ff9700 1 -- 172.21.15.45:0/770151369 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fcdadafad0 con 0x55fcdaed3900 2023-09-23T09:36:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb808ff9700 10 client.16269 handle_command_reply: tid=0 2023-09-23T09:36:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 client.16269 shutdown 2023-09-23T09:36:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae65de0 msgr2=0x55fcdae80040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae65de0 0x55fcdae80040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb81c033d20 tx=0x7fb81c005d40 comp rx=0 tx=0).stop 2023-09-23T09:36:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 shutdown_connections 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fcdaed3900 0x55fcdaed5d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fcdae7f720 0x55fcdae84b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcdae62f40 0x55fcdae7fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb808ff9700 0 client.16269 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 --2- 172.21.15.45:0/770151369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcdae65de0 0x55fcdae80040 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 >> 172.21.15.45:0/770151369 conn(0x55fcdadd8200 msgr2=0x55fcdae68370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 shutdown_connections 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 1 -- 172.21.15.45:0/770151369 wait complete. 2023-09-23T09:36:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.280+0000 7fb8336eab80 20 client.16269 trim_cache size 0 max 16384 2023-09-23T09:36:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:36:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:36:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:36:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:36:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb80c0a1790 msgr2=0x7fb80c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb80c0a1790 0x7fb80c0a3c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb818020000 tx=0x7fb818021000 comp rx=0 tx=0).stop 2023-09-23T09:36:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 msgr2=0x7fb82019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb82019dca0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb810004730 tx=0x7fb810004760 comp rx=0 tx=0).stop 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 shutdown_connections 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8201a20a0 0x7fb8201a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb80c0a1790 0x7fb80c0a3c50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8200fff40 0x7fb82019dca0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 --2- 172.21.15.45:0/3629358536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb820105060 0x7fb82019e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 >> 172.21.15.45:0/3629358536 conn(0x7fb8200fb7a0 msgr2=0x7fb820068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 shutdown_connections 2023-09-23T09:36:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:15.281+0000 7fb827500700 1 -- 172.21.15.45:0/3629358536 wait complete. 2023-09-23T09:36:15.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:15 smithi098 ceph-mon[102386]: pgmap v209: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-09-23T09:36:15.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:15 smithi098 ceph-mon[102386]: osdmap e89: 12 total, 12 up, 12 in 2023-09-23T09:36:15.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:15 smithi045 ceph-mon[98627]: pgmap v209: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-09-23T09:36:15.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:15 smithi045 ceph-mon[98627]: osdmap e89: 12 total, 12 up, 12 in 2023-09-23T09:36:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:15 smithi123 ceph-mon[103577]: pgmap v209: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-09-23T09:36:15.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:15 smithi123 ceph-mon[103577]: osdmap e89: 12 total, 12 up, 12 in 2023-09-23T09:36:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 390.0 s, 2160.4 tps, lat 45.698 ms stddev 53.052 2023-09-23T09:36:17.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:17 smithi098 ceph-mon[102386]: pgmap v211: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-09-23T09:36:17.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:17 smithi045 ceph-mon[98627]: pgmap v211: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-09-23T09:36:17.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:17 smithi123 ceph-mon[103577]: pgmap v211: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-09-23T09:36:19.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:19 smithi098 ceph-mon[102386]: pgmap v212: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2023-09-23T09:36:19.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:19 smithi045 ceph-mon[98627]: pgmap v212: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2023-09-23T09:36:19.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:19 smithi123 ceph-mon[103577]: pgmap v212: 97 pgs: 97 active+clean; 6.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2023-09-23T09:36:21.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:21 smithi098 ceph-mon[102386]: pgmap v213: 97 pgs: 97 active+clean; 7.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 677 op/s 2023-09-23T09:36:21.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:21 smithi045 ceph-mon[98627]: pgmap v213: 97 pgs: 97 active+clean; 7.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 677 op/s 2023-09-23T09:36:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:21 smithi123 ceph-mon[103577]: pgmap v213: 97 pgs: 97 active+clean; 7.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 677 op/s 2023-09-23T09:36:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 395.0 s, 2251.8 tps, lat 44.949 ms stddev 50.719 2023-09-23T09:36:23.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:23 smithi045 ceph-mon[98627]: pgmap v214: 97 pgs: 97 active+clean; 7.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 677 op/s 2023-09-23T09:36:23.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:23 smithi123 ceph-mon[103577]: pgmap v214: 97 pgs: 97 active+clean; 7.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 677 op/s 2023-09-23T09:36:24.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:23 smithi098 ceph-mon[102386]: pgmap v214: 97 pgs: 97 active+clean; 7.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 677 op/s 2023-09-23T09:36:25.841 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:25 smithi123 ceph-mon[103577]: pgmap v215: 97 pgs: 97 active+clean; 7.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.4 MiB/s wr, 682 op/s 2023-09-23T09:36:26.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:25 smithi098 ceph-mon[102386]: pgmap v215: 97 pgs: 97 active+clean; 7.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.4 MiB/s wr, 682 op/s 2023-09-23T09:36:26.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:25 smithi045 ceph-mon[98627]: pgmap v215: 97 pgs: 97 active+clean; 7.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.4 MiB/s wr, 682 op/s 2023-09-23T09:36:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 400.0 s, 943.2 tps, lat 101.378 ms stddev 180.566 2023-09-23T09:36:28.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:27 smithi098 ceph-mon[102386]: pgmap v216: 97 pgs: 97 active+clean; 7.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.0 MiB/s wr, 654 op/s 2023-09-23T09:36:28.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:27 smithi045 ceph-mon[98627]: pgmap v216: 97 pgs: 97 active+clean; 7.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.0 MiB/s wr, 654 op/s 2023-09-23T09:36:28.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:27 smithi123 ceph-mon[103577]: pgmap v216: 97 pgs: 97 active+clean; 7.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.0 MiB/s wr, 654 op/s 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 -- 172.21.15.45:0/3645329543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 msgr2=0x7ff720109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 --2- 172.21.15.45:0/3645329543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff720109ba0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff710009990 tx=0x7ff710009ca0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 -- 172.21.15.45:0/3645329543 shutdown_connections 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 --2- 172.21.15.45:0/3645329543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720112d70 0x7ff720115160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 --2- 172.21.15.45:0/3645329543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff720109ba0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 --2- 172.21.15.45:0/3645329543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff72010a0e0 0x7ff720112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 -- 172.21.15.45:0/3645329543 >> 172.21.15.45:0/3645329543 conn(0x7ff72006cee0 msgr2=0x7ff72006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 -- 172.21.15.45:0/3645329543 shutdown_connections 2023-09-23T09:36:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.450+0000 7ff725442700 1 -- 172.21.15.45:0/3645329543 wait complete. 2023-09-23T09:36:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 Processor -- start 2023-09-23T09:36:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 -- start start 2023-09-23T09:36:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff7201a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72010a0e0 0x7ff7201a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff720112d70 0x7ff72019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72019daa0 con 0x7ff7201097c0 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72019dc10 con 0x7ff720112d70 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.451+0000 7ff725442700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72019ddd0 con 0x7ff72010a0e0 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff7201a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff7201a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34924/0 (socket says 172.21.15.45:34924) 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 -- 172.21.15.45:0/3624669964 learned_addr learned my addr 172.21.15.45:0/3624669964 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71f7fe700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff720112d70 0x7ff72019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 -- 172.21.15.45:0/3624669964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72010a0e0 msgr2=0x7ff7201a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72010a0e0 0x7ff7201a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 -- 172.21.15.45:0/3624669964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff720112d70 msgr2=0x7ff72019d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff720112d70 0x7ff72019d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 -- 172.21.15.45:0/3624669964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100095f0 con 0x7ff7201097c0 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff71effd700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff7201a44d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098e0 tx=0x7ff710005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff707fff700 1 -- 172.21.15.45:0/3624669964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71001d070 con 0x7ff7201097c0 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72019e050 con 0x7ff7201097c0 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff707fff700 1 -- 172.21.15.45:0/3624669964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff710021410 con 0x7ff7201097c0 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.452+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7201a2240 con 0x7ff7201097c0 2023-09-23T09:36:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.453+0000 7ff707fff700 1 -- 172.21.15.45:0/3624669964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7100175f0 con 0x7ff7201097c0 2023-09-23T09:36:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 Processor -- start 2023-09-23T09:36:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 -- start start 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff707fff700 1 -- 172.21.15.45:0/3624669964 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7ff710021580 con 0x7ff7201097c0 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb11320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb11860 0x5634ecb1c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634eca7a400 con 0x5634ecb1c710 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634eca6bbc0 con 0x5634ecb11860 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.457+0000 7ff73162cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634eca694e0 con 0x5634ecb10f40 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb11320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb11320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37036/0 (socket says 172.21.15.45:37036) 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71f7fe700 1 -- 172.21.15.45:0/3782385751 learned_addr learned my addr 172.21.15.45:0/3782385751 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71effd700 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb11860 0x5634ecb1c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71e7fc700 1 -- 172.21.15.45:0/3782385751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 msgr2=0x5634ecb11320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb11320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71e7fc700 1 -- 172.21.15.45:0/3782385751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb11860 msgr2=0x5634ecb1c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb11860 0x5634ecb1c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff71e7fc700 1 -- 172.21.15.45:0/3782385751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634ecac7870 con 0x5634ecb1c710 2023-09-23T09:36:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff707fff700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff7080a1750 0x7ff7080a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.458+0000 7ff707fff700 1 -- 172.21.15.45:0/3624669964 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff7100ce280 con 0x7ff7201097c0 2023-09-23T09:36:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.459+0000 7ff71f7fe700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff7080a1750 0x7ff7080a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.460+0000 7ff71f7fe700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff7080a1750 0x7ff7080a3c10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7180095a0 tx=0x7ff71800d040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.460+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb1eb00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff714009ba0 tx=0x7ff714009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232ab977074228c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.461+0000 7ff7057fa700 1 -- 172.21.15.45:0/3782385751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714004230 con 0x5634ecb1c710 2023-09-23T09:36:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.461+0000 7ff7057fa700 1 -- 172.21.15.45:0/3782385751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff71400edc0 con 0x5634ecb1c710 2023-09-23T09:36:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.461+0000 7ff7057fa700 1 -- 172.21.15.45:0/3782385751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714005020 con 0x5634ecb1c710 2023-09-23T09:36:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 -- 172.21.15.45:0/3782385751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 msgr2=0x5634ecb1eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb1eb00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff714009ba0 tx=0x7ff714009eb0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 -- 172.21.15.45:0/3782385751 shutdown_connections 2023-09-23T09:36:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb1eb00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb11320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 --2- 172.21.15.45:0/3782385751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb11860 0x5634ecb1c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 -- 172.21.15.45:0/3782385751 >> 172.21.15.45:0/3782385751 conn(0x5634eca86200 msgr2=0x5634ecb0d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.462+0000 7ff73162cb80 1 -- 172.21.15.45:0/3782385751 shutdown_connections 2023-09-23T09:36:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.465+0000 7ff73162cb80 1 -- 172.21.15.45:0/3782385751 wait complete. 2023-09-23T09:36:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.465+0000 7ff73162cb80 1 -- start start 2023-09-23T09:36:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.465+0000 7ff73162cb80 1 Processor -- start 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.465+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb10f40 0x5634ecb1c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.465+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb19330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb19960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff73162cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634eca64dd0 con 0x5634ecb1c710 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff73162cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634eca016b0 con 0x5634ecb10f40 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff73162cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634eca01350 con 0x5634ecb11860 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff73162cb80 10 client.0 _ll_register_callbacks cb 0x5634ec85e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff71e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb19330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff71e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb19330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37046/0 (socket says 172.21.15.45:37046) 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff71e7fc700 1 -- 172.21.15.45:0/1013146731 learned_addr learned my addr 172.21.15.45:0/1013146731 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.466+0000 7ff71effd700 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb10f40 0x5634ecb1c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff71effd700 1 -- 172.21.15.45:0/1013146731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 msgr2=0x5634ecb19330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff71effd700 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb19330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff71effd700 1 -- 172.21.15.45:0/1013146731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 msgr2=0x5634ecb19960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff71effd700 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb19960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff71effd700 1 -- 172.21.15.45:0/1013146731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff714009880 con 0x5634ecb10f40 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff71effd700 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb10f40 0x5634ecb1c120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff7100cb970 tx=0x7ff710009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f28cb2741c00bc76 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff702ff5700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:36:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff702ff5700 1 -- 172.21.15.45:0/1013146731 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71009a910 con 0x5634ecb10f40 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634ecb05090 con 0x5634ecb10f40 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff702ff5700 1 -- 172.21.15.45:0/1013146731 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff71009b920 con 0x5634ecb10f40 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.467+0000 7ff702ff5700 1 -- 172.21.15.45:0/1013146731 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff710097560 con 0x5634ecb10f40 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 10 client.25991 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5634ec88fd80 con 0x5634ecb10f40 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff702ff5700 1 -- 172.21.15.45:0/1013146731 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634ecb05090 con 0x5634ecb10f40 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff702ff5700 1 -- 172.21.15.45:0/1013146731 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff7100a3090 con 0x5634ecb10f40 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 10 client.25991 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 10 client.25991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 20 client.25991 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb2bf20 0x5634ecb2e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 4 client.25991 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:36:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.468+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634eca5ddb0 con 0x5634ecb2bf20 2023-09-23T09:36:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.469+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb2bf20 0x5634ecb2e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.469+0000 7ff702ff5700 10 client.25991 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.469+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb2bf20 0x5634ecb2e340 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd4506bb087e803 server_cookie=8c57d5d503d22a12 in_seq=0 out_seq=0 2023-09-23T09:36:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.471+0000 7ff702ff5700 1 -- 172.21.15.45:0/1013146731 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5634eca5ddb0 con 0x5634ecb2bf20 2023-09-23T09:36:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.471+0000 7ff702ff5700 10 client.25991 handle_command_reply: tid=0 2023-09-23T09:36:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.471+0000 7ff73162cb80 1 client.25991 shutdown 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.471+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb10f40 msgr2=0x5634ecb1c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.471+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb10f40 0x5634ecb1c120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff7100cb970 tx=0x7ff710009ca0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 shutdown_connections 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb2bf20 0x5634ecb2e340 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb1c710 0x5634ecb19960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb10f40 0x5634ecb1c120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1013146731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb19330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 >> 172.21.15.45:0/1013146731 conn(0x5634eca86200 msgr2=0x5634ecb0cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff702ff5700 0 client.25991 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 shutdown_connections 2023-09-23T09:36:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 1 -- 172.21.15.45:0/1013146731 wait complete. 2023-09-23T09:36:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.472+0000 7ff73162cb80 20 client.25991 trim_cache size 0 max 16384 2023-09-23T09:36:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 Processor -- start 2023-09-23T09:36:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 -- start start 2023-09-23T09:36:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb10f40 0x5634ecb1bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 0x5634ecb18bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ec8e58c0 con 0x5634ecb10f40 2023-09-23T09:36:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ecb12a00 con 0x5634ecb1c710 2023-09-23T09:36:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff73162cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ecb0e100 con 0x5634ecb11860 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff71effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb10f40 0x5634ecb1bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff71effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb10f40 0x5634ecb1bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34946/0 (socket says 172.21.15.45:34946) 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.498+0000 7ff71effd700 1 -- 172.21.15.45:0/756975829 learned_addr learned my addr 172.21.15.45:0/756975829 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.499+0000 7ff71effd700 1 -- 172.21.15.45:0/756975829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 msgr2=0x5634ecb146d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.499+0000 7ff71effd700 1 --2- 172.21.15.45:0/756975829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.499+0000 7ff71effd700 1 -- 172.21.15.45:0/756975829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 msgr2=0x5634ecb18bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.499+0000 7ff71effd700 1 --2- 172.21.15.45:0/756975829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 0x5634ecb18bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.499+0000 7ff71effd700 1 -- 172.21.15.45:0/756975829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634ec878c00 con 0x5634ecb10f40 2023-09-23T09:36:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.499+0000 7ff71effd700 1 --2- 172.21.15.45:0/756975829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb10f40 0x5634ecb1bc50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7100d1ad0 tx=0x7ff7100ce1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdbaa1a358e8fa4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.500+0000 7ff705ffb700 1 -- 172.21.15.45:0/756975829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71009f150 con 0x5634ecb10f40 2023-09-23T09:36:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.500+0000 7ff705ffb700 1 -- 172.21.15.45:0/756975829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff710097560 con 0x5634ecb10f40 2023-09-23T09:36:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.500+0000 7ff705ffb700 1 -- 172.21.15.45:0/756975829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71009a930 con 0x5634ecb10f40 2023-09-23T09:36:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 -- 172.21.15.45:0/756975829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb10f40 msgr2=0x5634ecb1bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 --2- 172.21.15.45:0/756975829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb10f40 0x5634ecb1bc50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7100d1ad0 tx=0x7ff7100ce1c0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 -- 172.21.15.45:0/756975829 shutdown_connections 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 --2- 172.21.15.45:0/756975829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 0x5634ecb18bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 --2- 172.21.15.45:0/756975829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb10f40 0x5634ecb1bc50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 --2- 172.21.15.45:0/756975829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb11860 0x5634ecb146d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 -- 172.21.15.45:0/756975829 >> 172.21.15.45:0/756975829 conn(0x5634eca86200 msgr2=0x5634ecb279b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 -- 172.21.15.45:0/756975829 shutdown_connections 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.501+0000 7ff73162cb80 1 -- 172.21.15.45:0/756975829 wait complete. 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 -- start start 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 Processor -- start 2023-09-23T09:36:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb1f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb11860 0x5634ecb1f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 0x5634ecb7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ecb1ff20 con 0x5634ecb11860 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ecb20090 con 0x5634ecb1c710 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ecb7f2b0 con 0x5634ecb10f40 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff73162cb80 10 client.0 _ll_register_callbacks cb 0x5634ec85e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb11860 0x5634ecb1f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb1f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 0x5634ecb7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb11860 0x5634ecb1f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34954/0 (socket says 172.21.15.45:34954) 2023-09-23T09:36:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71e7fc700 1 -- 172.21.15.45:0/1356705974 learned_addr learned my addr 172.21.15.45:0/1356705974 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71e7fc700 1 -- 172.21.15.45:0/1356705974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 msgr2=0x5634ecb1f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb1f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71e7fc700 1 -- 172.21.15.45:0/1356705974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 msgr2=0x5634ecb7ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.502+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 0x5634ecb7ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff71e7fc700 1 -- 172.21.15.45:0/1356705974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634ec88fd80 con 0x5634ecb11860 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff71e7fc700 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb11860 0x5634ecb1f770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff714000c00 tx=0x7ff71400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6af6f6a74d4042 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff702ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff702ff5700 1 -- 172.21.15.45:0/1356705974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71401c040 con 0x5634ecb11860 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff702ff5700 1 -- 172.21.15.45:0/1356705974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff714014070 con 0x5634ecb11860 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff71400edc0 con 0x5634ecb11860 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff702ff5700 1 -- 172.21.15.45:0/1356705974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff714004830 con 0x5634ecb11860 2023-09-23T09:36:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 10 client.16284 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5634ec8b61f0 con 0x5634ecb11860 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff702ff5700 1 -- 172.21.15.45:0/1356705974 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff71400edc0 con 0x5634ecb11860 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff702ff5700 1 -- 172.21.15.45:0/1356705974 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff714018770 con 0x5634ecb11860 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 10 client.16284 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 20 client.16284 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb80a20 0x5634ecb82e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 4 client.16284 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.503+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5634eca5dad0 con 0x5634ecb80a20 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.504+0000 7ff71effd700 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb80a20 0x5634ecb82e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.504+0000 7ff71effd700 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb80a20 0x5634ecb82e40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a39dae4efea980 server_cookie=b1f07df9fffe1bfa in_seq=0 out_seq=0 2023-09-23T09:36:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.504+0000 7ff702ff5700 10 client.16284 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.613+0000 7ff702ff5700 1 -- 172.21.15.45:0/1356705974 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5634eca5dad0 con 0x5634ecb80a20 2023-09-23T09:36:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.613+0000 7ff702ff5700 10 client.16284 handle_command_reply: tid=0 2023-09-23T09:36:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.613+0000 7ff73162cb80 1 client.16284 shutdown 2023-09-23T09:36:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb11860 msgr2=0x5634ecb1f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb11860 0x5634ecb1f770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff714000c00 tx=0x7ff71400e900 comp rx=0 tx=0).stop 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 shutdown_connections 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5634ecb80a20 0x5634ecb82e40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5634ecb1c710 0x5634ecb7ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634ecb10f40 0x5634ecb1f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff702ff5700 0 client.16284 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 --2- 172.21.15.45:0/1356705974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ecb11860 0x5634ecb1f770 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 >> 172.21.15.45:0/1356705974 conn(0x5634eca86200 msgr2=0x5634ecb146d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 shutdown_connections 2023-09-23T09:36:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 1 -- 172.21.15.45:0/1356705974 wait complete. 2023-09-23T09:36:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.615+0000 7ff73162cb80 20 client.16284 trim_cache size 0 max 16384 2023-09-23T09:36:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:36:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:36:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:36:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:36:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff7080a1750 msgr2=0x7ff7080a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff7080a1750 0x7ff7080a3c10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7180095a0 tx=0x7ff71800d040 comp rx=0 tx=0).stop 2023-09-23T09:36:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 msgr2=0x7ff7201a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff7201a44d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098e0 tx=0x7ff710005720 comp rx=0 tx=0).stop 2023-09-23T09:36:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 shutdown_connections 2023-09-23T09:36:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff720112d70 0x7ff72019d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff7080a1750 0x7ff7080a3c10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7201097c0 0x7ff7201a44d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 --2- 172.21.15.45:0/3624669964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72010a0e0 0x7ff7201a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 >> 172.21.15.45:0/3624669964 conn(0x7ff72006cee0 msgr2=0x7ff720070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 shutdown_connections 2023-09-23T09:36:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:28.616+0000 7ff725442700 1 -- 172.21.15.45:0/3624669964 wait complete. 2023-09-23T09:36:30.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:29 smithi098 ceph-mon[102386]: pgmap v217: 97 pgs: 97 active+clean; 7.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-09-23T09:36:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:29 smithi045 ceph-mon[98627]: pgmap v217: 97 pgs: 97 active+clean; 7.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-09-23T09:36:30.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:29 smithi123 ceph-mon[103577]: pgmap v217: 97 pgs: 97 active+clean; 7.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-09-23T09:36:30.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:36:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 -- 172.21.15.45:0/1698978562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f4d0 msgr2=0x7f57ac10d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/1698978562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f4d0 0x7f57ac10d060 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f57a40098f0 tx=0x7f57a4009c00 comp rx=0 tx=0).stop 2023-09-23T09:36:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 -- 172.21.15.45:0/1698978562 shutdown_connections 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/1698978562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57ac11a600 0x7f57ac11c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/1698978562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f4d0 0x7f57ac10d060 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/1698978562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57ac10f940 0x7f57ac10d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 -- 172.21.15.45:0/1698978562 >> 172.21.15.45:0/1698978562 conn(0x7f57ac06d0e0 msgr2=0x7f57ac06d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 -- 172.21.15.45:0/1698978562 shutdown_connections 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.179+0000 7f57b3c4c700 1 -- 172.21.15.45:0/1698978562 wait complete. 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 Processor -- start 2023-09-23T09:36:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 -- start start 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57ac10f4d0 0x7f57ac1159a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f940 0x7f57ac115ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57ac11a600 0x7f57ac1a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac1a9890 con 0x7f57ac10f940 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac1a9a00 con 0x7f57ac11a600 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.180+0000 7f57b3c4c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac1a9bc0 con 0x7f57ac10f4d0 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f940 0x7f57ac115ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57ac10f4d0 0x7f57ac1159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57ac10f4d0 0x7f57ac1159a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37072/0 (socket says 172.21.15.45:37072) 2023-09-23T09:36:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b19e8700 1 -- 172.21.15.45:0/352165255 learned_addr learned my addr 172.21.15.45:0/352165255 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f940 0x7f57ac115ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34968/0 (socket says 172.21.15.45:34968) 2023-09-23T09:36:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b21e9700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57ac11a600 0x7f57ac1a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b11e7700 1 -- 172.21.15.45:0/352165255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57ac10f4d0 msgr2=0x7f57ac1159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b11e7700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57ac10f4d0 0x7f57ac1159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b11e7700 1 -- 172.21.15.45:0/352165255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57ac11a600 msgr2=0x7f57ac1a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b11e7700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57ac11a600 0x7f57ac1a9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b11e7700 1 -- 172.21.15.45:0/352165255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c01ad10 con 0x7f57ac10f940 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.181+0000 7f57b21e9700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57ac11a600 0x7f57ac1a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.182+0000 7f57b11e7700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f940 0x7f57ac115ee0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f579c01d340 tx=0x7f579c01d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.182+0000 7f57aaffd700 1 -- 172.21.15.45:0/352165255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c028070 con 0x7f57ac10f940 2023-09-23T09:36:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.182+0000 7f57aaffd700 1 -- 172.21.15.45:0/352165255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579c020040 con 0x7f57ac10f940 2023-09-23T09:36:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.182+0000 7f57aaffd700 1 -- 172.21.15.45:0/352165255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c01f050 con 0x7f57ac10f940 2023-09-23T09:36:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.183+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a4009520 con 0x7f57ac10f940 2023-09-23T09:36:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.183+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac1aa170 con 0x7f57ac10f940 2023-09-23T09:36:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.188+0000 7f57bfe36b80 1 Processor -- start 2023-09-23T09:36:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.188+0000 7f57bfe36b80 1 -- start start 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2400 0x56148ffc27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffcd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 0x56148ffcffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ff2b460 con 0x56148ffc2d20 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ff1cc20 con 0x56148ffcdbd0 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ff1a540 con 0x56148ffc2400 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffcd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffcd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34974/0 (socket says 172.21.15.45:34974) 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 -- 172.21.15.45:0/2618939438 learned_addr learned my addr 172.21.15.45:0/2618939438 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b21e9700 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2400 0x56148ffc27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 -- 172.21.15.45:0/2618939438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2400 msgr2=0x56148ffc27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2400 0x56148ffc27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 -- 172.21.15.45:0/2618939438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 msgr2=0x56148ffcffc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 0x56148ffcffc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 -- 172.21.15.45:0/2618939438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56148ff788d0 con 0x56148ffc2d20 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b21e9700 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2400 0x56148ffc27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.189+0000 7f57b19e8700 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffcd690 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005e90 tx=0x7f57a400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5105b3fb5a858b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.190+0000 7f5793fff700 1 -- 172.21.15.45:0/2618939438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a400f8d0 con 0x56148ffc2d20 2023-09-23T09:36:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.190+0000 7f5793fff700 1 -- 172.21.15.45:0/2618939438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a401d040 con 0x56148ffc2d20 2023-09-23T09:36:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.190+0000 7f5793fff700 1 -- 172.21.15.45:0/2618939438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4005640 con 0x56148ffc2d20 2023-09-23T09:36:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.190+0000 7f57aaffd700 1 -- 172.21.15.45:0/352165255 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f579c01a0d0 con 0x7f57ac10f940 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.190+0000 7f57aaffd700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f57940a1790 0x7f57940a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2618939438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 msgr2=0x56148ffcd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffcd690 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005e90 tx=0x7f57a400bef0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57aaffd700 1 -- 172.21.15.45:0/352165255 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f579c0dd8a0 con 0x7f57ac10f940 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2618939438 shutdown_connections 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 0x56148ffcffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2400 0x56148ffc27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2618939438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffcd690 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.191+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2618939438 >> 172.21.15.45:0/2618939438 conn(0x56148ff37260 msgr2=0x56148ffbe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:31.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.192+0000 7f57b21e9700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f57940a1790 0x7f57940a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.192+0000 7f57b21e9700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f57940a1790 0x7f57940a3c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57ac06a180 tx=0x7f57a0008040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:31.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.194+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2618939438 shutdown_connections 2023-09-23T09:36:31.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.195+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2618939438 wait complete. 2023-09-23T09:36:31.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.195+0000 7f57bfe36b80 1 -- start start 2023-09-23T09:36:31.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.195+0000 7f57bfe36b80 1 Processor -- start 2023-09-23T09:36:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.195+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffcd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.195+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffcaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ff15e30 con 0x56148ffc2d20 2023-09-23T09:36:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148feb2710 con 0x56148ffc2400 2023-09-23T09:36:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148feb23b0 con 0x56148ffcdbd0 2023-09-23T09:36:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57bfe36b80 10 client.0 _ll_register_callbacks cb 0x56148fd0f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b21e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffcaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34980/0 (socket says 172.21.15.45:34980) 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 -- 172.21.15.45:0/11524818 learned_addr learned my addr 172.21.15.45:0/11524818 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b19e8700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffcd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 -- 172.21.15.45:0/11524818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 msgr2=0x56148ffcaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffcaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 -- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 msgr2=0x56148ffcd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffcd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 -- 172.21.15.45:0/11524818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a000a570 con 0x56148ffc2d20 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b19e8700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffcd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b21e9700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffcaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57b11e7700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffca8b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f579c01ace0 tx=0x7f579c0aa260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf3c45705f205c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57917fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57917fa700 1 -- 172.21.15.45:0/11524818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c0ae950 con 0x56148ffc2d20 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57917fa700 1 -- 172.21.15.45:0/11524818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f579c0b2b00 con 0x56148ffc2d20 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.196+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56148ffb6250 con 0x56148ffc2d20 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57917fa700 1 -- 172.21.15.45:0/11524818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579c0a8ba0 con 0x56148ffc2d20 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 10 client.16296 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f57ac11d2f0 con 0x56148ffc2d20 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57917fa700 1 -- 172.21.15.45:0/11524818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56148ffb6250 con 0x56148ffc2d20 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57917fa700 1 -- 172.21.15.45:0/11524818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f579c0a9cf0 con 0x56148ffc2d20 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 20 client.16296 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56148ffdd560 0x56148ffdf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 4 client.16296 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56148ff0ee10 con 0x56148ffdd560 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.197+0000 7f57b19e8700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56148ffdd560 0x56148ffdf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.198+0000 7f57917fa700 10 client.16296 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:31.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.198+0000 7f57b19e8700 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56148ffdd560 0x56148ffdf980 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360bc32da4476e63 server_cookie=8728ea5101d3d5 in_seq=0 out_seq=0 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.199+0000 7f57917fa700 1 -- 172.21.15.45:0/11524818 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56148ff0ee10 con 0x56148ffdd560 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.199+0000 7f57917fa700 10 client.16296 handle_command_reply: tid=0 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.199+0000 7f57bfe36b80 1 client.16296 shutdown 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 msgr2=0x56148ffca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffca8b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f579c01ace0 tx=0x7f579c0aa260 comp rx=0 tx=0).stop 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 shutdown_connections 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56148ffdd560 0x56148ffdf980 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffcaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffcd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57917fa700 0 client.16296 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/11524818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffca8b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 >> 172.21.15.45:0/11524818 conn(0x56148ff37260 msgr2=0x56148ffbe380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 shutdown_connections 2023-09-23T09:36:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 1 -- 172.21.15.45:0/11524818 wait complete. 2023-09-23T09:36:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.202+0000 7f57bfe36b80 20 client.16296 trim_cache size 0 max 16384 2023-09-23T09:36:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 Processor -- start 2023-09-23T09:36:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 -- start start 2023-09-23T09:36:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffd46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffd8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffc5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148fd96920 con 0x56148ffc2d20 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ffbad90 con 0x56148ffc2400 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.227+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ffb6120 con 0x56148ffcdbd0 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffd8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffd46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffc5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffd46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56336/0 (socket says 172.21.15.45:56336) 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffd8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34988/0 (socket says 172.21.15.45:34988) 2023-09-23T09:36:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffc5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37096/0 (socket says 172.21.15.45:37096) 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b19e8700 1 -- 172.21.15.45:0/2758436715 learned_addr learned my addr 172.21.15.45:0/2758436715 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 -- 172.21.15.45:0/2758436715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 msgr2=0x56148ffd46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffd46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 -- 172.21.15.45:0/2758436715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 msgr2=0x56148ffd8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffd8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 -- 172.21.15.45:0/2758436715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56148fd40de0 con 0x56148ffcdbd0 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b19e8700 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffd46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b11e7700 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffd8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57b21e9700 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffc5bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57a000f5d0 tx=0x7f57a0012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=505263bd60dc2644 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.228+0000 7f57a8d86700 1 -- 172.21.15.45:0/2758436715 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0015bf0 con 0x56148ffcdbd0 2023-09-23T09:36:31.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57a8d86700 1 -- 172.21.15.45:0/2758436715 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a000c040 con 0x56148ffcdbd0 2023-09-23T09:36:31.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57a8d86700 1 -- 172.21.15.45:0/2758436715 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0013680 con 0x56148ffcdbd0 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2758436715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 msgr2=0x56148ffc5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffc5bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57a000f5d0 tx=0x7f57a0012000 comp rx=0 tx=0).stop 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2758436715 shutdown_connections 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffcdbd0 0x56148ffc5bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffc2400 0x56148ffd46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2758436715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2d20 0x56148ffd8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2758436715 >> 172.21.15.45:0/2758436715 conn(0x56148ff37260 msgr2=0x56148ffd99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2758436715 shutdown_connections 2023-09-23T09:36:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.229+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2758436715 wait complete. 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 -- start start 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 Processor -- start 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2400 0x56148ffd26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2d20 0x56148ffd2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 0x56148ffd3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ffd42f0 con 0x56148ffc2400 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ffd4460 con 0x56148ffcdbd0 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56148ffd4620 con 0x56148ffc2d20 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57bfe36b80 10 client.0 _ll_register_callbacks cb 0x7f579c0aa5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:31.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2400 0x56148ffd26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2d20 0x56148ffd2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57b19e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2400 0x56148ffd26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34992/0 (socket says 172.21.15.45:34992) 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57b19e8700 1 -- 172.21.15.45:0/2652873758 learned_addr learned my addr 172.21.15.45:0/2652873758 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57b11e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2d20 0x56148ffd2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37100/0 (socket says 172.21.15.45:37100) 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.230+0000 7f57b21e9700 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 0x56148ffd3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.231+0000 7f57b11e7700 1 -- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 msgr2=0x56148ffd3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.231+0000 7f57b11e7700 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 0x56148ffd3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.231+0000 7f57b11e7700 1 -- 172.21.15.45:0/2652873758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2400 msgr2=0x56148ffd26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.231+0000 7f57b11e7700 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2400 0x56148ffd26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.231+0000 7f57b11e7700 1 -- 172.21.15.45:0/2652873758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56148ff788d0 con 0x56148ffc2d20 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.231+0000 7f57b11e7700 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2d20 0x56148ffd2c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f579c0b25d0 tx=0x7f579c02d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b922aee244f531 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.231+0000 7f57917fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.233+0000 7f57917fa700 1 -- 172.21.15.45:0/2652873758 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c0aaf00 con 0x56148ffc2d20 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.233+0000 7f57917fa700 1 -- 172.21.15.45:0/2652873758 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f579c0b2b00 con 0x56148ffc2d20 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.233+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57a401d040 con 0x56148ffc2d20 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57917fa700 1 -- 172.21.15.45:0/2652873758 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579c0ae950 con 0x56148ffc2d20 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 10 client.25981 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f57ac11d2f0 con 0x56148ffc2d20 2023-09-23T09:36:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57917fa700 1 -- 172.21.15.45:0/2652873758 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a401d040 con 0x56148ffc2d20 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57917fa700 1 -- 172.21.15.45:0/2652873758 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f579c0c8030 con 0x56148ffc2d20 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 10 client.25981 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 20 client.25981 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561490032330 0x561490034750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 4 client.25981 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56148ff0eb30 con 0x561490032330 2023-09-23T09:36:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.234+0000 7f57b19e8700 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561490032330 0x561490034750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:31.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57917fa700 10 client.25981 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57b19e8700 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561490032330 0x561490034750 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c3b41339ba6e54 server_cookie=b657749ff5f85e7a in_seq=0 out_seq=0 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57917fa700 1 -- 172.21.15.45:0/2652873758 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56148ff0eb30 con 0x561490032330 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57917fa700 10 client.25981 handle_command_reply: tid=0 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57bfe36b80 1 client.25981 shutdown 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2d20 msgr2=0x56148ffd2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2d20 0x56148ffd2c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f579c0b25d0 tx=0x7f579c02d000 comp rx=0 tx=0).stop 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 shutdown_connections 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.235+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561490032330 0x561490034750 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56148ffcdbd0 0x56148ffd3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56148ffc2400 0x56148ffd26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57917fa700 0 client.25981 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57bfe36b80 1 --2- 172.21.15.45:0/2652873758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56148ffc2d20 0x56148ffd2c30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 >> 172.21.15.45:0/2652873758 conn(0x56148ff37260 msgr2=0x56148ffc5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 shutdown_connections 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57bfe36b80 1 -- 172.21.15.45:0/2652873758 wait complete. 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57bfe36b80 20 client.25981 trim_cache size 0 max 16384 2023-09-23T09:36:31.243 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:36:31.244 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:36:31.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f57940a1790 msgr2=0x7f57940a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f57940a1790 0x7f57940a3c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57ac06a180 tx=0x7f57a0008040 comp rx=0 tx=0).stop 2023-09-23T09:36:31.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f940 msgr2=0x7f57ac115ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:31.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f940 0x7f57ac115ee0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f579c01d340 tx=0x7f579c01d650 comp rx=0 tx=0).stop 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 shutdown_connections 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57ac11a600 0x7f57ac1a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57ac10f4d0 0x7f57ac1159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f57940a1790 0x7f57940a3c50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 --2- 172.21.15.45:0/352165255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57ac10f940 0x7f57ac115ee0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 >> 172.21.15.45:0/352165255 conn(0x7f57ac06d0e0 msgr2=0x7f57ac11bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.236+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 shutdown_connections 2023-09-23T09:36:31.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:36:31.237+0000 7f57b3c4c700 1 -- 172.21.15.45:0/352165255 wait complete. 2023-09-23T09:36:31.255 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:36:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 405.0 s, 1507.9 tps, lat 69.311 ms stddev 91.209 2023-09-23T09:36:32.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:31 smithi098 ceph-mon[102386]: pgmap v218: 97 pgs: 97 active+clean; 7.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 1.91k op/s 2023-09-23T09:36:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:31 smithi045 ceph-mon[98627]: pgmap v218: 97 pgs: 97 active+clean; 7.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 1.91k op/s 2023-09-23T09:36:32.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:31 smithi123 ceph-mon[103577]: pgmap v218: 97 pgs: 97 active+clean; 7.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 1.91k op/s 2023-09-23T09:36:34.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:33 smithi098 ceph-mon[102386]: pgmap v219: 97 pgs: 97 active+clean; 7.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2023-09-23T09:36:34.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:33 smithi045 ceph-mon[98627]: pgmap v219: 97 pgs: 97 active+clean; 7.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2023-09-23T09:36:34.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:33 smithi123 ceph-mon[103577]: pgmap v219: 97 pgs: 97 active+clean; 7.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2023-09-23T09:36:36.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:35 smithi098 ceph-mon[102386]: pgmap v220: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 34 MiB/s wr, 2.39k op/s 2023-09-23T09:36:36.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:35 smithi045 ceph-mon[98627]: pgmap v220: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 34 MiB/s wr, 2.39k op/s 2023-09-23T09:36:36.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:35 smithi123 ceph-mon[103577]: pgmap v220: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 34 MiB/s wr, 2.39k op/s 2023-09-23T09:36:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 410.0 s, 1626.0 tps, lat 59.496 ms stddev 70.554 2023-09-23T09:36:38.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:37 smithi098 ceph-mon[102386]: pgmap v221: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 2.12k op/s 2023-09-23T09:36:38.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:37 smithi045 ceph-mon[98627]: pgmap v221: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 2.12k op/s 2023-09-23T09:36:38.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:37 smithi123 ceph-mon[103577]: pgmap v221: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 2.12k op/s 2023-09-23T09:36:40.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:39 smithi098 ceph-mon[102386]: pgmap v222: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 35 MiB/s wr, 2.46k op/s 2023-09-23T09:36:40.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:39 smithi045 ceph-mon[98627]: pgmap v222: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 35 MiB/s wr, 2.46k op/s 2023-09-23T09:36:40.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:39 smithi123 ceph-mon[103577]: pgmap v222: 97 pgs: 97 active+clean; 7.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 35 MiB/s wr, 2.46k op/s 2023-09-23T09:36:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.804+0000 7f8292b8b700 1 -- 172.21.15.45:0/4240654243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10c1a0 msgr2=0x7f828c118660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.804+0000 7f8292b8b700 1 --2- 172.21.15.45:0/4240654243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10c1a0 0x7f828c118660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f827c009990 tx=0x7f827c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.804+0000 7f8292b8b700 1 -- 172.21.15.45:0/4240654243 shutdown_connections 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.804+0000 7f8292b8b700 1 --2- 172.21.15.45:0/4240654243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10c1a0 0x7f828c118660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.804+0000 7f8292b8b700 1 --2- 172.21.15.45:0/4240654243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 0x7f828c111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.804+0000 7f8292b8b700 1 --2- 172.21.15.45:0/4240654243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f828c10b7e0 0x7f828c10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.804+0000 7f8292b8b700 1 -- 172.21.15.45:0/4240654243 >> 172.21.15.45:0/4240654243 conn(0x7f828c06bbb0 msgr2=0x7f828c109e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.805+0000 7f8292b8b700 1 -- 172.21.15.45:0/4240654243 shutdown_connections 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.805+0000 7f8292b8b700 1 -- 172.21.15.45:0/4240654243 wait complete. 2023-09-23T09:36:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.806+0000 7f8292b8b700 1 Processor -- start 2023-09-23T09:36:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.806+0000 7f8292b8b700 1 -- start start 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8292b8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10b7e0 0x7f828c1a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8292b8b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f828c10c1a0 0x7f828c1a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8292b8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 0x7f828c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8292b8b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1a1c80 con 0x7f828c1117a0 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8292b8b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1a1df0 con 0x7f828c10c1a0 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8292b8b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1a1fb0 con 0x7f828c10b7e0 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8290927700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10b7e0 0x7f828c1a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8291128700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 0x7f828c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8291128700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 0x7f828c1a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35008/0 (socket says 172.21.15.45:35008) 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8290927700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10b7e0 0x7f828c1a77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37112/0 (socket says 172.21.15.45:37112) 2023-09-23T09:36:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8291128700 1 -- 172.21.15.45:0/2875540629 learned_addr learned my addr 172.21.15.45:0/2875540629 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f828bfff700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f828c10c1a0 0x7f828c1a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f828bfff700 1 -- 172.21.15.45:0/2875540629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10b7e0 msgr2=0x7f828c1a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f828bfff700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10b7e0 0x7f828c1a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f828bfff700 1 -- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 msgr2=0x7f828c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f828bfff700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 0x7f828c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f828bfff700 1 -- 172.21.15.45:0/2875540629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0095f0 con 0x7f828c10c1a0 2023-09-23T09:36:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f8291128700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 0x7f828c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.807+0000 7f828bfff700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f828c10c1a0 0x7f828c1a7cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f828400ef10 tx=0x7f828400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.809+0000 7f8289ffb700 1 -- 172.21.15.45:0/2875540629 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828400c9c0 con 0x7f828c10c1a0 2023-09-23T09:36:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.809+0000 7f8289ffb700 1 -- 172.21.15.45:0/2875540629 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f828400f040 con 0x7f828c10c1a0 2023-09-23T09:36:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.809+0000 7f8289ffb700 1 -- 172.21.15.45:0/2875540629 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284014480 con 0x7f828c10c1a0 2023-09-23T09:36:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.809+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c1a2230 con 0x7f828c10c1a0 2023-09-23T09:36:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.810+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c1ae7a0 con 0x7f828c10c1a0 2023-09-23T09:36:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.813+0000 7f8289ffb700 1 -- 172.21.15.45:0/2875540629 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f828400cb20 con 0x7f828c10c1a0 2023-09-23T09:36:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.814+0000 7f8289ffb700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82740a1860 0x7f82740a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.814+0000 7f8289ffb700 1 -- 172.21.15.45:0/2875540629 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f82840cd110 con 0x7f828c10c1a0 2023-09-23T09:36:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 Processor -- start 2023-09-23T09:36:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 -- start start 2023-09-23T09:36:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f8290927700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82740a1860 0x7f82740a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b11320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b27310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98a7a400 con 0x563a98b10f40 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98a6bbc0 con 0x563a98b24f20 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f829ed75b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98a694e0 con 0x563a98b11860 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f8290927700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b11320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f8290927700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b11320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35016/0 (socket says 172.21.15.45:35016) 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f8290927700 1 -- 172.21.15.45:0/4162064073 learned_addr learned my addr 172.21.15.45:0/4162064073 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f8291128700 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b27310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.815+0000 7f828bfff700 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.816+0000 7f828bfff700 1 -- 172.21.15.45:0/4162064073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 msgr2=0x563a98b27310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.816+0000 7f828bfff700 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b27310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.816+0000 7f828bfff700 1 -- 172.21.15.45:0/4162064073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 msgr2=0x563a98b11320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.816+0000 7f828bfff700 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b11320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.816+0000 7f8290927700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82740a1860 0x7f82740a3d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8280011630 tx=0x7f82800119f0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.816+0000 7f828bfff700 1 -- 172.21.15.45:0/4162064073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a98ac7870 con 0x563a98b11860 2023-09-23T09:36:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.816+0000 7f828bfff700 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b249e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f82840ca820 tx=0x7f8284099ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20eb8aad0c0d4213 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f8272ffd700 1 -- 172.21.15.45:0/4162064073 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828409a740 con 0x563a98b11860 2023-09-23T09:36:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f8272ffd700 1 -- 172.21.15.45:0/4162064073 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f828409a040 con 0x563a98b11860 2023-09-23T09:36:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f8272ffd700 1 -- 172.21.15.45:0/4162064073 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828409a2a0 con 0x563a98b11860 2023-09-23T09:36:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f829ed75b80 1 -- 172.21.15.45:0/4162064073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 msgr2=0x563a98b249e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f829ed75b80 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b249e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f82840ca820 tx=0x7f8284099ee0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f829ed75b80 1 -- 172.21.15.45:0/4162064073 shutdown_connections 2023-09-23T09:36:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f829ed75b80 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b27310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f829ed75b80 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b11320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.818+0000 7f829ed75b80 1 --2- 172.21.15.45:0/4162064073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b249e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.819+0000 7f829ed75b80 1 -- 172.21.15.45:0/4162064073 >> 172.21.15.45:0/4162064073 conn(0x563a98a86200 msgr2=0x563a98b0d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.819+0000 7f829ed75b80 1 -- 172.21.15.45:0/4162064073 shutdown_connections 2023-09-23T09:36:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.819+0000 7f829ed75b80 1 -- 172.21.15.45:0/4162064073 wait complete. 2023-09-23T09:36:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.819+0000 7f829ed75b80 1 -- start start 2023-09-23T09:36:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.819+0000 7f829ed75b80 1 Processor -- start 2023-09-23T09:36:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b11860 0x563a98b21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b22310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f829ed75b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98a64dd0 con 0x563a98b11860 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f829ed75b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98a016b0 con 0x563a98b24f20 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f829ed75b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98a01350 con 0x563a98b10f40 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f829ed75b80 10 client.0 _ll_register_callbacks cb 0x563a9885e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b249a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37126/0 (socket says 172.21.15.45:37126) 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 -- 172.21.15.45:0/1687604627 learned_addr learned my addr 172.21.15.45:0/1687604627 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f828bfff700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b11860 0x563a98b21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8291128700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b22310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 -- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 msgr2=0x563a98b22310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b22310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 -- 172.21.15.45:0/1687604627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b11860 msgr2=0x563a98b21c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b11860 0x563a98b21c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 -- 172.21.15.45:0/1687604627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c1ae7a0 con 0x563a98b10f40 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8291128700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b22310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f8290927700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b249a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8280004250 tx=0x7f82800047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bc23182057885c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.820+0000 7f826bfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f826bfff700 1 -- 172.21.15.45:0/1687604627 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280003e20 con 0x563a98b10f40 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a98b05090 con 0x563a98b10f40 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f826bfff700 1 -- 172.21.15.45:0/1687604627 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f828000f470 con 0x563a98b10f40 2023-09-23T09:36:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f826bfff700 1 -- 172.21.15.45:0/1687604627 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828000f630 con 0x563a98b10f40 2023-09-23T09:36:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f829ed75b80 10 client.25993 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563a9888fd80 con 0x563a98b10f40 2023-09-23T09:36:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f826bfff700 1 -- 172.21.15.45:0/1687604627 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a98b05090 con 0x563a98b10f40 2023-09-23T09:36:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.821+0000 7f826bfff700 1 -- 172.21.15.45:0/1687604627 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f82800208c0 con 0x563a98b10f40 2023-09-23T09:36:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.824+0000 7f829ed75b80 10 client.25993 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.824+0000 7f829ed75b80 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.824+0000 7f829ed75b80 20 client.25993 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.824+0000 7f829ed75b80 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b34980 0x563a98b36da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.824+0000 7f829ed75b80 4 client.25993 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.824+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a98a5ddb0 con 0x563a98b34980 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.824+0000 7f828bfff700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b34980 0x563a98b36da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.825+0000 7f828bfff700 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b34980 0x563a98b36da0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4175aae622afc65 server_cookie=7d834dbff8bf827d in_seq=0 out_seq=0 2023-09-23T09:36:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.825+0000 7f826bfff700 10 client.25993 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.826+0000 7f826bfff700 1 -- 172.21.15.45:0/1687604627 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a98a5ddb0 con 0x563a98b34980 2023-09-23T09:36:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.826+0000 7f826bfff700 10 client.25993 handle_command_reply: tid=0 2023-09-23T09:36:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.826+0000 7f829ed75b80 1 client.25993 shutdown 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 msgr2=0x563a98b249a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b249a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8280004250 tx=0x7f82800047c0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 shutdown_connections 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b34980 0x563a98b36da0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b22310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b249a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 --2- 172.21.15.45:0/1687604627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b11860 0x563a98b21c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 >> 172.21.15.45:0/1687604627 conn(0x563a98a86200 msgr2=0x563a98b0cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f826bfff700 0 client.25993 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 shutdown_connections 2023-09-23T09:36:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 1 -- 172.21.15.45:0/1687604627 wait complete. 2023-09-23T09:36:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.828+0000 7f829ed75b80 20 client.25993 trim_cache size 0 max 16384 2023-09-23T09:36:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.852+0000 7f829ed75b80 1 Processor -- start 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f829ed75b80 1 -- start start 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b30100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b1cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f829ed75b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a988e58c0 con 0x563a98b10f40 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f829ed75b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98b11f00 con 0x563a98b24f20 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f829ed75b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98b1da90 con 0x563a98b11860 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8291128700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b1cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8291128700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b1cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56380/0 (socket says 172.21.15.45:56380) 2023-09-23T09:36:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8291128700 1 -- 172.21.15.45:0/3282557294 learned_addr learned my addr 172.21.15.45:0/3282557294 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b30100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35028/0 (socket says 172.21.15.45:35028) 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 -- 172.21.15.45:0/3282557294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 msgr2=0x563a98b24540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 --2- 172.21.15.45:0/3282557294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 -- 172.21.15.45:0/3282557294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 msgr2=0x563a98b1cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 --2- 172.21.15.45:0/3282557294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b1cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 -- 172.21.15.45:0/3282557294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a98878c00 con 0x563a98b10f40 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f8290927700 1 --2- 172.21.15.45:0/3282557294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b30100 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f82800039e0 tx=0x7f8280005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4870f0b8bc0a650 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f82737fe700 1 -- 172.21.15.45:0/3282557294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280016050 con 0x563a98b10f40 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f82737fe700 1 -- 172.21.15.45:0/3282557294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f828002d070 con 0x563a98b10f40 2023-09-23T09:36:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.854+0000 7f82737fe700 1 -- 172.21.15.45:0/3282557294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828001f820 con 0x563a98b10f40 2023-09-23T09:36:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.855+0000 7f829ed75b80 1 -- 172.21.15.45:0/3282557294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 msgr2=0x563a98b30100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.855+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3282557294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b30100 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f82800039e0 tx=0x7f8280005dc0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.856+0000 7f829ed75b80 1 -- 172.21.15.45:0/3282557294 shutdown_connections 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.856+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3282557294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b24f20 0x563a98b1cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.856+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3282557294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b10f40 0x563a98b30100 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.856+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3282557294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b11860 0x563a98b24540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.856+0000 7f829ed75b80 1 -- 172.21.15.45:0/3282557294 >> 172.21.15.45:0/3282557294 conn(0x563a98a86200 msgr2=0x563a98b0baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.856+0000 7f829ed75b80 1 -- 172.21.15.45:0/3282557294 shutdown_connections 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.856+0000 7f829ed75b80 1 -- 172.21.15.45:0/3282557294 wait complete. 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.857+0000 7f829ed75b80 1 -- start start 2023-09-23T09:36:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.857+0000 7f829ed75b80 1 Processor -- start 2023-09-23T09:36:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b29990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b11860 0x563a98b29ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f829ed75b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b24f20 0x563a98b2a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f829ed75b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98b87770 con 0x563a98b24f20 2023-09-23T09:36:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f829ed75b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98b878b0 con 0x563a98b11860 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f829ed75b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a98b879f0 con 0x563a98b10f40 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f829ed75b80 10 client.0 _ll_register_callbacks cb 0x7f8280004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b29990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b29990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37140/0 (socket says 172.21.15.45:37140) 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 -- 172.21.15.45:0/3579514350 learned_addr learned my addr 172.21.15.45:0/3579514350 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8291128700 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b24f20 0x563a98b2a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f828bfff700 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b11860 0x563a98b29ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 -- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b11860 msgr2=0x563a98b29ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b11860 0x563a98b29ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 -- 172.21.15.45:0/3579514350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b24f20 msgr2=0x563a98b2a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b24f20 0x563a98b2a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.858+0000 7f8290927700 1 -- 172.21.15.45:0/3579514350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a9888fd80 con 0x563a98b10f40 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.859+0000 7f8290927700 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b29990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8280011ef0 tx=0x7f8280020a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e343bebfdc34dd1b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.859+0000 7f826bfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.859+0000 7f826bfff700 1 -- 172.21.15.45:0/3579514350 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280016070 con 0x563a98b10f40 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.859+0000 7f826bfff700 1 -- 172.21.15.45:0/3579514350 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8280020e40 con 0x563a98b10f40 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.859+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f828409a040 con 0x563a98b10f40 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.859+0000 7f826bfff700 1 -- 172.21.15.45:0/3579514350 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8280020650 con 0x563a98b10f40 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 10 client.25996 fetch_fsmap learned FSMap version 39 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563a988b61f0 con 0x563a98b10f40 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f826bfff700 1 -- 172.21.15.45:0/3579514350 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f828409a040 con 0x563a98b10f40 2023-09-23T09:36:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f826bfff700 1 -- 172.21.15.45:0/3579514350 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 39) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f82800053b0 con 0x563a98b10f40 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 10 client.25996 fetch_fsmap finished waiting for FSMap version 39 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 10 client.25996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 20 client.25996 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b88c20 0x563a98b8b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 4 client.25996 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.860+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a98a5dad0 con 0x563a98b88c20 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.861+0000 7f828bfff700 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b88c20 0x563a98b8b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.861+0000 7f826bfff700 10 client.25996 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:41.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.861+0000 7f828bfff700 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b88c20 0x563a98b8b040 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db65cd8e83a0c83a server_cookie=816bad6302dc17ea in_seq=0 out_seq=0 2023-09-23T09:36:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 415.0 s, 744.0 tps, lat 138.663 ms stddev 558.330 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f826bfff700 1 -- 172.21.15.45:0/3579514350 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a98a5dad0 con 0x563a98b88c20 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f826bfff700 10 client.25996 handle_command_reply: tid=0 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 client.25996 shutdown 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 msgr2=0x563a98b29990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b29990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8280011ef0 tx=0x7f8280020a30 comp rx=0 tx=0).stop 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 shutdown_connections 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563a98b88c20 0x563a98b8b040 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a98b24f20 0x563a98b2a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f826bfff700 0 client.25996 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a98b10f40 0x563a98b29990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 --2- 172.21.15.45:0/3579514350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a98b11860 0x563a98b29ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 >> 172.21.15.45:0/3579514350 conn(0x563a98a86200 msgr2=0x563a98b30100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 shutdown_connections 2023-09-23T09:36:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 1 -- 172.21.15.45:0/3579514350 wait complete. 2023-09-23T09:36:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:36:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:36:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:36:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:36:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.881+0000 7f829ed75b80 20 client.25996 trim_cache size 0 max 16384 2023-09-23T09:36:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82740a1860 msgr2=0x7f82740a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82740a1860 0x7f82740a3d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8280011630 tx=0x7f82800119f0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f828c10c1a0 msgr2=0x7f828c1a7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f828c10c1a0 0x7f828c1a7cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f828400ef10 tx=0x7f828400c3a0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 shutdown_connections 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82740a1860 0x7f82740a3d20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1117a0 0x7f828c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828c10b7e0 0x7f828c1a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.883+0000 7f8292b8b700 1 --2- 172.21.15.45:0/2875540629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f828c10c1a0 0x7f828c1a7cf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.884+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 >> 172.21.15.45:0/2875540629 conn(0x7f828c06bbb0 msgr2=0x7f828c10e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.884+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 shutdown_connections 2023-09-23T09:36:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:41.884+0000 7f8292b8b700 1 -- 172.21.15.45:0/2875540629 wait complete. 2023-09-23T09:36:42.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:41 smithi098 ceph-mon[102386]: pgmap v223: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 1.97k op/s 2023-09-23T09:36:42.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:41 smithi045 ceph-mon[98627]: pgmap v223: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 1.97k op/s 2023-09-23T09:36:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:41 smithi123 ceph-mon[103577]: pgmap v223: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 1.97k op/s 2023-09-23T09:36:43.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:42 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:36:43.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:42 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:36:43.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:42 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:36:44.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:43 smithi098 ceph-mon[102386]: pgmap v224: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2023-09-23T09:36:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:43 smithi045 ceph-mon[98627]: pgmap v224: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2023-09-23T09:36:44.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:43 smithi123 ceph-mon[103577]: pgmap v224: 97 pgs: 97 active+clean; 7.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2023-09-23T09:36:46.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:45 smithi098 ceph-mon[102386]: pgmap v225: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 22 MiB/s wr, 1.42k op/s 2023-09-23T09:36:46.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:45 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:36:46.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:45 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:36:46.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:45 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:36:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:45 smithi045 ceph-mon[98627]: pgmap v225: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 22 MiB/s wr, 1.42k op/s 2023-09-23T09:36:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:45 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:36:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:45 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:36:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:45 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:36:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:45 smithi123 ceph-mon[103577]: pgmap v225: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 22 MiB/s wr, 1.42k op/s 2023-09-23T09:36:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:45 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:36:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:45 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:36:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:45 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:36:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 420.0 s, 1426.8 tps, lat 69.666 ms stddev 83.681 2023-09-23T09:36:48.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:47 smithi098 ceph-mon[102386]: pgmap v226: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 809 op/s 2023-09-23T09:36:48.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:47 smithi045 ceph-mon[98627]: pgmap v226: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 809 op/s 2023-09-23T09:36:48.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:47 smithi123 ceph-mon[103577]: pgmap v226: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 809 op/s 2023-09-23T09:36:50.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:49 smithi098 ceph-mon[102386]: pgmap v227: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-09-23T09:36:50.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:49 smithi045 ceph-mon[98627]: pgmap v227: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-09-23T09:36:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:49 smithi123 ceph-mon[103577]: pgmap v227: 97 pgs: 97 active+clean; 7.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-09-23T09:36:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 425.0 s, 1640.8 tps, lat 60.770 ms stddev 70.087 2023-09-23T09:36:52.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:51 smithi098 ceph-mon[102386]: pgmap v228: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-09-23T09:36:52.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:51 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:52.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:51 smithi045 ceph-mon[98627]: pgmap v228: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-09-23T09:36:52.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:51 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:51 smithi123 ceph-mon[103577]: pgmap v228: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-09-23T09:36:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:51 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:54.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:53 smithi098 ceph-mon[102386]: pgmap v229: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-09-23T09:36:54.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:53 smithi045 ceph-mon[98627]: pgmap v229: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-09-23T09:36:54.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:53 smithi123 ceph-mon[103577]: pgmap v229: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-09-23T09:36:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d46d9d700 1 --2- 172.21.15.45:0/2754715580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 0x7f6d4810cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:36:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2754715580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100b10 msgr2=0x7f6d48104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2754715580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100b10 0x7f6d48104920 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0099c0 tx=0x7f6d3c009cd0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2754715580 shutdown_connections 2023-09-23T09:36:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2754715580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 0x7f6d4810cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2754715580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d48100160 0x7f6d48100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2754715580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100b10 0x7f6d48104920 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2754715580 >> 172.21.15.45:0/2754715580 conn(0x7f6d48076840 msgr2=0x7f6d48076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2754715580 shutdown_connections 2023-09-23T09:36:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.062+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2754715580 wait complete. 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 Processor -- start 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 -- start start 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100160 0x7f6d48199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 0x7f6d48199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d4819dcd0 0x7f6d4819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4819e750 con 0x7f6d48100160 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4819e8c0 con 0x7f6d48105050 2023-09-23T09:36:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.063+0000 7f6d4cf3d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4819ea80 con 0x7f6d4819dcd0 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100160 0x7f6d48199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100160 0x7f6d48199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35048/0 (socket says 172.21.15.45:35048) 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 -- 172.21.15.45:0/2345783535 learned_addr learned my addr 172.21.15.45:0/2345783535 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 0x7f6d48199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d46d9d700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d4819dcd0 0x7f6d4819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 -- 172.21.15.45:0/2345783535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d4819dcd0 msgr2=0x7f6d4819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d4819dcd0 0x7f6d4819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 -- 172.21.15.45:0/2345783535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 msgr2=0x7f6d48199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 0x7f6d48199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 -- 172.21.15.45:0/2345783535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d38009520 con 0x7f6d48100160 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 0x7f6d48199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.064+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100160 0x7f6d48199120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800eb30 tx=0x7f6d3800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.066+0000 7f6d377fe700 1 -- 172.21.15.45:0/2345783535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38009880 con 0x7f6d48100160 2023-09-23T09:36:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.066+0000 7f6d377fe700 1 -- 172.21.15.45:0/2345783535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d380044e0 con 0x7f6d48100160 2023-09-23T09:36:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.066+0000 7f6d377fe700 1 -- 172.21.15.45:0/2345783535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38003d80 con 0x7f6d48100160 2023-09-23T09:36:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.066+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3c0095f0 con 0x7f6d48100160 2023-09-23T09:36:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.066+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481a6220 con 0x7f6d48100160 2023-09-23T09:36:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.069+0000 7f6d377fe700 1 -- 172.21.15.45:0/2345783535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f6d38003f00 con 0x7f6d48100160 2023-09-23T09:36:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.070+0000 7f6d377fe700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6d300a1840 0x7f6d300a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.070+0000 7f6d377fe700 1 -- 172.21.15.45:0/2345783535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6d38014070 con 0x7f6d48100160 2023-09-23T09:36:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.070+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6d300a1840 0x7f6d300a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.070+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6d300a1840 0x7f6d300a3d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009990 tx=0x7f6d3c005fb0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.070+0000 7f6d59127b80 1 Processor -- start 2023-09-23T09:36:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d59127b80 1 -- start start 2023-09-23T09:36:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557342704710 0x557342706b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d59127b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557342662400 con 0x557342704710 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d59127b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557342653bc0 con 0x5573426f8f40 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d59127b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573426514e0 con 0x5573426f9860 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d46d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557342704710 0x557342706b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426f9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56406/0 (socket says 172.21.15.45:56406) 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 -- 172.21.15.45:0/2377205537 learned_addr learned my addr 172.21.15.45:0/2377205537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d46d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557342704710 0x557342706b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35060/0 (socket says 172.21.15.45:35060) 2023-09-23T09:36:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 -- 172.21.15.45:0/2377205537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 msgr2=0x5573427041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 -- 172.21.15.45:0/2377205537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557342704710 msgr2=0x557342706b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557342704710 0x557342706b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 -- 172.21.15.45:0/2377205537 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573426af870 con 0x5573426f8f40 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.071+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426f9320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d38097160 tx=0x7f6d3809a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f69759543a79df01 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.072+0000 7f6d34ff9700 1 -- 172.21.15.45:0/2377205537 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d380ba0f0 con 0x5573426f8f40 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.072+0000 7f6d34ff9700 1 -- 172.21.15.45:0/2377205537 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d3809a6f0 con 0x5573426f8f40 2023-09-23T09:36:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.072+0000 7f6d34ff9700 1 -- 172.21.15.45:0/2377205537 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38098b20 con 0x5573426f8f40 2023-09-23T09:36:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.072+0000 7f6d59127b80 1 -- 172.21.15.45:0/2377205537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 msgr2=0x5573426f9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.072+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426f9320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d38097160 tx=0x7f6d3809a0d0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.073+0000 7f6d59127b80 1 -- 172.21.15.45:0/2377205537 shutdown_connections 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.073+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557342704710 0x557342706b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.073+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426f9320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.073+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2377205537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.073+0000 7f6d59127b80 1 -- 172.21.15.45:0/2377205537 >> 172.21.15.45:0/2377205537 conn(0x55734266e200 msgr2=0x5573426f53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.073+0000 7f6d59127b80 1 -- 172.21.15.45:0/2377205537 shutdown_connections 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.073+0000 7f6d59127b80 1 -- 172.21.15.45:0/2377205537 wait complete. 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.074+0000 7f6d59127b80 1 -- start start 2023-09-23T09:36:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.074+0000 7f6d59127b80 1 Processor -- start 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.074+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f8f40 0x557342704140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.074+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557342704710 0x557342701b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d59127b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734264cdd0 con 0x5573426f8f40 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d59127b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573425e96b0 con 0x557342704710 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d59127b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573425e9350 con 0x5573426f9860 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d59127b80 10 client.0 _ll_register_callbacks cb 0x5573424467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f8f40 0x557342704140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d45d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d46d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557342704710 0x557342701b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f8f40 0x557342704140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35062/0 (socket says 172.21.15.45:35062) 2023-09-23T09:36:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 -- 172.21.15.45:0/2475294377 learned_addr learned my addr 172.21.15.45:0/2475294377 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 -- 172.21.15.45:0/2475294377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 msgr2=0x5573427013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 -- 172.21.15.45:0/2475294377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557342704710 msgr2=0x557342701b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557342704710 0x557342701b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 -- 172.21.15.45:0/2475294377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481a6220 con 0x5573426f8f40 2023-09-23T09:36:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d4659c700 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f8f40 0x557342704140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809a680 tx=0x7f6d3809a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6ec268198330c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:36:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d2dffb700 1 -- 172.21.15.45:0/2475294377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800cdd0 con 0x5573426f8f40 2023-09-23T09:36:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d2dffb700 1 -- 172.21.15.45:0/2475294377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6d38096e00 con 0x5573426f8f40 2023-09-23T09:36:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.075+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573426ed090 con 0x5573426f8f40 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d2dffb700 1 -- 172.21.15.45:0/2475294377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d38096fc0 con 0x5573426f8f40 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 10 client.16332 fetch_fsmap learned FSMap version 40 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557342477d80 con 0x5573426f8f40 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d2dffb700 1 -- 172.21.15.45:0/2475294377 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573426ed090 con 0x5573426f8f40 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d2dffb700 1 -- 172.21.15.45:0/2475294377 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6d38099be0 con 0x5573426f8f40 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 40 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 20 client.16332 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557342713ee0 0x5573427162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 4 client.16332 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:36:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.076+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557342645db0 con 0x557342713ee0 2023-09-23T09:36:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.077+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557342713ee0 0x5573427162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.078+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557342713ee0 0x5573427162d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a1e5072d0171c0 server_cookie=d2d6d29f5fe3e95d in_seq=0 out_seq=0 2023-09-23T09:36:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.078+0000 7f6d2dffb700 10 client.16332 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:55.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:54 smithi045 ceph-mon[98627]: pgmap v230: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:36:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d2dffb700 1 -- 172.21.15.45:0/2475294377 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557342645db0 con 0x557342713ee0 2023-09-23T09:36:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d2dffb700 10 client.16332 handle_command_reply: tid=0 2023-09-23T09:36:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 client.16332 shutdown 2023-09-23T09:36:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f8f40 msgr2=0x557342704140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f8f40 0x557342704140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809a680 tx=0x7f6d3809a550 comp rx=0 tx=0).stop 2023-09-23T09:36:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 shutdown_connections 2023-09-23T09:36:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557342713ee0 0x5573427162d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557342704710 0x557342701b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f8f40 0x557342704140 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d2dffb700 0 client.16332 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 --2- 172.21.15.45:0/2475294377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5573426f9860 0x5573427013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 >> 172.21.15.45:0/2475294377 conn(0x55734266e200 msgr2=0x5573426f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 shutdown_connections 2023-09-23T09:36:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 1 -- 172.21.15.45:0/2475294377 wait complete. 2023-09-23T09:36:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.079+0000 7f6d59127b80 20 client.16332 trim_cache size 0 max 16384 2023-09-23T09:36:55.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:54 smithi123 ceph-mon[103577]: pgmap v230: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:36:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.098+0000 7f6d59127b80 1 Processor -- start 2023-09-23T09:36:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.098+0000 7f6d59127b80 1 -- start start 2023-09-23T09:36:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x55734270b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342700ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d59127b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573424cd8c0 con 0x5573426f9860 2023-09-23T09:36:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d59127b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573426ecf80 con 0x5573426f8f40 2023-09-23T09:36:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d59127b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573426f9c40 con 0x557342704710 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d45d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x55734270b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d46d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342700ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d45d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x55734270b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35072/0 (socket says 172.21.15.45:35072) 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.099+0000 7f6d45d9b700 1 -- 172.21.15.45:0/4137254405 learned_addr learned my addr 172.21.15.45:0/4137254405 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d45d9b700 1 -- 172.21.15.45:0/4137254405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 msgr2=0x557342700ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342700ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d45d9b700 1 -- 172.21.15.45:0/4137254405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 msgr2=0x5573426fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d4659c700 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d45d9b700 1 -- 172.21.15.45:0/4137254405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557342460c00 con 0x5573426f9860 2023-09-23T09:36:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x55734270b260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0099c0 tx=0x7f6d3c01a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fbd773e40db14b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d357fa700 1 -- 172.21.15.45:0/4137254405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00bea0 con 0x5573426f9860 2023-09-23T09:36:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d357fa700 1 -- 172.21.15.45:0/4137254405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d3c01e040 con 0x5573426f9860 2023-09-23T09:36:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.100+0000 7f6d357fa700 1 -- 172.21.15.45:0/4137254405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c023460 con 0x5573426f9860 2023-09-23T09:36:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.101+0000 7f6d59127b80 1 -- 172.21.15.45:0/4137254405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 msgr2=0x55734270b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.101+0000 7f6d59127b80 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x55734270b260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0099c0 tx=0x7f6d3c01a070 comp rx=0 tx=0).stop 2023-09-23T09:36:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 -- 172.21.15.45:0/4137254405 shutdown_connections 2023-09-23T09:36:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342700ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x5573426fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 --2- 172.21.15.45:0/4137254405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x55734270b260 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 -- 172.21.15.45:0/4137254405 >> 172.21.15.45:0/4137254405 conn(0x55734266e200 msgr2=0x5573426f4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 -- 172.21.15.45:0/4137254405 shutdown_connections 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 -- 172.21.15.45:0/4137254405 wait complete. 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 -- start start 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.102+0000 7f6d59127b80 1 Processor -- start 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x55734270b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x5573427095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d59127b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342709b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d59127b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734270a210 con 0x5573426f9860 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d59127b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734270a350 con 0x5573426f8f40 2023-09-23T09:36:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d59127b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557342770500 con 0x557342704710 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d59127b80 10 client.0 _ll_register_callbacks cb 0x7f6d40009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d45d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x5573427095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d4659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x55734270b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d45d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x5573427095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35078/0 (socket says 172.21.15.45:35078) 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d45d9b700 1 -- 172.21.15.45:0/3062375085 learned_addr learned my addr 172.21.15.45:0/3062375085 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d46d9d700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342709b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d4659c700 1 -- 172.21.15.45:0/3062375085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 msgr2=0x557342709b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d4659c700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342709b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d4659c700 1 -- 172.21.15.45:0/3062375085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 msgr2=0x5573427095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d4659c700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x5573427095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d4659c700 1 -- 172.21.15.45:0/3062375085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4810d410 con 0x5573426f8f40 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x5573427095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.103+0000 7f6d46d9d700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342709b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d4659c700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x55734270b220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809a3f0 tx=0x7f6d38003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54df23cbb7d31aa0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d2dffb700 1 -- 172.21.15.45:0/3062375085 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38099de0 con 0x5573426f8f40 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d2dffb700 1 -- 172.21.15.45:0/3062375085 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6d3800cdd0 con 0x5573426f8f40 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d3809a6f0 con 0x5573426f8f40 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d2dffb700 1 -- 172.21.15.45:0/3062375085 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d380a2880 con 0x5573426f8f40 2023-09-23T09:36:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 10 client.26033 fetch_fsmap learned FSMap version 40 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55734249e1f0 con 0x5573426f8f40 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d2dffb700 1 -- 172.21.15.45:0/3062375085 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d3809a6f0 con 0x5573426f8f40 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d2dffb700 1 -- 172.21.15.45:0/3062375085 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6d380a8020 con 0x5573426f8f40 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 10 client.26033 fetch_fsmap finished waiting for FSMap version 40 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 20 client.26033 populate_metadata read hostname 'smithi045' 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573427715a0 0x557342773990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 4 client.26033 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.104+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557342645ad0 con 0x5573427715a0 2023-09-23T09:36:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.105+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573427715a0 0x557342773990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:36:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.105+0000 7f6d2dffb700 10 client.26033 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.105+0000 7f6d45d9b700 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573427715a0 0x557342773990 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08647089d2d9a66 server_cookie=b0863d3f3827b378 in_seq=0 out_seq=0 2023-09-23T09:36:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.174+0000 7f6d2dffb700 1 -- 172.21.15.45:0/3062375085 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557342645ad0 con 0x5573427715a0 2023-09-23T09:36:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.174+0000 7f6d2dffb700 10 client.26033 handle_command_reply: tid=0 2023-09-23T09:36:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.174+0000 7f6d59127b80 1 client.26033 shutdown 2023-09-23T09:36:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.175+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 msgr2=0x55734270b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.175+0000 7f6d59127b80 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x55734270b220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809a3f0 tx=0x7f6d38003820 comp rx=0 tx=0).stop 2023-09-23T09:36:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 shutdown_connections 2023-09-23T09:36:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5573427715a0 0x557342773990 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557342704710 0x557342709b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573426f8f40 0x55734270b220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 --2- 172.21.15.45:0/3062375085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573426f9860 0x5573427095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 >> 172.21.15.45:0/3062375085 conn(0x55734266e200 msgr2=0x5573426f4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d2dffb700 0 client.26033 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:36:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 shutdown_connections 2023-09-23T09:36:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.176+0000 7f6d59127b80 1 -- 172.21.15.45:0/3062375085 wait complete. 2023-09-23T09:36:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.177+0000 7f6d59127b80 20 client.26033 trim_cache size 0 max 16384 2023-09-23T09:36:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:36:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:36:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:36:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:36:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6d300a1840 msgr2=0x7f6d300a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6d300a1840 0x7f6d300a3d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009990 tx=0x7f6d3c005fb0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100160 msgr2=0x7f6d48199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:36:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100160 0x7f6d48199120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800eb30 tx=0x7f6d3800ee40 comp rx=0 tx=0).stop 2023-09-23T09:36:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 shutdown_connections 2023-09-23T09:36:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d4819dcd0 0x7f6d4819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6d300a1840 0x7f6d300a3d00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d48100160 0x7f6d48199120 secure :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800eb30 tx=0x7f6d3800ee40 comp rx=0 tx=0).stop 2023-09-23T09:36:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 --2- 172.21.15.45:0/2345783535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d48105050 0x7f6d48199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:36:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.178+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 >> 172.21.15.45:0/2345783535 conn(0x7f6d48076840 msgr2=0x7f6d480fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:36:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.180+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 shutdown_connections 2023-09-23T09:36:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:36:55.181+0000 7f6d4cf3d700 1 -- 172.21.15.45:0/2345783535 wait complete. 2023-09-23T09:36:55.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:54 smithi098 ceph-mon[102386]: pgmap v230: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:36:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 430.0 s, 2031.8 tps, lat 49.948 ms stddev 59.423 2023-09-23T09:36:57.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:57 smithi098 ceph-mon[102386]: pgmap v231: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:36:57.829 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:57 smithi045 ceph-mon[98627]: pgmap v231: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:36:57.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:57 smithi123 ceph-mon[103577]: pgmap v231: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:36:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:58 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:58.829 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:58 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:58.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:58 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:36:59.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:36:59 smithi098 ceph-mon[102386]: pgmap v232: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-09-23T09:36:59.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:36:59 smithi045 ceph-mon[98627]: pgmap v232: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-09-23T09:36:59.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:36:59 smithi123 ceph-mon[103577]: pgmap v232: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-09-23T09:37:01.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:37:01.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.519+0000 7fa7e511b700 1 -- 172.21.15.45:0/1565778167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e010f4d0 msgr2=0x7fa7e010d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.519+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1565778167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e010f4d0 0x7fa7e010d060 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800b0a0 tx=0x7fa7d800b3b0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.535+0000 7fa7e511b700 1 -- 172.21.15.45:0/1565778167 shutdown_connections 2023-09-23T09:37:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.535+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1565778167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e011a600 0x7fa7e011c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.535+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1565778167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e010f4d0 0x7fa7e010d060 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.535+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1565778167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e010f940 0x7fa7e010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.535+0000 7fa7e511b700 1 -- 172.21.15.45:0/1565778167 >> 172.21.15.45:0/1565778167 conn(0x7fa7e006d0e0 msgr2=0x7fa7e006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.542+0000 7fa7e511b700 1 -- 172.21.15.45:0/1565778167 shutdown_connections 2023-09-23T09:37:01.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:01 smithi045 ceph-mon[98627]: pgmap v233: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2023-09-23T09:37:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.551+0000 7fa7e511b700 1 -- 172.21.15.45:0/1565778167 wait complete. 2023-09-23T09:37:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.552+0000 7fa7e511b700 1 Processor -- start 2023-09-23T09:37:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.556+0000 7fa7e511b700 1 -- start start 2023-09-23T09:37:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.556+0000 7fa7e511b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e010f4d0 0x7fa7e01a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.556+0000 7fa7e511b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e010f940 0x7fa7e01a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.556+0000 7fa7e511b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e011a600 0x7fa7e01aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.557+0000 7fa7e511b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01aadb0 con 0x7fa7e011a600 2023-09-23T09:37:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.557+0000 7fa7e511b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01aaf20 con 0x7fa7e010f940 2023-09-23T09:37:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.557+0000 7fa7e511b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01ab0e0 con 0x7fa7e010f4d0 2023-09-23T09:37:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.557+0000 7fa7ded9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e010f4d0 0x7fa7e01a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.557+0000 7fa7ded9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e010f4d0 0x7fa7e01a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37198/0 (socket says 172.21.15.45:37198) 2023-09-23T09:37:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.557+0000 7fa7ded9d700 1 -- 172.21.15.45:0/1730722200 learned_addr learned my addr 172.21.15.45:0/1730722200 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.565+0000 7fa7ded9d700 1 -- 172.21.15.45:0/1730722200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e010f940 msgr2=0x7fa7e01a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.565+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e010f940 0x7fa7e01a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.565+0000 7fa7ded9d700 1 -- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e011a600 msgr2=0x7fa7e01aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.565+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e011a600 0x7fa7e01aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.566+0000 7fa7ded9d700 1 -- 172.21.15.45:0/1730722200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8009d00 con 0x7fa7e010f4d0 2023-09-23T09:37:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.566+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e010f4d0 0x7fa7e01a5dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8003f80 tx=0x7fa7d8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:01.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.580+0000 7fa7c7fff700 1 -- 172.21.15.45:0/1730722200 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800e050 con 0x7fa7e010f4d0 2023-09-23T09:37:01.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.580+0000 7fa7c7fff700 1 -- 172.21.15.45:0/1730722200 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d8005570 con 0x7fa7e010f4d0 2023-09-23T09:37:01.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.580+0000 7fa7c7fff700 1 -- 172.21.15.45:0/1730722200 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d801ba40 con 0x7fa7e010f4d0 2023-09-23T09:37:01.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.581+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e01ab360 con 0x7fa7e010f4d0 2023-09-23T09:37:01.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.581+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e010ffc0 con 0x7fa7e010f4d0 2023-09-23T09:37:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.591+0000 7fa7f1305b80 1 Processor -- start 2023-09-23T09:37:01.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.592+0000 7fa7c7fff700 1 -- 172.21.15.45:0/1730722200 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fa7d80056e0 con 0x7fa7e010f4d0 2023-09-23T09:37:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.593+0000 7fa7c7fff700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7c80a1920 0x7fa7c80a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.593+0000 7fa7c7fff700 1 -- 172.21.15.45:0/1730722200 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7d80ceca0 con 0x7fa7e010f4d0 2023-09-23T09:37:01.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.596+0000 7fa7f1305b80 1 -- start start 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.596+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.596+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 0x55a6bccbc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.596+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccbefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.596+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bcc1a460 con 0x55a6bccb1400 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.596+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bcc0bc20 con 0x55a6bccbcbd0 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.596+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bcc09540 con 0x55a6bccb1d20 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.597+0000 7fa7df59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccbefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.597+0000 7fa7df59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccbefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56448/0 (socket says 172.21.15.45:56448) 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.597+0000 7fa7df59e700 1 -- 172.21.15.45:0/1385309453 learned_addr learned my addr 172.21.15.45:0/1385309453 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.598+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.600+0000 7fa7ded9d700 1 -- 172.21.15.45:0/1385309453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 msgr2=0x55a6bccbc690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.600+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 0x55a6bccbc690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.600+0000 7fa7ded9d700 1 -- 172.21.15.45:0/1385309453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 msgr2=0x55a6bccbefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.600+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccbefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.600+0000 7fa7ded9d700 1 -- 172.21.15.45:0/1385309453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6bcc678d0 con 0x55a6bccb1400 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.601+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb17e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80cc350 tx=0x7fa7d801b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e2b9b4f0951792 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.601+0000 7fa7c57fa700 1 -- 172.21.15.45:0/1385309453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d809bc00 con 0x55a6bccb1400 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.601+0000 7fa7c57fa700 1 -- 172.21.15.45:0/1385309453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d809fd50 con 0x55a6bccb1400 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.601+0000 7fa7c57fa700 1 -- 172.21.15.45:0/1385309453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d809c060 con 0x55a6bccb1400 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.602+0000 7fa7f1305b80 1 -- 172.21.15.45:0/1385309453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 msgr2=0x55a6bccb17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.602+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb17e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80cc350 tx=0x7fa7d801b450 comp rx=0 tx=0).stop 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.605+0000 7fa7df59e700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7c80a1920 0x7fa7c80a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.605+0000 7fa7f1305b80 1 -- 172.21.15.45:0/1385309453 shutdown_connections 2023-09-23T09:37:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.605+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccbefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.605+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb17e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.605+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/1385309453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 0x55a6bccbc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.605+0000 7fa7f1305b80 1 -- 172.21.15.45:0/1385309453 >> 172.21.15.45:0/1385309453 conn(0x55a6bcc26260 msgr2=0x55a6bccad790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.606+0000 7fa7df59e700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7c80a1920 0x7fa7c80a3de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009fb0 tx=0x7fa7d0009440 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.611+0000 7fa7f1305b80 1 -- 172.21.15.45:0/1385309453 shutdown_connections 2023-09-23T09:37:01.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.618+0000 7fa7f1305b80 1 -- 172.21.15.45:0/1385309453 wait complete. 2023-09-23T09:37:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.619+0000 7fa7f1305b80 1 -- start start 2023-09-23T09:37:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.619+0000 7fa7f1305b80 1 Processor -- start 2023-09-23T09:37:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1400 0x55a6bccbcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1d20 0x55a6bccbd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bcc04e30 con 0x55a6bccbcbd0 2023-09-23T09:37:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bcba1710 con 0x55a6bccb1d20 2023-09-23T09:37:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bcba13b0 con 0x55a6bccb1400 2023-09-23T09:37:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7f1305b80 10 client.0 _ll_register_callbacks cb 0x55a6bc9fe800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7df59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7df59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35104/0 (socket says 172.21.15.45:35104) 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.620+0000 7fa7df59e700 1 -- 172.21.15.45:0/2672038147 learned_addr learned my addr 172.21.15.45:0/2672038147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.621+0000 7fa7df59e700 1 -- 172.21.15.45:0/2672038147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1400 msgr2=0x55a6bccbcfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.621+0000 7fa7df59e700 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1400 0x55a6bccbcfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.621+0000 7fa7df59e700 1 -- 172.21.15.45:0/2672038147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1d20 msgr2=0x55a6bccbd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.621+0000 7fa7df59e700 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1d20 0x55a6bccbd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.621+0000 7fa7df59e700 1 -- 172.21.15.45:0/2672038147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e010ffc0 con 0x55a6bccbcbd0 2023-09-23T09:37:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.621+0000 7fa7df59e700 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc8030 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0010910 tx=0x7fa7d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b28b7d04ff8a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.626+0000 7fa7c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.626+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/2672038147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000f850 con 0x55a6bccbcbd0 2023-09-23T09:37:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.626+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/2672038147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7d000f9b0 con 0x55a6bccbcbd0 2023-09-23T09:37:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.628+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6bcca5250 con 0x55a6bccbcbd0 2023-09-23T09:37:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.631+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/2672038147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7d000fb70 con 0x55a6bccbcbd0 2023-09-23T09:37:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.639+0000 7fa7f1305b80 10 client.16347 fetch_fsmap learned FSMap version 41 2023-09-23T09:37:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.639+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a6bca2fde0 con 0x55a6bccbcbd0 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/2672038147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6bcca5250 con 0x55a6bccbcbd0 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/2672038147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa7d000e3b0 con 0x55a6bccbcbd0 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7f1305b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 41 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7f1305b80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7f1305b80 20 client.16347 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcccc460 0x55a6bccce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7f1305b80 4 client.16347 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:37:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.640+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6bcbfde10 con 0x55a6bcccc460 2023-09-23T09:37:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.641+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcccc460 0x55a6bccce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.641+0000 7fa7c2ff5700 10 client.16347 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.641+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcccc460 0x55a6bccce880 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890f61277d2ba40d server_cookie=4339ee07138dae94 in_seq=0 out_seq=0 2023-09-23T09:37:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.648+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/2672038147 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a6bcbfde10 con 0x55a6bcccc460 2023-09-23T09:37:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.648+0000 7fa7c2ff5700 10 client.16347 handle_command_reply: tid=0 2023-09-23T09:37:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.648+0000 7fa7f1305b80 1 client.16347 shutdown 2023-09-23T09:37:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.652+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccbcbd0 msgr2=0x55a6bccc8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.652+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc8030 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0010910 tx=0x7fa7d0009260 comp rx=0 tx=0).stop 2023-09-23T09:37:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.663+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 shutdown_connections 2023-09-23T09:37:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.663+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcccc460 0x55a6bccce880 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.663+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc8030 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.663+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1400 0x55a6bccbcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.663+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/2672038147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1d20 0x55a6bccbd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.663+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 >> 172.21.15.45:0/2672038147 conn(0x55a6bcc26260 msgr2=0x55a6bccad320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.663+0000 7fa7c2ff5700 0 client.16347 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.670+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 shutdown_connections 2023-09-23T09:37:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.672+0000 7fa7f1305b80 1 -- 172.21.15.45:0/2672038147 wait complete. 2023-09-23T09:37:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.672+0000 7fa7f1305b80 20 client.16347 trim_cache size 0 max 16384 2023-09-23T09:37:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.713+0000 7fa7f1305b80 1 Processor -- start 2023-09-23T09:37:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.715+0000 7fa7f1305b80 1 -- start start 2023-09-23T09:37:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.715+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.715+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 0x55a6bccb76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.715+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.715+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bca85920 con 0x55a6bccb1400 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.715+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bccbbd30 con 0x55a6bccbcbd0 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.715+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bcca7860 con 0x55a6bccb1d20 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35108/0 (socket says 172.21.15.45:35108) 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 -- 172.21.15.45:0/865178312 learned_addr learned my addr 172.21.15.45:0/865178312 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 -- 172.21.15.45:0/865178312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 msgr2=0x55a6bccb76a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:37:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/865178312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 0x55a6bccb76a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 -- 172.21.15.45:0/865178312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 msgr2=0x55a6bccc7720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:37:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/865178312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.716+0000 7fa7ded9d700 1 -- 172.21.15.45:0/865178312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6bcca9480 con 0x55a6bccb1400 2023-09-23T09:37:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.717+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/865178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb7160 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa7d809b900 tx=0x7fa7d809b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822f812670d41810 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.728+0000 7fa7c5ffb700 1 -- 172.21.15.45:0/865178312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80a3f00 con 0x55a6bccb1400 2023-09-23T09:37:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.728+0000 7fa7c5ffb700 1 -- 172.21.15.45:0/865178312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d8099d40 con 0x55a6bccb1400 2023-09-23T09:37:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.728+0000 7fa7c5ffb700 1 -- 172.21.15.45:0/865178312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d809be60 con 0x55a6bccb1400 2023-09-23T09:37:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.729+0000 7fa7f1305b80 1 -- 172.21.15.45:0/865178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 msgr2=0x55a6bccb7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.729+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/865178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb7160 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa7d809b900 tx=0x7fa7d809b9d0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.734+0000 7fa7f1305b80 1 -- 172.21.15.45:0/865178312 shutdown_connections 2023-09-23T09:37:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.734+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/865178312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.734+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/865178312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1400 0x55a6bccb7160 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.734+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/865178312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccb1d20 0x55a6bccb76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.734+0000 7fa7f1305b80 1 -- 172.21.15.45:0/865178312 >> 172.21.15.45:0/865178312 conn(0x55a6bcc26260 msgr2=0x55a6bccb65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.735+0000 7fa7f1305b80 1 -- 172.21.15.45:0/865178312 shutdown_connections 2023-09-23T09:37:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.735+0000 7fa7f1305b80 1 -- 172.21.15.45:0/865178312 wait complete. 2023-09-23T09:37:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 -- start start 2023-09-23T09:37:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 Processor -- start 2023-09-23T09:37:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1400 0x55a6bccc0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1d20 0x55a6bccc0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bccc1de0 con 0x55a6bccb1d20 2023-09-23T09:37:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bccc1f50 con 0x55a6bccb1400 2023-09-23T09:37:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bccc2110 con 0x55a6bccbcbd0 2023-09-23T09:37:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.736+0000 7fa7f1305b80 10 client.0 _ll_register_callbacks cb 0x7fa7d00104d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1d20 0x55a6bccc0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1d20 0x55a6bccc0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35112/0 (socket says 172.21.15.45:35112) 2023-09-23T09:37:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 -- 172.21.15.45:0/943410313 learned_addr learned my addr 172.21.15.45:0/943410313 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 -- 172.21.15.45:0/943410313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccbcbd0 msgr2=0x55a6bccc16a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:37:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc16a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 -- 172.21.15.45:0/943410313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1400 msgr2=0x55a6bccc0210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:37:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1400 0x55a6bccc0210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.737+0000 7fa7de59c700 1 -- 172.21.15.45:0/943410313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6bcc678d0 con 0x55a6bccb1d20 2023-09-23T09:37:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.738+0000 7fa7de59c700 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1d20 0x55a6bccc0750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009d40 tx=0x7fa7d400b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fe12b1c905fae2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.738+0000 7fa7c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.738+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/943410313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400bde0 con 0x55a6bccb1d20 2023-09-23T09:37:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.738+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/943410313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7d4011420 con 0x55a6bccb1d20 2023-09-23T09:37:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.739+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7d809fd50 con 0x55a6bccb1d20 2023-09-23T09:37:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.739+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/943410313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7d40115e0 con 0x55a6bccb1d20 2023-09-23T09:37:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.742+0000 7fa7f1305b80 10 client.16353 fetch_fsmap learned FSMap version 41 2023-09-23T09:37:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.742+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a6bca18c60 con 0x55a6bccb1d20 2023-09-23T09:37:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/943410313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d809fd50 con 0x55a6bccb1d20 2023-09-23T09:37:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/943410313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa7d401a8e0 con 0x55a6bccb1d20 2023-09-23T09:37:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7f1305b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 41 2023-09-23T09:37:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7f1305b80 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7f1305b80 20 client.16353 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcd216d0 0x55a6bcd23af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7f1305b80 4 client.16353 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:37:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.743+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6bcbfdb30 con 0x55a6bcd216d0 2023-09-23T09:37:01.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.747+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcd216d0 0x55a6bcd23af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:01.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.774+0000 7fa7ded9d700 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcd216d0 0x55a6bcd23af0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c18f045622fcc5 server_cookie=a1129810933318ae in_seq=0 out_seq=0 2023-09-23T09:37:01.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.779+0000 7fa7c2ff5700 10 client.16353 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:01.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.780+0000 7fa7c2ff5700 1 -- 172.21.15.45:0/943410313 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a6bcbfdb30 con 0x55a6bcd216d0 2023-09-23T09:37:01.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.780+0000 7fa7c2ff5700 10 client.16353 handle_command_reply: tid=0 2023-09-23T09:37:01.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.780+0000 7fa7f1305b80 1 client.16353 shutdown 2023-09-23T09:37:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.785+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1d20 msgr2=0x55a6bccc0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.785+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1d20 0x55a6bccc0750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009d40 tx=0x7fa7d400b3c0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.790+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 shutdown_connections 2023-09-23T09:37:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.790+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a6bcd216d0 0x55a6bcd23af0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.790+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6bccbcbd0 0x55a6bccc16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.790+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6bccb1400 0x55a6bccc0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.790+0000 7fa7f1305b80 1 --2- 172.21.15.45:0/943410313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6bccb1d20 0x55a6bccc0750 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.790+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 >> 172.21.15.45:0/943410313 conn(0x55a6bcc26260 msgr2=0x55a6bccb7160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.794+0000 7fa7c2ff5700 0 client.16353 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.794+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 shutdown_connections 2023-09-23T09:37:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.805+0000 7fa7f1305b80 1 -- 172.21.15.45:0/943410313 wait complete. 2023-09-23T09:37:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.806+0000 7fa7f1305b80 20 client.16353 trim_cache size 0 max 16384 2023-09-23T09:37:01.814 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:37:01.814 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:37:01.814 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:37:01.814 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:37:01.814 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:37:01.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:37:01.815 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:37:01.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:37:01.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:37:01.815 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:37:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.820+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7c80a1920 msgr2=0x7fa7c80a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.820+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7c80a1920 0x7fa7c80a3de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009fb0 tx=0x7fa7d0009440 comp rx=0 tx=0).stop 2023-09-23T09:37:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.820+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e010f4d0 msgr2=0x7fa7e01a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.820+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e010f4d0 0x7fa7e01a5dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8003f80 tx=0x7fa7d8009500 comp rx=0 tx=0).stop 2023-09-23T09:37:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.829+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 shutdown_connections 2023-09-23T09:37:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.829+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7c80a1920 0x7fa7c80a3de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.829+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e011a600 0x7fa7e01aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.829+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7e010f4d0 0x7fa7e01a5dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.829+0000 7fa7e511b700 1 --2- 172.21.15.45:0/1730722200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7e010f940 0x7fa7e01a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.829+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 >> 172.21.15.45:0/1730722200 conn(0x7fa7e006d0e0 msgr2=0x7fa7e01185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:01.840 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:01 smithi123 ceph-mon[103577]: pgmap v233: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2023-09-23T09:37:01.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.836+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 shutdown_connections 2023-09-23T09:37:01.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:01.836+0000 7fa7e511b700 1 -- 172.21.15.45:0/1730722200 wait complete. 2023-09-23T09:37:01.872 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:37:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 435.0 s, 1637.6 tps, lat 55.146 ms stddev 67.128 2023-09-23T09:37:02.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:01 smithi098 ceph-mon[102386]: pgmap v233: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2023-09-23T09:37:04.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:03 smithi098 ceph-mon[102386]: pgmap v234: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 13 MiB/s wr, 752 op/s 2023-09-23T09:37:04.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:03 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:37:04.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:03 smithi045 ceph-mon[98627]: pgmap v234: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 13 MiB/s wr, 752 op/s 2023-09-23T09:37:04.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:03 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:37:04.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:03 smithi123 ceph-mon[103577]: pgmap v234: 97 pgs: 97 active+clean; 7.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 13 MiB/s wr, 752 op/s 2023-09-23T09:37:04.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:03 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:37:06.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:05 smithi098 ceph-mon[102386]: pgmap v235: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 27 MiB/s wr, 1.91k op/s 2023-09-23T09:37:06.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:05 smithi045 ceph-mon[98627]: pgmap v235: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 27 MiB/s wr, 1.91k op/s 2023-09-23T09:37:06.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:05 smithi123 ceph-mon[103577]: pgmap v235: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 27 MiB/s wr, 1.91k op/s 2023-09-23T09:37:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 440.0 s, 1510.6 tps, lat 72.178 ms stddev 113.683 2023-09-23T09:37:08.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:07 smithi098 ceph-mon[102386]: pgmap v236: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:37:08.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:07 smithi045 ceph-mon[98627]: pgmap v236: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:37:08.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:07 smithi123 ceph-mon[103577]: pgmap v236: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:37:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.334+0000 7f33ff443700 1 -- 172.21.15.45:0/781929827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 msgr2=0x7f33f8106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.334+0000 7f33ff443700 1 --2- 172.21.15.45:0/781929827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 0x7f33f8106180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f33f40099c0 tx=0x7f33f4009cd0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 -- 172.21.15.45:0/781929827 shutdown_connections 2023-09-23T09:37:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 --2- 172.21.15.45:0/781929827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f8068bd0 0x7f33f810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 --2- 172.21.15.45:0/781929827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 0x7f33f8106180 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 --2- 172.21.15.45:0/781929827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33f8068210 0x7f33f8068690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 -- 172.21.15.45:0/781929827 >> 172.21.15.45:0/781929827 conn(0x7f33f8076840 msgr2=0x7f33f8076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 -- 172.21.15.45:0/781929827 shutdown_connections 2023-09-23T09:37:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 -- 172.21.15.45:0/781929827 wait complete. 2023-09-23T09:37:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 Processor -- start 2023-09-23T09:37:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.335+0000 7f33ff443700 1 -- start start 2023-09-23T09:37:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33ff443700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f8068210 0x7f33f80fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33ff443700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33f8068bd0 0x7f33f8100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33ff443700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 0x7f33f8100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33ff443700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f81a9850 con 0x7f33f8105da0 2023-09-23T09:37:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33fc9de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33f8068bd0 0x7f33f8100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33ff443700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819af70 con 0x7f33f8068210 2023-09-23T09:37:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33ff443700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819b130 con 0x7f33f8068bd0 2023-09-23T09:37:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33fc9de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33f8068bd0 0x7f33f8100500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37230/0 (socket says 172.21.15.45:37230) 2023-09-23T09:37:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33fc9de700 1 -- 172.21.15.45:0/2761163783 learned_addr learned my addr 172.21.15.45:0/2761163783 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.336+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 0x7f33f8100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.337+0000 7f33fd1df700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f8068210 0x7f33f80fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.337+0000 7f33fd9e0700 1 -- 172.21.15.45:0/2761163783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33f8068bd0 msgr2=0x7f33f8100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.337+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33f8068bd0 0x7f33f8100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.337+0000 7f33fd9e0700 1 -- 172.21.15.45:0/2761163783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f8068210 msgr2=0x7f33f80fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.337+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f8068210 0x7f33f80fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.337+0000 7f33fd9e0700 1 -- 172.21.15.45:0/2761163783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f40095f0 con 0x7f33f8105da0 2023-09-23T09:37:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.338+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 0x7f33f8100a40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f33f000efc0 tx=0x7f33f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.338+0000 7f33ee7fc700 1 -- 172.21.15.45:0/2761163783 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0018070 con 0x7f33f8105da0 2023-09-23T09:37:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.338+0000 7f33ee7fc700 1 -- 172.21.15.45:0/2761163783 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f000f040 con 0x7f33f8105da0 2023-09-23T09:37:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.338+0000 7f33ee7fc700 1 -- 172.21.15.45:0/2761163783 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000cdf0 con 0x7f33f8105da0 2023-09-23T09:37:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.339+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f819b3b0 con 0x7f33f8105da0 2023-09-23T09:37:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.339+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f819b870 con 0x7f33f8105da0 2023-09-23T09:37:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.354+0000 7f33ee7fc700 1 -- 172.21.15.45:0/2761163783 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f33f0003fc0 con 0x7f33f8105da0 2023-09-23T09:37:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.354+0000 7f340b62db80 1 Processor -- start 2023-09-23T09:37:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.355+0000 7f33ee7fc700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f33e40a18e0 0x7f33e40a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.355+0000 7f33ee7fc700 1 -- 172.21.15.45:0/2761163783 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f33f00cdd90 con 0x7f33f8105da0 2023-09-23T09:37:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.357+0000 7f33fc9de700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f33e40a18e0 0x7f33e40a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.357+0000 7f33fc9de700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f33e40a18e0 0x7f33e40a3da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f33f81019c0 tx=0x7f33e8009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f340b62db80 1 -- start start 2023-09-23T09:37:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f340b62db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7100f40 0x5559a7101320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f340b62db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7101860 0x5559a710c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f340b62db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a710eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f340b62db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a706a400 con 0x5559a7101860 2023-09-23T09:37:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f340b62db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a705bbc0 con 0x5559a7100f40 2023-09-23T09:37:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f340b62db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a70594e0 con 0x5559a710c710 2023-09-23T09:37:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f33fd1df700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7100f40 0x5559a7101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f33fc9de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7101860 0x5559a710c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f33fd1df700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7100f40 0x5559a7101320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56482/0 (socket says 172.21.15.45:56482) 2023-09-23T09:37:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.363+0000 7f33fd1df700 1 -- 172.21.15.45:0/558924692 learned_addr learned my addr 172.21.15.45:0/558924692 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.364+0000 7f33fc9de700 1 -- 172.21.15.45:0/558924692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 msgr2=0x5559a710eb00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:37:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.364+0000 7f33fc9de700 1 --2- 172.21.15.45:0/558924692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a710eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.364+0000 7f33fc9de700 1 -- 172.21.15.45:0/558924692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7100f40 msgr2=0x5559a7101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.364+0000 7f33fc9de700 1 --2- 172.21.15.45:0/558924692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7100f40 0x5559a7101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.364+0000 7f33fc9de700 1 -- 172.21.15.45:0/558924692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559a70b7870 con 0x5559a7101860 2023-09-23T09:37:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.364+0000 7f33fc9de700 1 --2- 172.21.15.45:0/558924692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7101860 0x5559a710c1d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f33e8011ab0 tx=0x7f33e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb1cd483d7544c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.368+0000 7f33e37fe700 1 -- 172.21.15.45:0/558924692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8028480 con 0x5559a7101860 2023-09-23T09:37:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.369+0000 7f33e37fe700 1 -- 172.21.15.45:0/558924692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33e8021070 con 0x5559a7101860 2023-09-23T09:37:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.369+0000 7f33e37fe700 1 -- 172.21.15.45:0/558924692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e801d950 con 0x5559a7101860 2023-09-23T09:37:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.369+0000 7f340b62db80 1 -- 172.21.15.45:0/558924692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7101860 msgr2=0x5559a710c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 --2- 172.21.15.45:0/558924692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7101860 0x5559a710c1d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f33e8011ab0 tx=0x7f33e8014000 comp rx=0 tx=0).stop 2023-09-23T09:37:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 -- 172.21.15.45:0/558924692 shutdown_connections 2023-09-23T09:37:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 --2- 172.21.15.45:0/558924692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a710eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 --2- 172.21.15.45:0/558924692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7100f40 0x5559a7101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 --2- 172.21.15.45:0/558924692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7101860 0x5559a710c1d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 -- 172.21.15.45:0/558924692 >> 172.21.15.45:0/558924692 conn(0x5559a7076200 msgr2=0x5559a70fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 -- 172.21.15.45:0/558924692 shutdown_connections 2023-09-23T09:37:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 -- 172.21.15.45:0/558924692 wait complete. 2023-09-23T09:37:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 -- start start 2023-09-23T09:37:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.370+0000 7f340b62db80 1 Processor -- start 2023-09-23T09:37:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f340b62db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a7100f40 0x5559a7109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f340b62db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a71099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f340b62db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a710c710 0x5559a7109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f340b62db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a7054dd0 con 0x5559a710c710 2023-09-23T09:37:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f340b62db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6ff16b0 con 0x5559a7101860 2023-09-23T09:37:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f340b62db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6ff1350 con 0x5559a7100f40 2023-09-23T09:37:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f340b62db80 10 client.0 _ll_register_callbacks cb 0x5559a6e4e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f33fd9e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a710c710 0x5559a7109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.371+0000 7f33fd9e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a710c710 0x5559a7109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35136/0 (socket says 172.21.15.45:35136) 2023-09-23T09:37:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33fd9e0700 1 -- 172.21.15.45:0/2536117667 learned_addr learned my addr 172.21.15.45:0/2536117667 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33fd9e0700 1 -- 172.21.15.45:0/2536117667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a7100f40 msgr2=0x5559a7109460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a7100f40 0x5559a7109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33fd9e0700 1 -- 172.21.15.45:0/2536117667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 msgr2=0x5559a71099a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:37:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a71099a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33fd9e0700 1 -- 172.21.15.45:0/2536117667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e8017070 con 0x5559a710c710 2023-09-23T09:37:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a710c710 0x5559a7109ee0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f33f009a9f0 tx=0x7f33f009a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208b8488cd26cda3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.372+0000 7f33e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f33e0ff9700 1 -- 172.21.15.45:0/2536117667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f009acf0 con 0x5559a710c710 2023-09-23T09:37:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559a70f5090 con 0x5559a710c710 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f33e0ff9700 1 -- 172.21.15.45:0/2536117667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f33f009ee40 con 0x5559a710c710 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f33e0ff9700 1 -- 172.21.15.45:0/2536117667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f00a2ff0 con 0x5559a710c710 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 10 client.16365 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5559a6e7fd80 con 0x5559a710c710 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f33e0ff9700 1 -- 172.21.15.45:0/2536117667 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559a70f5090 con 0x5559a710c710 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f33e0ff9700 1 -- 172.21.15.45:0/2536117667 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f33f00a2b60 con 0x5559a710c710 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 10 client.16365 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 20 client.16365 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a711c430 0x5559a711e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 4 client.16365 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:37:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.373+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559a704ddb0 con 0x5559a711c430 2023-09-23T09:37:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.374+0000 7f33fd1df700 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a711c430 0x5559a711e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.374+0000 7f33fd1df700 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a711c430 0x5559a711e870 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce310b58b311ad7 server_cookie=674d4384e5e445a7 in_seq=0 out_seq=0 2023-09-23T09:37:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.374+0000 7f33e0ff9700 10 client.16365 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.375+0000 7f33e0ff9700 1 -- 172.21.15.45:0/2536117667 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559a704ddb0 con 0x5559a711c430 2023-09-23T09:37:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.375+0000 7f33e0ff9700 10 client.16365 handle_command_reply: tid=0 2023-09-23T09:37:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.375+0000 7f340b62db80 1 client.16365 shutdown 2023-09-23T09:37:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a710c710 msgr2=0x5559a7109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a710c710 0x5559a7109ee0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f33f009a9f0 tx=0x7f33f009a780 comp rx=0 tx=0).stop 2023-09-23T09:37:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 shutdown_connections 2023-09-23T09:37:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a711c430 0x5559a711e870 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a710c710 0x5559a7109ee0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a7100f40 0x5559a7109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 --2- 172.21.15.45:0/2536117667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a71099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 >> 172.21.15.45:0/2536117667 conn(0x5559a7076200 msgr2=0x5559a70fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f33e0ff9700 0 client.16365 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 shutdown_connections 2023-09-23T09:37:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 1 -- 172.21.15.45:0/2536117667 wait complete. 2023-09-23T09:37:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.382+0000 7f340b62db80 20 client.16365 trim_cache size 0 max 16384 2023-09-23T09:37:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 Processor -- start 2023-09-23T09:37:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 -- start start 2023-09-23T09:37:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a71132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a71047b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a7104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6ed58c0 con 0x5559a7100f40 2023-09-23T09:37:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a7102a50 con 0x5559a7101860 2023-09-23T09:37:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f340b62db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a70f7540 con 0x5559a710c710 2023-09-23T09:37:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f33fd1df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a71132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f33fd1df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a71132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35142/0 (socket says 172.21.15.45:35142) 2023-09-23T09:37:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f33fd1df700 1 -- 172.21.15.45:0/3883436068 learned_addr learned my addr 172.21.15.45:0/3883436068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f33fc9de700 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a71047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.406+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a7104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33fd1df700 1 -- 172.21.15.45:0/3883436068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 msgr2=0x5559a7104cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33fd1df700 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a7104cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33fd1df700 1 -- 172.21.15.45:0/3883436068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 msgr2=0x5559a71047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33fd1df700 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a71047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33fd1df700 1 -- 172.21.15.45:0/3883436068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559a6e68c00 con 0x5559a7100f40 2023-09-23T09:37:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33fd1df700 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a71132c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f33f40098e0 tx=0x7f33f4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b844f7678e8c449f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33e3fff700 1 -- 172.21.15.45:0/3883436068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f401d070 con 0x5559a7100f40 2023-09-23T09:37:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33e3fff700 1 -- 172.21.15.45:0/3883436068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f401e040 con 0x5559a7100f40 2023-09-23T09:37:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.407+0000 7f33e3fff700 1 -- 172.21.15.45:0/3883436068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4026300 con 0x5559a7100f40 2023-09-23T09:37:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 -- 172.21.15.45:0/3883436068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 msgr2=0x5559a71132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a71132c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f33f40098e0 tx=0x7f33f4009fd0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 -- 172.21.15.45:0/3883436068 shutdown_connections 2023-09-23T09:37:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a7104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a71132c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 --2- 172.21.15.45:0/3883436068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a71047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 -- 172.21.15.45:0/3883436068 >> 172.21.15.45:0/3883436068 conn(0x5559a7076200 msgr2=0x5559a7109c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 -- 172.21.15.45:0/3883436068 shutdown_connections 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 -- 172.21.15.45:0/3883436068 wait complete. 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 -- start start 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.408+0000 7f340b62db80 1 Processor -- start 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f340b62db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a7164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f340b62db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a7164940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f340b62db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a716f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f340b62db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a7165010 con 0x5559a7100f40 2023-09-23T09:37:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f340b62db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a716f7d0 con 0x5559a7101860 2023-09-23T09:37:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f340b62db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a716f910 con 0x5559a710c710 2023-09-23T09:37:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f340b62db80 10 client.0 _ll_register_callbacks cb 0x7f33f009aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a716f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fc9de700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a7164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a716f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37258/0 (socket says 172.21.15.45:37258) 2023-09-23T09:37:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 -- 172.21.15.45:0/634082798 learned_addr learned my addr 172.21.15.45:0/634082798 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd1df700 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a7164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 -- 172.21.15.45:0/634082798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 msgr2=0x5559a7164940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a7164940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 -- 172.21.15.45:0/634082798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 msgr2=0x5559a7164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a7164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 -- 172.21.15.45:0/634082798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f810e570 con 0x5559a710c710 2023-09-23T09:37:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd1df700 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a7164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:37:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33fd9e0700 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a716f290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ef90 tx=0x7f33f009a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb547d20c277906b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.409+0000 7f33e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:37:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f33e0ff9700 1 -- 172.21.15.45:0/634082798 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f009acf0 con 0x5559a710c710 2023-09-23T09:37:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33e8021070 con 0x5559a710c710 2023-09-23T09:37:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f33e0ff9700 1 -- 172.21.15.45:0/634082798 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f33f00a2ff0 con 0x5559a710c710 2023-09-23T09:37:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f33e0ff9700 1 -- 172.21.15.45:0/634082798 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f00ae3a0 con 0x5559a710c710 2023-09-23T09:37:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 10 client.26032 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5559a6ea61f0 con 0x5559a710c710 2023-09-23T09:37:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f33e0ff9700 1 -- 172.21.15.45:0/634082798 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f007d020 con 0x5559a710c710 2023-09-23T09:37:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f33e0ff9700 1 -- 172.21.15.45:0/634082798 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f33f009fcf0 con 0x5559a710c710 2023-09-23T09:37:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 10 client.26032 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 20 client.26032 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a7170d40 0x5559a7173160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 4 client.26032 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:37:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559a704dad0 con 0x5559a7170d40 2023-09-23T09:37:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.410+0000 7f33fd1df700 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a7170d40 0x5559a7173160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.411+0000 7f33e0ff9700 10 client.26032 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.411+0000 7f33fd1df700 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a7170d40 0x5559a7173160 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba3331a2054ca2b server_cookie=d27ad2d63167dc3f in_seq=0 out_seq=0 2023-09-23T09:37:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f33e0ff9700 1 -- 172.21.15.45:0/634082798 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559a704dad0 con 0x5559a7170d40 2023-09-23T09:37:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f33e0ff9700 10 client.26032 handle_command_reply: tid=0 2023-09-23T09:37:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 client.26032 shutdown 2023-09-23T09:37:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 msgr2=0x5559a716f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a716f290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ef90 tx=0x7f33f009a8e0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 shutdown_connections 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5559a7170d40 0x5559a7173160 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a710c710 0x5559a716f290 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559a7100f40 0x5559a7164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 --2- 172.21.15.45:0/634082798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5559a7101860 0x5559a7164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f33e0ff9700 0 client.26032 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 >> 172.21.15.45:0/634082798 conn(0x5559a7076200 msgr2=0x5559a7109c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 shutdown_connections 2023-09-23T09:37:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.432+0000 7f340b62db80 1 -- 172.21.15.45:0/634082798 wait complete. 2023-09-23T09:37:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f340b62db80 20 client.26032 trim_cache size 0 max 16384 2023-09-23T09:37:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:37:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:37:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:37:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:37:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f33e40a18e0 msgr2=0x7f33e40a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f33e40a18e0 0x7f33e40a3da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f33f81019c0 tx=0x7f33e8009260 comp rx=0 tx=0).stop 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 msgr2=0x7f33f8100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 0x7f33f8100a40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f33f000efc0 tx=0x7f33f000c3a0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 shutdown_connections 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f33e40a18e0 0x7f33e40a3da0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33f8105da0 0x7f33f8100a40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f8068210 0x7f33f80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 --2- 172.21.15.45:0/2761163783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33f8068bd0 0x7f33f8100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 >> 172.21.15.45:0/2761163783 conn(0x7f33f8076840 msgr2=0x7f33f80fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 shutdown_connections 2023-09-23T09:37:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:08.433+0000 7f33ff443700 1 -- 172.21.15.45:0/2761163783 wait complete. 2023-09-23T09:37:09.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:08 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:37:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:08 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:37:09.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:08 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:37:10.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:09 smithi098 ceph-mon[102386]: pgmap v237: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2023-09-23T09:37:10.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:09 smithi045 ceph-mon[98627]: pgmap v237: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2023-09-23T09:37:10.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:09 smithi123 ceph-mon[103577]: pgmap v237: 97 pgs: 97 active+clean; 7.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2023-09-23T09:37:11.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:10 smithi098 ceph-mon[102386]: osdmap e90: 12 total, 12 up, 12 in 2023-09-23T09:37:11.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:10 smithi045 ceph-mon[98627]: osdmap e90: 12 total, 12 up, 12 in 2023-09-23T09:37:11.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:10 smithi123 ceph-mon[103577]: osdmap e90: 12 total, 12 up, 12 in 2023-09-23T09:37:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 445.0 s, 1557.6 tps, lat 63.682 ms stddev 88.593 2023-09-23T09:37:12.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:11 smithi098 ceph-mon[102386]: pgmap v239: 97 pgs: 97 active+clean; 7.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2023-09-23T09:37:12.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:11 smithi045 ceph-mon[98627]: pgmap v239: 97 pgs: 97 active+clean; 7.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2023-09-23T09:37:12.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:11 smithi123 ceph-mon[103577]: pgmap v239: 97 pgs: 97 active+clean; 7.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2023-09-23T09:37:14.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:13 smithi098 ceph-mon[102386]: pgmap v240: 97 pgs: 97 active+clean; 7.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2023-09-23T09:37:14.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:13 smithi045 ceph-mon[98627]: pgmap v240: 97 pgs: 97 active+clean; 7.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2023-09-23T09:37:14.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:13 smithi123 ceph-mon[103577]: pgmap v240: 97 pgs: 97 active+clean; 7.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2023-09-23T09:37:16.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:15 smithi098 ceph-mon[102386]: pgmap v241: 97 pgs: 97 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 10 MiB/s wr, 648 op/s 2023-09-23T09:37:16.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:15 smithi098 ceph-mon[102386]: osdmap e91: 12 total, 12 up, 12 in 2023-09-23T09:37:16.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:15 smithi045 ceph-mon[98627]: pgmap v241: 97 pgs: 97 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 10 MiB/s wr, 648 op/s 2023-09-23T09:37:16.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:15 smithi045 ceph-mon[98627]: osdmap e91: 12 total, 12 up, 12 in 2023-09-23T09:37:16.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:15 smithi123 ceph-mon[103577]: pgmap v241: 97 pgs: 97 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 10 MiB/s wr, 648 op/s 2023-09-23T09:37:16.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:15 smithi123 ceph-mon[103577]: osdmap e91: 12 total, 12 up, 12 in 2023-09-23T09:37:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 450.0 s, 1854.6 tps, lat 53.839 ms stddev 61.999 2023-09-23T09:37:18.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:17 smithi098 ceph-mon[102386]: pgmap v243: 97 pgs: 97 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.8 MiB/s wr, 627 op/s 2023-09-23T09:37:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:17 smithi045 ceph-mon[98627]: pgmap v243: 97 pgs: 97 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.8 MiB/s wr, 627 op/s 2023-09-23T09:37:18.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:17 smithi123 ceph-mon[103577]: pgmap v243: 97 pgs: 97 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.8 MiB/s wr, 627 op/s 2023-09-23T09:37:20.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:19 smithi098 ceph-mon[102386]: pgmap v244: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.6 MiB/s wr, 553 op/s 2023-09-23T09:37:20.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:19 smithi045 ceph-mon[98627]: pgmap v244: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.6 MiB/s wr, 553 op/s 2023-09-23T09:37:20.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:19 smithi123 ceph-mon[103577]: pgmap v244: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.6 MiB/s wr, 553 op/s 2023-09-23T09:37:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 -- 172.21.15.45:0/2537696972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e0100190 msgr2=0x7fb2e0100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2537696972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e0100190 0x7fb2e0100570 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009990 tx=0x7fb2dc009ca0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 -- 172.21.15.45:0/2537696972 shutdown_connections 2023-09-23T09:37:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2537696972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0105010 0x7fb2e010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2537696972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e0100190 0x7fb2e0100570 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2537696972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2e0100b40 0x7fb2e01048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 -- 172.21.15.45:0/2537696972 >> 172.21.15.45:0/2537696972 conn(0x7fb2e0077bd0 msgr2=0x7fb2e0077fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 -- 172.21.15.45:0/2537696972 shutdown_connections 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 -- 172.21.15.45:0/2537696972 wait complete. 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.573+0000 7fb2e8067700 1 Processor -- start 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e8067700 1 -- start start 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e8067700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0100190 0x7fb2e0197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e8067700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2e0105010 0x7fb2e0197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e8067700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e019bbf0 0x7fb2e019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e8067700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e019c670 con 0x7fb2e0100190 2023-09-23T09:37:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e8067700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e019c7e0 con 0x7fb2e0105010 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e8067700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e019c9a0 con 0x7fb2e019bbf0 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0100190 0x7fb2e0197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0100190 0x7fb2e0197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35162/0 (socket says 172.21.15.45:35162) 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2445981568 learned_addr learned my addr 172.21.15.45:0/2445981568 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e6604700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e019bbf0 0x7fb2e019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5602700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2e0105010 0x7fb2e0197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2445981568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e019bbf0 msgr2=0x7fb2e019c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e019bbf0 0x7fb2e019c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2445981568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2e0105010 msgr2=0x7fb2e0197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2e0105010 0x7fb2e0197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2445981568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8009c40 con 0x7fb2e0100190 2023-09-23T09:37:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.574+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0100190 0x7fb2e0197040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc005910 tx=0x7fb2dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.575+0000 7fb2d6ffd700 1 -- 172.21.15.45:0/2445981568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc01d070 con 0x7fb2e0100190 2023-09-23T09:37:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.575+0000 7fb2d6ffd700 1 -- 172.21.15.45:0/2445981568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2dc022470 con 0x7fb2e0100190 2023-09-23T09:37:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.575+0000 7fb2d6ffd700 1 -- 172.21.15.45:0/2445981568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc017650 con 0x7fb2e0100190 2023-09-23T09:37:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.578+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc0095f0 con 0x7fb2e0100190 2023-09-23T09:37:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.578+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e01a40e0 con 0x7fb2e0100190 2023-09-23T09:37:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.581+0000 7fb2d6ffd700 1 -- 172.21.15.45:0/2445981568 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fb2dc0225e0 con 0x7fb2e0100190 2023-09-23T09:37:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.581+0000 7fb2d6ffd700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb2cc0a1860 0x7fb2cc0a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.581+0000 7fb2d6ffd700 1 -- 172.21.15.45:0/2445981568 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb2dc0ce060 con 0x7fb2e0100190 2023-09-23T09:37:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.581+0000 7fb2e5602700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb2cc0a1860 0x7fb2cc0a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 Processor -- start 2023-09-23T09:37:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2e5602700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb2cc0a1860 0x7fb2cc0a3d20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0198860 tx=0x7fb2d0009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 -- start start 2023-09-23T09:37:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a474f40 0x561a0a475320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a4801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a480710 0x561a0a482b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a3de400 con 0x561a0a480710 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a3cfbc0 con 0x561a0a474f40 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a3cd4e0 con 0x561a0a475860 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2e6604700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a480710 0x561a0a482b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2e5602700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a4801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a474f40 0x561a0a475320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2e6604700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a480710 0x561a0a482b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35174/0 (socket says 172.21.15.45:35174) 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.582+0000 7fb2e6604700 1 -- 172.21.15.45:0/112791029 learned_addr learned my addr 172.21.15.45:0/112791029 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2e5602700 1 -- 172.21.15.45:0/112791029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a474f40 msgr2=0x561a0a475320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2e5602700 1 --2- 172.21.15.45:0/112791029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a474f40 0x561a0a475320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2e5602700 1 -- 172.21.15.45:0/112791029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a480710 msgr2=0x561a0a482b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2e5602700 1 --2- 172.21.15.45:0/112791029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a480710 0x561a0a482b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2e5602700 1 -- 172.21.15.45:0/112791029 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a0a42b870 con 0x561a0a475860 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2e5602700 1 --2- 172.21.15.45:0/112791029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a4801d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0011c60 tx=0x7fb2d0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a33e1147f2de023 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2cbfff700 1 -- 172.21.15.45:0/112791029 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0004230 con 0x561a0a475860 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2cbfff700 1 -- 172.21.15.45:0/112791029 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d0021070 con 0x561a0a475860 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.583+0000 7fb2cbfff700 1 -- 172.21.15.45:0/112791029 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0003bf0 con 0x561a0a475860 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 -- 172.21.15.45:0/112791029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 msgr2=0x561a0a4801d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/112791029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a4801d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0011c60 tx=0x7fb2d0014000 comp rx=0 tx=0).stop 2023-09-23T09:37:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 -- 172.21.15.45:0/112791029 shutdown_connections 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/112791029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a480710 0x561a0a482b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/112791029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a474f40 0x561a0a475320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/112791029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a4801d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 -- 172.21.15.45:0/112791029 >> 172.21.15.45:0/112791029 conn(0x561a0a3ea200 msgr2=0x561a0a4713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 -- 172.21.15.45:0/112791029 shutdown_connections 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.584+0000 7fb2f4251b80 1 -- 172.21.15.45:0/112791029 wait complete. 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 -- start start 2023-09-23T09:37:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 Processor -- start 2023-09-23T09:37:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a480150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a47d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a3c8dd0 con 0x561a0a474f40 2023-09-23T09:37:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a3656b0 con 0x561a0a480710 2023-09-23T09:37:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a365350 con 0x561a0a475860 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2f4251b80 10 client.0 _ll_register_callbacks cb 0x561a0a1c27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a480150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a480150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35176/0 (socket says 172.21.15.45:35176) 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2640932068 learned_addr learned my addr 172.21.15.45:0/2640932068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.585+0000 7fb2e5602700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a47d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e6604700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2640932068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 msgr2=0x561a0a47d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a47d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 msgr2=0x561a0a47db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e5e03700 1 -- 172.21.15.45:0/2640932068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d0017070 con 0x561a0a474f40 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e6604700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a480150 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc097a90 tx=0x7fb2dc09aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cac8fb317940ce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2c97fa700 1 -- 172.21.15.45:0/2640932068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc09b7a0 con 0x561a0a474f40 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2c97fa700 1 -- 172.21.15.45:0/2640932068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb2dc09b020 con 0x561a0a474f40 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a0a469090 con 0x561a0a474f40 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2c97fa700 1 -- 172.21.15.45:0/2640932068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2dc09f170 con 0x561a0a474f40 2023-09-23T09:37:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2f4251b80 10 client.16380 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561a0a1f3d80 con 0x561a0a474f40 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.586+0000 7fb2c97fa700 1 -- 172.21.15.45:0/2640932068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a0a469090 con 0x561a0a474f40 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2c97fa700 1 -- 172.21.15.45:0/2640932068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb2dc0aebd0 con 0x561a0a474f40 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2f4251b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2f4251b80 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2f4251b80 20 client.16380 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a48fe40 0x561a0a492260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2f4251b80 4 client.16380 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a0a3c1db0 con 0x561a0a48fe40 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2e5602700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a48fe40 0x561a0a492260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2e5602700 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a48fe40 0x561a0a492260 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e645dfa051c75de server_cookie=295f059a95565be5 in_seq=0 out_seq=0 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.587+0000 7fb2c97fa700 10 client.16380 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.588+0000 7fb2c97fa700 1 -- 172.21.15.45:0/2640932068 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a0a3c1db0 con 0x561a0a48fe40 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.588+0000 7fb2c97fa700 10 client.16380 handle_command_reply: tid=0 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.588+0000 7fb2f4251b80 1 client.16380 shutdown 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 msgr2=0x561a0a480150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a480150 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc097a90 tx=0x7fb2dc09aa00 comp rx=0 tx=0).stop 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 shutdown_connections 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a48fe40 0x561a0a492260 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a480150 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/2640932068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a475860 0x561a0a47d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.589+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 >> 172.21.15.45:0/2640932068 conn(0x561a0a3ea200 msgr2=0x561a0a470f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.590+0000 7fb2c97fa700 0 client.16380 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.590+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 shutdown_connections 2023-09-23T09:37:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.590+0000 7fb2f4251b80 1 -- 172.21.15.45:0/2640932068 wait complete. 2023-09-23T09:37:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.591+0000 7fb2f4251b80 20 client.16380 trim_cache size 0 max 16384 2023-09-23T09:37:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.608+0000 7fb2f4251b80 1 Processor -- start 2023-09-23T09:37:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.608+0000 7fb2f4251b80 1 -- start start 2023-09-23T09:37:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.608+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a474f40 0x561a0a47fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.608+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a475860 0x561a0a47cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.608+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a2498c0 con 0x561a0a475860 2023-09-23T09:37:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a46b540 con 0x561a0a480710 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a47e9e0 con 0x561a0a474f40 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a474f40 0x561a0a47fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5602700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a475860 0x561a0a47cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a474f40 0x561a0a47fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37290/0 (socket says 172.21.15.45:37290) 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 -- 172.21.15.45:0/1867278774 learned_addr learned my addr 172.21.15.45:0/1867278774 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e6604700 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 -- 172.21.15.45:0/1867278774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 msgr2=0x561a0a47d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 -- 172.21.15.45:0/1867278774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a475860 msgr2=0x561a0a47cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a475860 0x561a0a47cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 -- 172.21.15.45:0/1867278774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a0a1dcc00 con 0x561a0a474f40 2023-09-23T09:37:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5602700 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a475860 0x561a0a47cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.609+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a474f40 0x561a0a47fcf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0a2df0 tx=0x7fb2dc0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ec4cc86b9a1ace server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2d4ff9700 1 -- 172.21.15.45:0/1867278774 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc09b720 con 0x561a0a474f40 2023-09-23T09:37:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2d4ff9700 1 -- 172.21.15.45:0/1867278774 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2dc09b020 con 0x561a0a474f40 2023-09-23T09:37:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2d4ff9700 1 -- 172.21.15.45:0/1867278774 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0a2fd0 con 0x561a0a474f40 2023-09-23T09:37:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 -- 172.21.15.45:0/1867278774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a474f40 msgr2=0x561a0a47fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a474f40 0x561a0a47fcf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0a2df0 tx=0x7fb2dc0094b0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 -- 172.21.15.45:0/1867278774 shutdown_connections 2023-09-23T09:37:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a480710 0x561a0a47d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a474f40 0x561a0a47fcf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/1867278774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a475860 0x561a0a47cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 -- 172.21.15.45:0/1867278774 >> 172.21.15.45:0/1867278774 conn(0x561a0a3ea200 msgr2=0x561a0a478710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 -- 172.21.15.45:0/1867278774 shutdown_connections 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.610+0000 7fb2f4251b80 1 -- 172.21.15.45:0/1867278774 wait complete. 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 -- start start 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 Processor -- start 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a485240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a475860 0x561a0a485780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a480710 0x561a0a485cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a4e2c70 con 0x561a0a474f40 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a4e2de0 con 0x561a0a475860 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0a4e2fa0 con 0x561a0a480710 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2f4251b80 10 client.0 _ll_register_callbacks cb 0x561a0a1c27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a485240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2e5e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a485240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35190/0 (socket says 172.21.15.45:35190) 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2e6604700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a480710 0x561a0a485cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2e5e03700 1 -- 172.21.15.45:0/215099290 learned_addr learned my addr 172.21.15.45:0/215099290 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.611+0000 7fb2e5602700 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a475860 0x561a0a485780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2e5e03700 1 -- 172.21.15.45:0/215099290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a480710 msgr2=0x561a0a485cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a480710 0x561a0a485cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2e5e03700 1 -- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a475860 msgr2=0x561a0a485780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a475860 0x561a0a485780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2e5e03700 1 -- 172.21.15.45:0/215099290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a0a42b870 con 0x561a0a474f40 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2e6604700 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a480710 0x561a0a485cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2e5e03700 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a485240 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0aef70 tx=0x7fb2dc0aee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6446ec4a476bda7e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2c97fa700 1 -- 172.21.15.45:0/215099290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc09b480 con 0x561a0a474f40 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2c97fa700 1 -- 172.21.15.45:0/215099290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb2dc09b720 con 0x561a0a474f40 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2d0021070 con 0x561a0a474f40 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2c97fa700 1 -- 172.21.15.45:0/215099290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2dc0b92d0 con 0x561a0a474f40 2023-09-23T09:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2f4251b80 10 client.16386 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561a0a1f3d80 con 0x561a0a474f40 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2c97fa700 1 -- 172.21.15.45:0/215099290 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d0021070 con 0x561a0a474f40 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.612+0000 7fb2c97fa700 1 -- 172.21.15.45:0/215099290 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb2dc09a080 con 0x561a0a474f40 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2f4251b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2f4251b80 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2f4251b80 20 client.16386 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a4eca10 0x561a0a4eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2f4251b80 4 client.16386 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a0a3c1ad0 con 0x561a0a4eca10 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2e5602700 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a4eca10 0x561a0a4eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2c97fa700 10 client.16386 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.613+0000 7fb2e5602700 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a4eca10 0x561a0a4eee00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e09a1623c997a0 server_cookie=dabb5cb51a9d1dd2 in_seq=0 out_seq=0 2023-09-23T09:37:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.638+0000 7fb2c97fa700 1 -- 172.21.15.45:0/215099290 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a0a3c1ad0 con 0x561a0a4eca10 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.638+0000 7fb2c97fa700 10 client.16386 handle_command_reply: tid=0 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.638+0000 7fb2f4251b80 1 client.16386 shutdown 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.638+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 msgr2=0x561a0a485240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.638+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a485240 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0aef70 tx=0x7fb2dc0aee00 comp rx=0 tx=0).stop 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 shutdown_connections 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561a0a4eca10 0x561a0a4eee00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a0a480710 0x561a0a485cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a0a474f40 0x561a0a485240 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 --2- 172.21.15.45:0/215099290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a0a475860 0x561a0a485780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 >> 172.21.15.45:0/215099290 conn(0x561a0a3ea200 msgr2=0x561a0a470590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2c97fa700 0 client.16386 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 shutdown_connections 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 1 -- 172.21.15.45:0/215099290 wait complete. 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2f4251b80 20 client.16386 trim_cache size 0 max 16384 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:37:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:37:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb2cc0a1860 msgr2=0x7fb2cc0a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb2cc0a1860 0x7fb2cc0a3d20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0198860 tx=0x7fb2d0009260 comp rx=0 tx=0).stop 2023-09-23T09:37:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0100190 msgr2=0x7fb2e0197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.639+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0100190 0x7fb2e0197040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc005910 tx=0x7fb2dc004730 comp rx=0 tx=0).stop 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 shutdown_connections 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2e019bbf0 0x7fb2e019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb2cc0a1860 0x7fb2cc0a3d20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2e0100190 0x7fb2e0197040 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 --2- 172.21.15.45:0/2445981568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2e0105010 0x7fb2e0197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 >> 172.21.15.45:0/2445981568 conn(0x7fb2e0077bd0 msgr2=0x7fb2e00692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 shutdown_connections 2023-09-23T09:37:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:21.640+0000 7fb2e8067700 1 -- 172.21.15.45:0/2445981568 wait complete. 2023-09-23T09:37:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 455.0 s, 1384.5 tps, lat 72.629 ms stddev 128.388 2023-09-23T09:37:22.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:21 smithi098 ceph-mon[102386]: pgmap v245: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-09-23T09:37:22.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:21 smithi123 ceph-mon[103577]: pgmap v245: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-09-23T09:37:22.092 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:21 smithi045 ceph-mon[98627]: pgmap v245: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-09-23T09:37:24.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:23 smithi098 ceph-mon[102386]: pgmap v246: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-09-23T09:37:24.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:23 smithi045 ceph-mon[98627]: pgmap v246: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-09-23T09:37:24.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:23 smithi123 ceph-mon[103577]: pgmap v246: 97 pgs: 97 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-09-23T09:37:26.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:25 smithi098 ceph-mon[102386]: pgmap v247: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 MiB/s wr, 1.89k op/s 2023-09-23T09:37:26.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:25 smithi045 ceph-mon[98627]: pgmap v247: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 MiB/s wr, 1.89k op/s 2023-09-23T09:37:26.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:25 smithi123 ceph-mon[103577]: pgmap v247: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 MiB/s wr, 1.89k op/s 2023-09-23T09:37:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 460.0 s, 1494.6 tps, lat 66.541 ms stddev 71.204 2023-09-23T09:37:28.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:27 smithi098 ceph-mon[102386]: pgmap v248: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.80k op/s 2023-09-23T09:37:28.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:27 smithi045 ceph-mon[98627]: pgmap v248: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.80k op/s 2023-09-23T09:37:28.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:27 smithi123 ceph-mon[103577]: pgmap v248: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.80k op/s 2023-09-23T09:37:30.086 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:29 smithi045 ceph-mon[98627]: pgmap v249: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2023-09-23T09:37:30.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:29 smithi123 ceph-mon[103577]: pgmap v249: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2023-09-23T09:37:30.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:29 smithi098 ceph-mon[102386]: pgmap v249: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2023-09-23T09:37:31.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:31 smithi098 ceph-mon[102386]: pgmap v250: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 26 MiB/s wr, 1.85k op/s 2023-09-23T09:37:31.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:31 smithi045 ceph-mon[98627]: pgmap v250: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 26 MiB/s wr, 1.85k op/s 2023-09-23T09:37:31.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:31 smithi123 ceph-mon[103577]: pgmap v250: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 26 MiB/s wr, 1.85k op/s 2023-09-23T09:37:31.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:37:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 465.0 s, 1016.0 tps, lat 97.284 ms stddev 119.497 2023-09-23T09:37:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 -- 172.21.15.45:0/1293434915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107f00 msgr2=0x7f38b8114260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 --2- 172.21.15.45:0/1293434915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107f00 0x7f38b8114260 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f38b000b0a0 tx=0x7f38b000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 -- 172.21.15.45:0/1293434915 shutdown_connections 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 --2- 172.21.15.45:0/1293434915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107f00 0x7f38b8114260 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 --2- 172.21.15.45:0/1293434915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b810d4c0 0x7f38b810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 --2- 172.21.15.45:0/1293434915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107540 0x7f38b81079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 -- 172.21.15.45:0/1293434915 >> 172.21.15.45:0/1293434915 conn(0x7f38b8076cc0 msgr2=0x7f38b8105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 -- 172.21.15.45:0/1293434915 shutdown_connections 2023-09-23T09:37:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.044+0000 7f38bd71a700 1 -- 172.21.15.45:0/1293434915 wait complete. 2023-09-23T09:37:32.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.045+0000 7f38bd71a700 1 Processor -- start 2023-09-23T09:37:32.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38bd71a700 1 -- start start 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38bd71a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107540 0x7f38b819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38bd71a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107f00 0x7f38b819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38bd71a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b810d4c0 0x7f38b81a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38bd71a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81a2550 con 0x7f38b8107540 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38bd71a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81a26c0 con 0x7f38b8107f00 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38bd71a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81a2880 con 0x7f38b810d4c0 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107540 0x7f38b819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107f00 0x7f38b819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107540 0x7f38b819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35204/0 (socket says 172.21.15.45:35204) 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107f00 0x7f38b819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56556/0 (socket says 172.21.15.45:56556) 2023-09-23T09:37:32.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 -- 172.21.15.45:0/4051082986 learned_addr learned my addr 172.21.15.45:0/4051082986 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b77fe700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b810d4c0 0x7f38b81a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 -- 172.21.15.45:0/4051082986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b810d4c0 msgr2=0x7f38b81a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b810d4c0 0x7f38b81a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 -- 172.21.15.45:0/4051082986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107f00 msgr2=0x7f38b819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107f00 0x7f38b819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b6ffd700 1 -- 172.21.15.45:0/4051082986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0009d00 con 0x7f38b8107540 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.046+0000 7f38b67fc700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107f00 0x7f38b819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.047+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107540 0x7f38b819d570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f38a800e9a0 tx=0x7f38a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.047+0000 7f389ffff700 1 -- 172.21.15.45:0/4051082986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8009790 con 0x7f38b8107540 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.047+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b81a2b60 con 0x7f38b8107540 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.047+0000 7f389ffff700 1 -- 172.21.15.45:0/4051082986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a8010430 con 0x7f38b8107540 2023-09-23T09:37:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.047+0000 7f389ffff700 1 -- 172.21.15.45:0/4051082986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8018680 con 0x7f38b8107540 2023-09-23T09:37:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.047+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b81aa3a0 con 0x7f38b8107540 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f389ffff700 1 -- 172.21.15.45:0/4051082986 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f38a80105a0 con 0x7f38b8107540 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 Processor -- start 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 -- start start 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bc37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065b2c460 con 0x55c065bc3d20 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065b1dc20 con 0x55c065bc3400 2023-09-23T09:37:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38c9904b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065b1b540 con 0x55c065bcebd0 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38b77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bc37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38b77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bc37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56560/0 (socket says 172.21.15.45:56560) 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.052+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35212/0 (socket says 172.21.15.45:35212) 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b77fe700 1 -- 172.21.15.45:0/4167622486 learned_addr learned my addr 172.21.15.45:0/4167622486 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b67fc700 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f389ffff700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f38a00a1710 0x7f38a00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b77fe700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f38a00a1710 0x7f38a00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f389ffff700 1 -- 172.21.15.45:0/4051082986 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f38a8014070 con 0x7f38b8107540 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b6ffd700 1 -- 172.21.15.45:0/4167622486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 msgr2=0x55c065bd0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b6ffd700 1 -- 172.21.15.45:0/4167622486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 msgr2=0x55c065bc37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bc37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b6ffd700 1 -- 172.21.15.45:0/4167622486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c065b798d0 con 0x55c065bc3d20 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b77fe700 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bc37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bce690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f38a80cac20 tx=0x7f38a8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad475ddd9bbff87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f38b77fe700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f38a00a1710 0x7f38a00a3bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006130 tx=0x7f38b000b3b0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f389d7fa700 1 -- 172.21.15.45:0/4167622486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8099da0 con 0x55c065bc3d20 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f389d7fa700 1 -- 172.21.15.45:0/4167622486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a809ab40 con 0x55c065bc3d20 2023-09-23T09:37:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.053+0000 7f389d7fa700 1 -- 172.21.15.45:0/4167622486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80109a0 con 0x55c065bc3d20 2023-09-23T09:37:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 -- 172.21.15.45:0/4167622486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 msgr2=0x55c065bce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bce690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f38a80cac20 tx=0x7f38a8010c80 comp rx=0 tx=0).stop 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 -- 172.21.15.45:0/4167622486 shutdown_connections 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bc37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 --2- 172.21.15.45:0/4167622486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bce690 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 -- 172.21.15.45:0/4167622486 >> 172.21.15.45:0/4167622486 conn(0x55c065b38260 msgr2=0x55c065bbf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 -- 172.21.15.45:0/4167622486 shutdown_connections 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.054+0000 7f38c9904b80 1 -- 172.21.15.45:0/4167622486 wait complete. 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 -- start start 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 Processor -- start 2023-09-23T09:37:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bcb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bcc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065b16e30 con 0x55c065bc3400 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065ab3710 con 0x55c065bcebd0 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065ab33b0 con 0x55c065bc3d20 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38c9904b80 10 client.0 _ll_register_callbacks cb 0x55c065910800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38b77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bcc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bcb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38b67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38b77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bcc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56572/0 (socket says 172.21.15.45:56572) 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bcb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35218/0 (socket says 172.21.15.45:35218) 2023-09-23T09:37:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.055+0000 7f38b77fe700 1 -- 172.21.15.45:0/68844259 learned_addr learned my addr 172.21.15.45:0/68844259 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38b77fe700 1 -- 172.21.15.45:0/68844259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 msgr2=0x55c065bcbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38b77fe700 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38b77fe700 1 -- 172.21.15.45:0/68844259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 msgr2=0x55c065bcb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38b77fe700 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bcb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38b77fe700 1 -- 172.21.15.45:0/68844259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b8114b50 con 0x55c065bcebd0 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38b77fe700 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bcc380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f38b000bb60 tx=0x7f38b00099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87303c58c30e829 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f389aff5700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f389aff5700 1 -- 172.21.15.45:0/68844259 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b000e040 con 0x55c065bcebd0 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c065bb7250 con 0x55c065bcebd0 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f389aff5700 1 -- 172.21.15.45:0/68844259 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f38b0020690 con 0x55c065bcebd0 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f389aff5700 1 -- 172.21.15.45:0/68844259 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38b0020890 con 0x55c065bcebd0 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38c9904b80 10 client.26081 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f38b81aa3a0 con 0x55c065bcebd0 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.056+0000 7f389aff5700 1 -- 172.21.15.45:0/68844259 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c065bb7250 con 0x55c065bcebd0 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f389aff5700 1 -- 172.21.15.45:0/68844259 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f38b0019070 con 0x55c065bcebd0 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38c9904b80 10 client.26081 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38c9904b80 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38c9904b80 20 client.26081 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38c9904b80 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065bde640 0x55c065be0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38c9904b80 4 client.26081 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c065b0fe10 con 0x55c065bde640 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065bde640 0x55c065be0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f389aff5700 10 client.26081 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.057+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065bde640 0x55c065be0a60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ff52a1c3188a1c server_cookie=a9b6ccdb99ae4539 in_seq=0 out_seq=0 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f389aff5700 1 -- 172.21.15.45:0/68844259 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c065b0fe10 con 0x55c065bde640 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f389aff5700 10 client.26081 handle_command_reply: tid=0 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 client.26081 shutdown 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 msgr2=0x55c065bcc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bcc380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f38b000bb60 tx=0x7f38b00099e0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 shutdown_connections 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065bde640 0x55c065be0a60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bcc380 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f389aff5700 0 client.26081 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bcb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 --2- 172.21.15.45:0/68844259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.058+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 >> 172.21.15.45:0/68844259 conn(0x55c065b38260 msgr2=0x55c065bbf320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.059+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 shutdown_connections 2023-09-23T09:37:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.059+0000 7f38c9904b80 1 -- 172.21.15.45:0/68844259 wait complete. 2023-09-23T09:37:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.059+0000 7f38c9904b80 20 client.26081 trim_cache size 0 max 16384 2023-09-23T09:37:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 Processor -- start 2023-09-23T09:37:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 -- start start 2023-09-23T09:37:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bc63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065997920 con 0x55c065bc3400 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065bc77b0 con 0x55c065bcebd0 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38c9904b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065bce9b0 con 0x55c065bc3d20 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bc63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bc63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35226/0 (socket says 172.21.15.45:35226) 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 -- 172.21.15.45:0/1985346080 learned_addr learned my addr 172.21.15.45:0/1985346080 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b77fe700 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b67fc700 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 -- 172.21.15.45:0/1985346080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 msgr2=0x55c065bcc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 -- 172.21.15.45:0/1985346080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 msgr2=0x55c065bca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 -- 172.21.15.45:0/1985346080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c065941de0 con 0x55c065bc3400 2023-09-23T09:37:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b77fe700 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.080+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bc63b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f38a80cb1a0 tx=0x7f38a80968b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e389d1384be9ddb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f389dffb700 1 -- 172.21.15.45:0/1985346080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a809e590 con 0x55c065bc3400 2023-09-23T09:37:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f389dffb700 1 -- 172.21.15.45:0/1985346080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a80987e0 con 0x55c065bc3400 2023-09-23T09:37:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f389dffb700 1 -- 172.21.15.45:0/1985346080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8099b90 con 0x55c065bc3400 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 -- 172.21.15.45:0/1985346080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 msgr2=0x55c065bc63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bc63b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f38a80cb1a0 tx=0x7f38a80968b0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 -- 172.21.15.45:0/1985346080 shutdown_connections 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bcebd0 0x55c065bca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3400 0x55c065bc63b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1985346080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bc3d20 0x55c065bcc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 -- 172.21.15.45:0/1985346080 >> 172.21.15.45:0/1985346080 conn(0x55c065b38260 msgr2=0x55c065bca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.081+0000 7f38c9904b80 1 -- 172.21.15.45:0/1985346080 shutdown_connections 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 -- 172.21.15.45:0/1985346080 wait complete. 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 -- start start 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 Processor -- start 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bd4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bd54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065bd3cc0 con 0x55c065bc3d20 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065bd3e30 con 0x55c065bc3400 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c065bd3ff0 con 0x55c065bcebd0 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38c9904b80 10 client.0 _ll_register_callbacks cb 0x7f38b0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38b67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bd54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38b6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bd4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38b67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bd54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35234/0 (socket says 172.21.15.45:35234) 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38b77fe700 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38b67fc700 1 -- 172.21.15.45:0/1865243542 learned_addr learned my addr 172.21.15.45:0/1865243542 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.082+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bd4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56582/0 (socket says 172.21.15.45:56582) 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38b67fc700 1 -- 172.21.15.45:0/1865243542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 msgr2=0x55c065bd36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38b67fc700 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38b67fc700 1 -- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 msgr2=0x55c065bd4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38b67fc700 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bd4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38b67fc700 1 -- 172.21.15.45:0/1865243542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b81aa3a0 con 0x55c065bc3d20 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38b67fc700 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bd54c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f38ac003930 tx=0x7f38ac003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee5420e7f1826e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f389aff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f389aff5700 1 -- 172.21.15.45:0/1865243542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac016040 con 0x55c065bc3d20 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f389aff5700 1 -- 172.21.15.45:0/1865243542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f38ac004a30 con 0x55c065bc3d20 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38a809ab40 con 0x55c065bc3d20 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f389aff5700 1 -- 172.21.15.45:0/1865243542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38ac004bf0 con 0x55c065bc3d20 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38c9904b80 10 client.16404 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c065b798d0 con 0x55c065bc3d20 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f389aff5700 1 -- 172.21.15.45:0/1865243542 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a809ab40 con 0x55c065bc3d20 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f389aff5700 1 -- 172.21.15.45:0/1865243542 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f38ac0050e0 con 0x55c065bc3d20 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38c9904b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38c9904b80 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38c9904b80 20 client.16404 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.083+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065c336e0 0x55c065c35b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 4 client.16404 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c065b0fb30 con 0x55c065c336e0 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065c336e0 0x55c065c35b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f389aff5700 10 client.16404 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38b6ffd700 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065c336e0 0x55c065c35b00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31d2c2a340c4f4c server_cookie=319af5a1227d19b8 in_seq=0 out_seq=0 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f389aff5700 1 -- 172.21.15.45:0/1865243542 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c065b0fb30 con 0x55c065c336e0 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f389aff5700 10 client.16404 handle_command_reply: tid=0 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 client.16404 shutdown 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 msgr2=0x55c065bd54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bd54c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f38ac003930 tx=0x7f38ac003c40 comp rx=0 tx=0).stop 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 shutdown_connections 2023-09-23T09:37:32.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c065c336e0 0x55c065c35b00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c065bcebd0 0x55c065bd36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c065bc3400 0x55c065bd4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f38c9904b80 1 --2- 172.21.15.45:0/1865243542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c065bc3d20 0x55c065bd54c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.084+0000 7f389aff5700 0 client.16404 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 >> 172.21.15.45:0/1865243542 conn(0x55c065b38260 msgr2=0x55c065bca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 shutdown_connections 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38c9904b80 1 -- 172.21.15.45:0/1865243542 wait complete. 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38c9904b80 20 client.16404 trim_cache size 0 max 16384 2023-09-23T09:37:32.093 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:37:32.094 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:37:32.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f38a00a1710 msgr2=0x7f38a00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f38a00a1710 0x7f38a00a3bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006130 tx=0x7f38b000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107540 msgr2=0x7f38b819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:32.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107540 0x7f38b819d570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f38a800e9a0 tx=0x7f38a800ecb0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 shutdown_connections 2023-09-23T09:37:32.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b810d4c0 0x7f38b81a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f38a00a1710 0x7f38a00a3bd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38b8107540 0x7f38b819d570 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 --2- 172.21.15.45:0/4051082986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b8107f00 0x7f38b819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:32.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 >> 172.21.15.45:0/4051082986 conn(0x7f38b8076cc0 msgr2=0x7f38b810b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:32.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 shutdown_connections 2023-09-23T09:37:32.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:37:32.085+0000 7f38bd71a700 1 -- 172.21.15.45:0/4051082986 wait complete. 2023-09-23T09:37:32.101 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:37:33.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:33 smithi098 ceph-mon[102386]: pgmap v251: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 24 MiB/s wr, 1.69k op/s 2023-09-23T09:37:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:33 smithi045 ceph-mon[98627]: pgmap v251: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 24 MiB/s wr, 1.69k op/s 2023-09-23T09:37:33.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:33 smithi123 ceph-mon[103577]: pgmap v251: 97 pgs: 97 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 24 MiB/s wr, 1.69k op/s 2023-09-23T09:37:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 -- 172.21.15.45:0/750330750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95601019b0 msgr2=0x7f95601059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 --2- 172.21.15.45:0/750330750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95601019b0 0x7f95601059f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f95500098f0 tx=0x7f9550009c00 comp rx=0 tx=0).stop 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 -- 172.21.15.45:0/750330750 shutdown_connections 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 --2- 172.21.15.45:0/750330750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560106120 0x7f956010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 --2- 172.21.15.45:0/750330750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9560101000 0x7f95601013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 --2- 172.21.15.45:0/750330750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95601019b0 0x7f95601059f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 -- 172.21.15.45:0/750330750 >> 172.21.15.45:0/750330750 conn(0x7f95600fc880 msgr2=0x7f95600feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 -- 172.21.15.45:0/750330750 shutdown_connections 2023-09-23T09:37:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.753+0000 7f9568195700 1 -- 172.21.15.45:0/750330750 wait complete. 2023-09-23T09:37:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 Processor -- start 2023-09-23T09:37:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 -- start start 2023-09-23T09:37:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560101000 0x7f95601970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9560106120 0x7f95601975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f956019bc50 0x7f956019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956019c6d0 con 0x7f9560106120 2023-09-23T09:37:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956019c840 con 0x7f956019bc50 2023-09-23T09:37:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9568195700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956019ca00 con 0x7f9560101000 2023-09-23T09:37:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9565f31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560101000 0x7f95601970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.754+0000 7f9565730700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9560106120 0x7f95601975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560101000 0x7f95601970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37352/0 (socket says 172.21.15.45:37352) 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 -- 172.21.15.45:0/3329754379 learned_addr learned my addr 172.21.15.45:0/3329754379 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9566732700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f956019bc50 0x7f956019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 -- 172.21.15.45:0/3329754379 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f956019bc50 msgr2=0x7f956019c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f956019bc50 0x7f956019c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 -- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9560106120 msgr2=0x7f95601975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9560106120 0x7f95601975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 -- 172.21.15.45:0/3329754379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550009520 con 0x7f9560101000 2023-09-23T09:37:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565730700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9560106120 0x7f95601975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9565f31700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560101000 0x7f95601970a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e9a0 tx=0x7f955c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9556ffd700 1 -- 172.21.15.45:0/3329754379 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c009790 con 0x7f9560101000 2023-09-23T09:37:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95601a3f60 con 0x7f9560101000 2023-09-23T09:37:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9556ffd700 1 -- 172.21.15.45:0/3329754379 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f955c010430 con 0x7f9560101000 2023-09-23T09:37:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9556ffd700 1 -- 172.21.15.45:0/3329754379 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c018680 con 0x7f9560101000 2023-09-23T09:37:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.755+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95601a4390 con 0x7f9560101000 2023-09-23T09:37:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.760+0000 7f957437fb80 1 Processor -- start 2023-09-23T09:37:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.760+0000 7f957437fb80 1 -- start start 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.761+0000 7f957437fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423504f40 0x55a423505320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.761+0000 7f957437fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423505860 0x55a4235211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.761+0000 7f957437fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423521720 0x55a423523b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.761+0000 7f957437fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42346e400 con 0x55a423505860 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.761+0000 7f957437fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42345fbc0 con 0x55a423521720 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.761+0000 7f957437fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42345d4e0 con 0x55a423504f40 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.764+0000 7f9556ffd700 1 -- 172.21.15.45:0/3329754379 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f955c0105a0 con 0x7f9560101000 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.765+0000 7f9556ffd700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f954c0a1790 0x7f954c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.765+0000 7f9556ffd700 1 -- 172.21.15.45:0/3329754379 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f955c014070 con 0x7f9560101000 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.769+0000 7f9565730700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423521720 0x55a423523b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.771+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423505860 0x55a4235211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.771+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423505860 0x55a4235211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35254/0 (socket says 172.21.15.45:35254) 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.771+0000 7f9565f31700 1 -- 172.21.15.45:0/1416504794 learned_addr learned my addr 172.21.15.45:0/1416504794 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.772+0000 7f9566732700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f954c0a1790 0x7f954c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.772+0000 7f9566732700 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423504f40 0x55a423505320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.773+0000 7f9566732700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f954c0a1790 0x7f954c0a3c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95580088a0 tx=0x7f95580092b0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.778+0000 7f9565730700 1 -- 172.21.15.45:0/1416504794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423504f40 msgr2=0x55a423505320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.778+0000 7f9565730700 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423504f40 0x55a423505320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.778+0000 7f9565730700 1 -- 172.21.15.45:0/1416504794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423505860 msgr2=0x55a4235211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.778+0000 7f9565730700 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423505860 0x55a4235211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.778+0000 7f9565730700 1 -- 172.21.15.45:0/1416504794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4234bb870 con 0x55a423521720 2023-09-23T09:37:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.786+0000 7f9565730700 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423521720 0x55a423523b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9550009230 tx=0x7f9550005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad90761d1835ba9b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.791+0000 7f954bfff700 1 -- 172.21.15.45:0/1416504794 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955001c050 con 0x55a423521720 2023-09-23T09:37:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.791+0000 7f954bfff700 1 -- 172.21.15.45:0/1416504794 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9550025070 con 0x55a423521720 2023-09-23T09:37:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.791+0000 7f954bfff700 1 -- 172.21.15.45:0/1416504794 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95500052a0 con 0x55a423521720 2023-09-23T09:37:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.792+0000 7f957437fb80 1 -- 172.21.15.45:0/1416504794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423521720 msgr2=0x55a423523b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.792+0000 7f957437fb80 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423521720 0x55a423523b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9550009230 tx=0x7f9550005dc0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.793+0000 7f957437fb80 1 -- 172.21.15.45:0/1416504794 shutdown_connections 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.793+0000 7f957437fb80 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423521720 0x55a423523b10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.793+0000 7f957437fb80 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423504f40 0x55a423505320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.793+0000 7f957437fb80 1 --2- 172.21.15.45:0/1416504794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423505860 0x55a4235211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.793+0000 7f957437fb80 1 -- 172.21.15.45:0/1416504794 >> 172.21.15.45:0/1416504794 conn(0x55a42347a200 msgr2=0x55a4235013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.794+0000 7f957437fb80 1 -- 172.21.15.45:0/1416504794 shutdown_connections 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.794+0000 7f957437fb80 1 -- 172.21.15.45:0/1416504794 wait complete. 2023-09-23T09:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.794+0000 7f957437fb80 1 -- start start 2023-09-23T09:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.794+0000 7f957437fb80 1 Processor -- start 2023-09-23T09:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f957437fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a4235147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f957437fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423514d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f957437fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a42350fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f957437fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423458dd0 con 0x55a423504f40 2023-09-23T09:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f957437fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4233f56b0 con 0x55a423505860 2023-09-23T09:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f957437fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4233f5350 con 0x55a423521720 2023-09-23T09:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f957437fb80 10 client.0 _ll_register_callbacks cb 0x55a4232527a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a4235147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a4235147e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35260/0 (socket says 172.21.15.45:35260) 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 -- 172.21.15.45:0/2753443712 learned_addr learned my addr 172.21.15.45:0/2753443712 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9566732700 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a42350fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565730700 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423514d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 -- 172.21.15.45:0/2753443712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 msgr2=0x55a42350fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a42350fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 -- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 msgr2=0x55a423514d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423514d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.795+0000 7f9565f31700 1 -- 172.21.15.45:0/2753443712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956010e510 con 0x55a423504f40 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f9565f31700 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a4235147e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f955c098fb0 tx=0x7f955c0cd280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593932640f531f06 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f95497fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f95497fa700 1 -- 172.21.15.45:0/2753443712 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c09cc40 con 0x55a423504f40 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f95497fa700 1 -- 172.21.15.45:0/2753443712 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f955c09c540 con 0x55a423504f40 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4234f9090 con 0x55a423504f40 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f95497fa700 1 -- 172.21.15.45:0/2753443712 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955c0a0690 con 0x55a423504f40 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 10 client.16416 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a423283d80 con 0x55a423504f40 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f95497fa700 1 -- 172.21.15.45:0/2753443712 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4234f9090 con 0x55a423504f40 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f95497fa700 1 -- 172.21.15.45:0/2753443712 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f955c0ba070 con 0x55a423504f40 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 10 client.16416 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 20 client.16416 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a42351fe30 0x55a423520240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 4 client.16416 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.796+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a423451db0 con 0x55a42351fe30 2023-09-23T09:37:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.797+0000 7f9565730700 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a42351fe30 0x55a423520240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.797+0000 7f95497fa700 10 client.16416 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.797+0000 7f9565730700 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a42351fe30 0x55a423520240 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43b49094f856d40 server_cookie=25ac751223c2b725 in_seq=0 out_seq=0 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f95497fa700 1 -- 172.21.15.45:0/2753443712 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a423451db0 con 0x55a42351fe30 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f95497fa700 10 client.16416 handle_command_reply: tid=0 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 client.16416 shutdown 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 msgr2=0x55a4235147e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a4235147e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f955c098fb0 tx=0x7f955c0cd280 comp rx=0 tx=0).stop 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 shutdown_connections 2023-09-23T09:37:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a42351fe30 0x55a423520240 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a42350fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a4235147e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 --2- 172.21.15.45:0/2753443712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423514d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f95497fa700 0 client.16416 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 >> 172.21.15.45:0/2753443712 conn(0x55a42347a200 msgr2=0x55a423500f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 shutdown_connections 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.798+0000 7f957437fb80 1 -- 172.21.15.45:0/2753443712 wait complete. 2023-09-23T09:37:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.799+0000 7f957437fb80 20 client.16416 trim_cache size 0 max 16384 2023-09-23T09:37:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 Processor -- start 2023-09-23T09:37:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 -- start start 2023-09-23T09:37:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a423510450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a4234ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a423527b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4232d98c0 con 0x55a423504f40 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423500060 con 0x55a423505860 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f957437fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4235280a0 con 0x55a423521720 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a423510450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a423510450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35268/0 (socket says 172.21.15.45:35268) 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 -- 172.21.15.45:0/1900162919 learned_addr learned my addr 172.21.15.45:0/1900162919 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9566732700 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a423527b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565730700 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a4234ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 -- 172.21.15.45:0/1900162919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 msgr2=0x55a423527b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a423527b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 -- 172.21.15.45:0/1900162919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 msgr2=0x55a4234ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a4234ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 -- 172.21.15.45:0/1900162919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42326cc00 con 0x55a423504f40 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9565f31700 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a423510450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f955c01e000 tx=0x7f955c09c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938785e0feb138e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.814+0000 7f9554ff9700 1 -- 172.21.15.45:0/1900162919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c09cc40 con 0x55a423504f40 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f9554ff9700 1 -- 172.21.15.45:0/1900162919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f955c085c70 con 0x55a423504f40 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f9554ff9700 1 -- 172.21.15.45:0/1900162919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c09ba30 con 0x55a423504f40 2023-09-23T09:37:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 -- 172.21.15.45:0/1900162919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 msgr2=0x55a423510450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a423510450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f955c01e000 tx=0x7f955c09c310 comp rx=0 tx=0).stop 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 -- 172.21.15.45:0/1900162919 shutdown_connections 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a423527b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a423510450 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 --2- 172.21.15.45:0/1900162919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a4234ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 -- 172.21.15.45:0/1900162919 >> 172.21.15.45:0/1900162919 conn(0x55a42347a200 msgr2=0x55a42350a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 -- 172.21.15.45:0/1900162919 shutdown_connections 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 -- 172.21.15.45:0/1900162919 wait complete. 2023-09-23T09:37:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.815+0000 7f957437fb80 1 -- start start 2023-09-23T09:37:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 1 Processor -- start 2023-09-23T09:37:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a42350d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423509060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a4235095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423509ae0 con 0x55a423504f40 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423509c50 con 0x55a423505860 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423509e10 con 0x55a423521720 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f957437fb80 10 client.0 _ll_register_callbacks cb 0x7f955c0ccd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a42350d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a42350d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35274/0 (socket says 172.21.15.45:35274) 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 -- 172.21.15.45:0/2411278785 learned_addr learned my addr 172.21.15.45:0/2411278785 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9566732700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a4235095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565730700 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423509060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 -- 172.21.15.45:0/2411278785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 msgr2=0x55a4235095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a4235095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 -- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 msgr2=0x55a423509060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423509060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.816+0000 7f9565f31700 1 -- 172.21.15.45:0/2411278785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a423283d80 con 0x55a423504f40 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f9565730700 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423509060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f9565f31700 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a42350d4f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f955c000c00 tx=0x7f955c09bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8dbb4a9dd306ca server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f95497fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f95497fa700 1 -- 172.21.15.45:0/2411278785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c09c540 con 0x55a423504f40 2023-09-23T09:37:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f95497fa700 1 -- 172.21.15.45:0/2411278785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f955c0a0690 con 0x55a423504f40 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9550025070 con 0x55a423504f40 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f95497fa700 1 -- 172.21.15.45:0/2411278785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955c09cc40 con 0x55a423504f40 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 10 client.16422 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a4232aa1f0 con 0x55a423504f40 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f95497fa700 1 -- 172.21.15.45:0/2411278785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9550025070 con 0x55a423504f40 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f95497fa700 1 -- 172.21.15.45:0/2411278785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f955c0ba050 con 0x55a423504f40 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 10 client.16422 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 20 client.16422 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a423574bd0 0x55a423576fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 4 client.16422 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:37:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.817+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a423451ad0 con 0x55a423574bd0 2023-09-23T09:37:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.818+0000 7f9565730700 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a423574bd0 0x55a423576fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.818+0000 7f9565730700 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a423574bd0 0x55a423576fc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d45ba67c4c09f7 server_cookie=bae46852e76b6de3 in_seq=0 out_seq=0 2023-09-23T09:37:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.818+0000 7f95497fa700 10 client.16422 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f95497fa700 1 -- 172.21.15.45:0/2411278785 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a423451ad0 con 0x55a423574bd0 2023-09-23T09:37:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f95497fa700 10 client.16422 handle_command_reply: tid=0 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 client.16422 shutdown 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 msgr2=0x55a42350d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a42350d4f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f955c000c00 tx=0x7f955c09bfd0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 shutdown_connections 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a423574bd0 0x55a423576fc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a423521720 0x55a4235095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f95497fa700 0 client.16422 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a423504f40 0x55a42350d4f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 --2- 172.21.15.45:0/2411278785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a423505860 0x55a423509060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 >> 172.21.15.45:0/2411278785 conn(0x55a42347a200 msgr2=0x55a42350e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 shutdown_connections 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.831+0000 7f957437fb80 1 -- 172.21.15.45:0/2411278785 wait complete. 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f957437fb80 20 client.16422 trim_cache size 0 max 16384 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:37:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:37:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f954c0a1790 msgr2=0x7f954c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f954c0a1790 0x7f954c0a3c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95580088a0 tx=0x7f95580092b0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560101000 msgr2=0x7f95601970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560101000 0x7f95601970a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e9a0 tx=0x7f955c00ecb0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 shutdown_connections 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f956019bc50 0x7f956019c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9560101000 0x7f95601970a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f954c0a1790 0x7f954c0a3c50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 --2- 172.21.15.45:0/3329754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9560106120 0x7f95601975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 >> 172.21.15.45:0/3329754379 conn(0x7f95600fc880 msgr2=0x7f95601049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 shutdown_connections 2023-09-23T09:37:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:34.832+0000 7f9568195700 1 -- 172.21.15.45:0/3329754379 wait complete. 2023-09-23T09:37:35.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:35 smithi098 ceph-mon[102386]: pgmap v252: 97 pgs: 97 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 37 MiB/s wr, 2.59k op/s 2023-09-23T09:37:35.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:35 smithi045 ceph-mon[98627]: pgmap v252: 97 pgs: 97 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 37 MiB/s wr, 2.59k op/s 2023-09-23T09:37:35.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:35 smithi123 ceph-mon[103577]: pgmap v252: 97 pgs: 97 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 37 MiB/s wr, 2.59k op/s 2023-09-23T09:37:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 470.0 s, 1113.8 tps, lat 91.308 ms stddev 250.773 2023-09-23T09:37:37.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:37 smithi098 ceph-mon[102386]: pgmap v253: 97 pgs: 97 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-23T09:37:37.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:37 smithi045 ceph-mon[98627]: pgmap v253: 97 pgs: 97 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-23T09:37:37.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:37 smithi123 ceph-mon[103577]: pgmap v253: 97 pgs: 97 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-23T09:37:39.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:39 smithi098 ceph-mon[102386]: pgmap v254: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2023-09-23T09:37:39.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:39 smithi045 ceph-mon[98627]: pgmap v254: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2023-09-23T09:37:39.839 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:39 smithi123 ceph-mon[103577]: pgmap v254: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2023-09-23T09:37:41.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:41 smithi045 ceph-mon[98627]: pgmap v255: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2023-09-23T09:37:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:41 smithi123 ceph-mon[103577]: pgmap v255: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2023-09-23T09:37:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 475.0 s, 1684.5 tps, lat 59.512 ms stddev 70.175 2023-09-23T09:37:42.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:41 smithi098 ceph-mon[102386]: pgmap v255: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2023-09-23T09:37:44.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:43 smithi098 ceph-mon[102386]: pgmap v256: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2023-09-23T09:37:44.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:43 smithi045 ceph-mon[98627]: pgmap v256: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2023-09-23T09:37:44.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:43 smithi123 ceph-mon[103577]: pgmap v256: 97 pgs: 97 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2023-09-23T09:37:46.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:45 smithi098 ceph-mon[102386]: pgmap v257: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 31 MiB/s wr, 2.01k op/s 2023-09-23T09:37:46.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:45 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:37:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:45 smithi045 ceph-mon[98627]: pgmap v257: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 31 MiB/s wr, 2.01k op/s 2023-09-23T09:37:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:45 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:37:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:45 smithi123 ceph-mon[103577]: pgmap v257: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 31 MiB/s wr, 2.01k op/s 2023-09-23T09:37:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:45 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:37:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 480.0 s, 1246.6 tps, lat 80.208 ms stddev 97.450 2023-09-23T09:37:47.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:46 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:37:47.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:46 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:37:47.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:46 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:37:47.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:46 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:37:47.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:46 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:37:47.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:46 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:37:47.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:46 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:37:47.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:46 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:37:47.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:46 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 -- 172.21.15.45:0/2534814513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 msgr2=0x7fb55410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/2534814513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb55410cb30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009990 tx=0x7fb54c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 -- 172.21.15.45:0/2534814513 shutdown_connections 2023-09-23T09:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/2534814513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb55410cb30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/2534814513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb554105d90 0x7fb554106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/2534814513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 0x7fb554100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 -- 172.21.15.45:0/2534814513 >> 172.21.15.45:0/2534814513 conn(0x7fb554076e20 msgr2=0x7fb554077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 -- 172.21.15.45:0/2534814513 shutdown_connections 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.989+0000 7fb55bf3b700 1 -- 172.21.15.45:0/2534814513 wait complete. 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 Processor -- start 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 -- start start 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 0x7fb554198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb554199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb554105d90 0x7fb55419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419e1c0 con 0x7fb554100690 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419e330 con 0x7fb5540ffcd0 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55bf3b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419e4f0 con 0x7fb554105d90 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb5594d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb554199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 0x7fb554198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb5594d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb554199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35290/0 (socket says 172.21.15.45:35290) 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 0x7fb554198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56638/0 (socket says 172.21.15.45:56638) 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb5594d6700 1 -- 172.21.15.45:0/3456766298 learned_addr learned my addr 172.21.15.45:0/3456766298 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb55a4d8700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb554105d90 0x7fb55419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb5594d6700 1 -- 172.21.15.45:0/3456766298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb554105d90 msgr2=0x7fb55419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb5594d6700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb554105d90 0x7fb55419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb5594d6700 1 -- 172.21.15.45:0/3456766298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 msgr2=0x7fb554198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.990+0000 7fb5594d6700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 0x7fb554198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb5594d6700 1 -- 172.21.15.45:0/3456766298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c0095f0 con 0x7fb554100690 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb559cd7700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 0x7fb554198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb5594d6700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb554199510 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb54400e9a0 tx=0x7fb54400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb54affd700 1 -- 172.21.15.45:0/3456766298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544009790 con 0x7fb554100690 2023-09-23T09:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb54affd700 1 -- 172.21.15.45:0/3456766298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb544010430 con 0x7fb554100690 2023-09-23T09:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55419e7d0 con 0x7fb554100690 2023-09-23T09:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb54affd700 1 -- 172.21.15.45:0/3456766298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544018680 con 0x7fb554100690 2023-09-23T09:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.991+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541a6010 con 0x7fb554100690 2023-09-23T09:37:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.993+0000 7fb54affd700 1 -- 172.21.15.45:0/3456766298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fb5440105a0 con 0x7fb554100690 2023-09-23T09:37:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.994+0000 7fb54affd700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb5400a1710 0x7fb5400a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.994+0000 7fb559cd7700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb5400a1710 0x7fb5400a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.994+0000 7fb54affd700 1 -- 172.21.15.45:0/3456766298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb544014070 con 0x7fb554100690 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.994+0000 7fb559cd7700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb5400a1710 0x7fb5400a3bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb550009480 tx=0x7fb55000b040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 Processor -- start 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 -- start start 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bc860 0x56206b5c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5c7710 0x56206b5c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b525400 con 0x56206b5bc860 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b516bc0 con 0x56206b5c7710 2023-09-23T09:37:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb568125b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b5144e0 con 0x56206b5bbf40 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb55a4d8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5c7710 0x56206b5c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5bc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37402/0 (socket says 172.21.15.45:37402) 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb559cd7700 1 -- 172.21.15.45:0/3691052149 learned_addr learned my addr 172.21.15.45:0/3691052149 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.995+0000 7fb5594d6700 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bc860 0x56206b5c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb55a4d8700 1 -- 172.21.15.45:0/3691052149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 msgr2=0x56206b5bc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb55a4d8700 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5bc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb55a4d8700 1 -- 172.21.15.45:0/3691052149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bc860 msgr2=0x56206b5c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb55a4d8700 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bc860 0x56206b5c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb55a4d8700 1 -- 172.21.15.45:0/3691052149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206b572870 con 0x56206b5c7710 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb55a4d8700 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5c7710 0x56206b5c9b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006010 tx=0x7fb54c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba2fbcea3f97de7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb53ffff700 1 -- 172.21.15.45:0/3691052149 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c01d070 con 0x56206b5c7710 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb53ffff700 1 -- 172.21.15.45:0/3691052149 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54c01e040 con 0x56206b5c7710 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.996+0000 7fb53ffff700 1 -- 172.21.15.45:0/3691052149 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c004860 con 0x56206b5c7710 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 -- 172.21.15.45:0/3691052149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5c7710 msgr2=0x56206b5c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5c7710 0x56206b5c9b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006010 tx=0x7fb54c005e80 comp rx=0 tx=0).stop 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 -- 172.21.15.45:0/3691052149 shutdown_connections 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5c7710 0x56206b5c9b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5bc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 --2- 172.21.15.45:0/3691052149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bc860 0x56206b5c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 -- 172.21.15.45:0/3691052149 >> 172.21.15.45:0/3691052149 conn(0x56206b531200 msgr2=0x56206b5b83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 -- 172.21.15.45:0/3691052149 shutdown_connections 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.997+0000 7fb568125b80 1 -- 172.21.15.45:0/3691052149 wait complete. 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 -- start start 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 Processor -- start 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 0x56206b5c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b50fdd0 con 0x56206b5c7710 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b4ac6b0 con 0x56206b5bc860 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.998+0000 7fb568125b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b4ac350 con 0x56206b5bbf40 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb568125b80 10 client.0 _ll_register_callbacks cb 0x56206b3097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37408/0 (socket says 172.21.15.45:37408) 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 -- 172.21.15.45:0/4235173299 learned_addr learned my addr 172.21.15.45:0/4235173299 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb5594d6700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb55a4d8700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 0x56206b5c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 -- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 msgr2=0x56206b5c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 -- 172.21.15.45:0/4235173299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 msgr2=0x56206b5c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 0x56206b5c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 -- 172.21.15.45:0/4235173299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55410d420 con 0x56206b5bbf40 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb5594d6700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb559cd7700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c70d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5500176c0 tx=0x7fb550004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b627c8c70359c82 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb53d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb53d7fa700 1 -- 172.21.15.45:0/4235173299 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55000ee80 con 0x56206b5bbf40 2023-09-23T09:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56206b5b0090 con 0x56206b5bbf40 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:47.999+0000 7fb53d7fa700 1 -- 172.21.15.45:0/4235173299 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb55001c040 con 0x56206b5bbf40 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb53d7fa700 1 -- 172.21.15.45:0/4235173299 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb550017e60 con 0x56206b5bbf40 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 10 client.26098 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56206b33ad80 con 0x56206b5bbf40 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb53d7fa700 1 -- 172.21.15.45:0/4235173299 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56206b5b0090 con 0x56206b5bbf40 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb53d7fa700 1 -- 172.21.15.45:0/4235173299 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb550031070 con 0x56206b5bbf40 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 20 client.26098 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b5d7230 0x56206b5d9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 4 client.26098 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:37:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56206b508db0 con 0x56206b5d7230 2023-09-23T09:37:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.000+0000 7fb5594d6700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b5d7230 0x56206b5d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.001+0000 7fb53d7fa700 10 client.26098 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.001+0000 7fb5594d6700 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b5d7230 0x56206b5d9650 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27894b562f3d74b6 server_cookie=e51f62e74ab288ee in_seq=0 out_seq=0 2023-09-23T09:37:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb53d7fa700 1 -- 172.21.15.45:0/4235173299 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56206b508db0 con 0x56206b5d7230 2023-09-23T09:37:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb53d7fa700 10 client.26098 handle_command_reply: tid=0 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 client.26098 shutdown 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 msgr2=0x56206b5c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c70d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5500176c0 tx=0x7fb550004ad0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 shutdown_connections 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b5d7230 0x56206b5d9650 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 0x56206b5c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c70d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb53d7fa700 0 client.26098 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 --2- 172.21.15.45:0/4235173299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 >> 172.21.15.45:0/4235173299 conn(0x56206b531200 msgr2=0x56206b5b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 shutdown_connections 2023-09-23T09:37:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.002+0000 7fb568125b80 1 -- 172.21.15.45:0/4235173299 wait complete. 2023-09-23T09:37:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.003+0000 7fb568125b80 20 client.26098 trim_cache size 0 max 16384 2023-09-23T09:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.031+0000 7fb568125b80 1 Processor -- start 2023-09-23T09:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.031+0000 7fb568125b80 1 -- start start 2023-09-23T09:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb568125b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb568125b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb568125b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 0x56206b5bf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb568125b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b3908c0 con 0x56206b5c7710 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb568125b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b5c01e0 con 0x56206b5bc860 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb568125b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b5b9950 con 0x56206b5bbf40 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb5594d6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37416/0 (socket says 172.21.15.45:37416) 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb5594d6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56662/0 (socket says 172.21.15.45:56662) 2023-09-23T09:37:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb559cd7700 1 -- 172.21.15.45:0/3932124703 learned_addr learned my addr 172.21.15.45:0/3932124703 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb5594d6700 1 -- 172.21.15.45:0/3932124703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 msgr2=0x56206b5c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb5594d6700 1 --2- 172.21.15.45:0/3932124703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb5594d6700 1 -- 172.21.15.45:0/3932124703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 msgr2=0x56206b5bf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb5594d6700 1 --2- 172.21.15.45:0/3932124703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 0x56206b5bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.032+0000 7fb5594d6700 1 -- 172.21.15.45:0/3932124703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206b323c00 con 0x56206b5bc860 2023-09-23T09:37:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.033+0000 7fb5594d6700 1 --2- 172.21.15.45:0/3932124703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c6c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb5440cac20 tx=0x7fb54401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1d187317ed9306 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.033+0000 7fb548ff9700 1 -- 172.21.15.45:0/3932124703 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54409ab40 con 0x56206b5bc860 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.033+0000 7fb548ff9700 1 -- 172.21.15.45:0/3932124703 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54409a440 con 0x56206b5bc860 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.033+0000 7fb548ff9700 1 -- 172.21.15.45:0/3932124703 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54409e3c0 con 0x56206b5bc860 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 -- 172.21.15.45:0/3932124703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 msgr2=0x56206b5c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 --2- 172.21.15.45:0/3932124703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c6c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb5440cac20 tx=0x7fb54401a040 comp rx=0 tx=0).stop 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 -- 172.21.15.45:0/3932124703 shutdown_connections 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 --2- 172.21.15.45:0/3932124703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5c7710 0x56206b5bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 --2- 172.21.15.45:0/3932124703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5bbf40 0x56206b5c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 --2- 172.21.15.45:0/3932124703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5c6c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 -- 172.21.15.45:0/3932124703 >> 172.21.15.45:0/3932124703 conn(0x56206b531200 msgr2=0x56206b5d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 -- 172.21.15.45:0/3932124703 shutdown_connections 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.034+0000 7fb568125b80 1 -- 172.21.15.45:0/3932124703 wait complete. 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 -- start start 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 Processor -- start 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bbf40 0x56206b5cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5c7710 0x56206b632870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b5cc830 con 0x56206b5bbf40 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b5cc9a0 con 0x56206b5bc860 2023-09-23T09:37:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206b621030 con 0x56206b5c7710 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb568125b80 10 client.0 _ll_register_callbacks cb 0x7fb55000ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.035+0000 7fb5594d6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb55a4d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5c7710 0x56206b632870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb559cd7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bbf40 0x56206b5cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb5594d6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56666/0 (socket says 172.21.15.45:56666) 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb55a4d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5c7710 0x56206b632870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37426/0 (socket says 172.21.15.45:37426) 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb5594d6700 1 -- 172.21.15.45:0/2247099918 learned_addr learned my addr 172.21.15.45:0/2247099918 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb559cd7700 1 -- 172.21.15.45:0/2247099918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5c7710 msgr2=0x56206b632870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb559cd7700 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5c7710 0x56206b632870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb559cd7700 1 -- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 msgr2=0x56206b5cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb559cd7700 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb559cd7700 1 -- 172.21.15.45:0/2247099918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541a6010 con 0x56206b5bbf40 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb5594d6700 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb559cd7700 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bbf40 0x56206b5cbb30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb55002b850 tx=0x7fb55002b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ccd0715be22393 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb53d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb53d7fa700 1 -- 172.21.15.45:0/2247099918 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550004d50 con 0x56206b5bbf40 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb53d7fa700 1 -- 172.21.15.45:0/2247099918 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb55001c070 con 0x56206b5bbf40 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.036+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb54c01e040 con 0x56206b5bbf40 2023-09-23T09:37:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb53d7fa700 1 -- 172.21.15.45:0/2247099918 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb55002ba70 con 0x56206b5bbf40 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 10 client.16437 fetch_fsmap learned FSMap version 43 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56206b3611f0 con 0x56206b5bbf40 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb53d7fa700 1 -- 172.21.15.45:0/2247099918 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb550045020 con 0x56206b5bbf40 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb53d7fa700 1 -- 172.21.15.45:0/2247099918 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb550039070 con 0x56206b5bbf40 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 10 client.16437 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 20 client.16437 populate_metadata read hostname 'smithi045' 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b622740 0x56206b624b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 4 client.16437 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:37:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56206b508ad0 con 0x56206b622740 2023-09-23T09:37:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.037+0000 7fb5594d6700 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b622740 0x56206b624b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:37:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.038+0000 7fb53d7fa700 10 client.16437 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.038+0000 7fb5594d6700 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b622740 0x56206b624b60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceeb9258dbff3d77 server_cookie=e24192ea7525efdb in_seq=0 out_seq=0 2023-09-23T09:37:48.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:47 smithi098 ceph-mon[102386]: pgmap v258: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.11k op/s 2023-09-23T09:37:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.065+0000 7fb53d7fa700 1 -- 172.21.15.45:0/2247099918 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56206b508ad0 con 0x56206b622740 2023-09-23T09:37:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.065+0000 7fb53d7fa700 10 client.16437 handle_command_reply: tid=0 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.065+0000 7fb568125b80 1 client.16437 shutdown 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.065+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bbf40 msgr2=0x56206b5cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.065+0000 7fb568125b80 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bbf40 0x56206b5cbb30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb55002b850 tx=0x7fb55002b6c0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 shutdown_connections 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56206b622740 0x56206b624b60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56206b5c7710 0x56206b632870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb53d7fa700 0 client.16437 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56206b5bbf40 0x56206b5cbb30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 --2- 172.21.15.45:0/2247099918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56206b5bc860 0x56206b5cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 >> 172.21.15.45:0/2247099918 conn(0x56206b531200 msgr2=0x56206b5bf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 shutdown_connections 2023-09-23T09:37:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 1 -- 172.21.15.45:0/2247099918 wait complete. 2023-09-23T09:37:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.066+0000 7fb568125b80 20 client.16437 trim_cache size 0 max 16384 2023-09-23T09:37:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:37:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:37:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:37:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb5400a1710 msgr2=0x7fb5400a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb5400a1710 0x7fb5400a3bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb550009480 tx=0x7fb55000b040 comp rx=0 tx=0).stop 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 msgr2=0x7fb554199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb554199510 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb54400e9a0 tx=0x7fb54400ecb0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 shutdown_connections 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb554105d90 0x7fb55419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5540ffcd0 0x7fb554198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb5400a1710 0x7fb5400a3bd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 --2- 172.21.15.45:0/3456766298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb554100690 0x7fb554199510 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:37:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 >> 172.21.15.45:0/3456766298 conn(0x7fb554076e20 msgr2=0x7fb55410ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:37:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 shutdown_connections 2023-09-23T09:37:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:37:48.067+0000 7fb55bf3b700 1 -- 172.21.15.45:0/3456766298 wait complete. 2023-09-23T09:37:48.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:47 smithi045 ceph-mon[98627]: pgmap v258: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.11k op/s 2023-09-23T09:37:48.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:47 smithi123 ceph-mon[103577]: pgmap v258: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.11k op/s 2023-09-23T09:37:50.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:49 smithi098 ceph-mon[102386]: pgmap v259: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2023-09-23T09:37:50.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:49 smithi045 ceph-mon[98627]: pgmap v259: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2023-09-23T09:37:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:49 smithi123 ceph-mon[103577]: pgmap v259: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2023-09-23T09:37:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 485.0 s, 2033.2 tps, lat 49.225 ms stddev 54.386 2023-09-23T09:37:52.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:51 smithi098 ceph-mon[102386]: pgmap v260: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2023-09-23T09:37:52.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:51 smithi045 ceph-mon[98627]: pgmap v260: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2023-09-23T09:37:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:51 smithi123 ceph-mon[103577]: pgmap v260: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2023-09-23T09:37:54.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:53 smithi098 ceph-mon[102386]: pgmap v261: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2023-09-23T09:37:54.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:53 smithi045 ceph-mon[98627]: pgmap v261: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2023-09-23T09:37:54.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:53 smithi123 ceph-mon[103577]: pgmap v261: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2023-09-23T09:37:56.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:55 smithi098 ceph-mon[102386]: pgmap v262: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 27 MiB/s wr, 1.72k op/s 2023-09-23T09:37:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:55 smithi045 ceph-mon[98627]: pgmap v262: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 27 MiB/s wr, 1.72k op/s 2023-09-23T09:37:56.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:55 smithi123 ceph-mon[103577]: pgmap v262: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 27 MiB/s wr, 1.72k op/s 2023-09-23T09:37:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 490.0 s, 1782.2 tps, lat 56.434 ms stddev 63.638 2023-09-23T09:37:58.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:57 smithi098 ceph-mon[102386]: pgmap v263: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-23T09:37:58.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:57 smithi045 ceph-mon[98627]: pgmap v263: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-23T09:37:58.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:57 smithi123 ceph-mon[103577]: pgmap v263: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-23T09:38:00.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:37:59 smithi098 ceph-mon[102386]: pgmap v264: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-23T09:38:00.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:37:59 smithi045 ceph-mon[98627]: pgmap v264: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-23T09:38:00.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:37:59 smithi123 ceph-mon[103577]: pgmap v264: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-23T09:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.199+0000 7f95ca307700 1 -- 172.21.15.45:0/935751340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4100250 msgr2=0x7f95c4105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.199+0000 7f95ca307700 1 --2- 172.21.15.45:0/935751340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4100250 0x7f95c4105b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009990 tx=0x7f95b8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.199+0000 7f95ca307700 1 -- 172.21.15.45:0/935751340 shutdown_connections 2023-09-23T09:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 --2- 172.21.15.45:0/935751340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c41062c0 0x7f95c410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 --2- 172.21.15.45:0/935751340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95c40ff8a0 0x7f95c40ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 --2- 172.21.15.45:0/935751340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4100250 0x7f95c4105b90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 -- 172.21.15.45:0/935751340 >> 172.21.15.45:0/935751340 conn(0x7f95c4076140 msgr2=0x7f95c4076550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 -- 172.21.15.45:0/935751340 shutdown_connections 2023-09-23T09:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 -- 172.21.15.45:0/935751340 wait complete. 2023-09-23T09:38:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 Processor -- start 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.200+0000 7f95ca307700 1 -- start start 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95ca307700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c40ff8a0 0x7f95c4100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95ca307700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95c41062c0 0x7f95c4100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95ca307700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41012b0 0x7f95c41a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95ca307700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c41a9e60 con 0x7f95c41062c0 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95ca307700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c41a9fa0 con 0x7f95c40ff8a0 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95ca307700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c41aa160 con 0x7f95c41012b0 2023-09-23T09:38:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95c41062c0 0x7f95c4100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c88a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41012b0 0x7f95c41a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95c41062c0 0x7f95c4100d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35330/0 (socket says 172.21.15.45:35330) 2023-09-23T09:38:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c37fe700 1 -- 172.21.15.45:0/4293955328 learned_addr learned my addr 172.21.15.45:0/4293955328 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c3fff700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c40ff8a0 0x7f95c4100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c88a4700 1 -- 172.21.15.45:0/4293955328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c40ff8a0 msgr2=0x7f95c4100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c88a4700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c40ff8a0 0x7f95c4100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c88a4700 1 -- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95c41062c0 msgr2=0x7f95c4100d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c88a4700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95c41062c0 0x7f95c4100d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c88a4700 1 -- 172.21.15.45:0/4293955328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc009c40 con 0x7f95c41012b0 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.201+0000 7f95c3fff700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c40ff8a0 0x7f95c4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.202+0000 7f95c88a4700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41012b0 0x7f95c41a9920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00efb0 tx=0x7f95bc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.203+0000 7f95c17fa700 1 -- 172.21.15.45:0/4293955328 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00ca50 con 0x7f95c41012b0 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.203+0000 7f95c17fa700 1 -- 172.21.15.45:0/4293955328 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95bc00f040 con 0x7f95c41012b0 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.203+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b80095f0 con 0x7f95c41012b0 2023-09-23T09:38:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.203+0000 7f95c17fa700 1 -- 172.21.15.45:0/4293955328 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc004660 con 0x7f95c41012b0 2023-09-23T09:38:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.203+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41aa7c0 con 0x7f95c41012b0 2023-09-23T09:38:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.206+0000 7f95c17fa700 1 -- 172.21.15.45:0/4293955328 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f95bc0047c0 con 0x7f95c41012b0 2023-09-23T09:38:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.206+0000 7f95c17fa700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95ac0a1860 0x7f95ac0a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.206+0000 7f95c17fa700 1 -- 172.21.15.45:0/4293955328 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95bc0cce40 con 0x7f95c41012b0 2023-09-23T09:38:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.206+0000 7f95c3fff700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95ac0a1860 0x7f95ac0a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95c3fff700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95ac0a1860 0x7f95ac0a3d20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009cb0 tx=0x7f95b4009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 Processor -- start 2023-09-23T09:38:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 -- start start 2023-09-23T09:38:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca8b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fc9f4400 con 0x55e9fca96710 2023-09-23T09:38:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fc9e5bc0 con 0x55e9fca8b860 2023-09-23T09:38:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.207+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fc9e34e0 con 0x55e9fca8af40 2023-09-23T09:38:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56690/0 (socket says 172.21.15.45:56690) 2023-09-23T09:38:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 -- 172.21.15.45:0/1624593324 learned_addr learned my addr 172.21.15.45:0/1624593324 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c88a4700 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c3fff700 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca8b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 -- 172.21.15.45:0/1624593324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 msgr2=0x55e9fca8b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca8b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 -- 172.21.15.45:0/1624593324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 msgr2=0x55e9fca98b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca98b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 -- 172.21.15.45:0/1624593324 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9fca41870 con 0x55e9fca8b860 2023-09-23T09:38:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c88a4700 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca98b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95c37fe700 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca961d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005f50 tx=0x7f95b800b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d68eb95f0e3a43e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.208+0000 7f95aa7fc700 1 -- 172.21.15.45:0/1624593324 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b801d070 con 0x55e9fca8b860 2023-09-23T09:38:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95aa7fc700 1 -- 172.21.15.45:0/1624593324 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95b801e040 con 0x55e9fca8b860 2023-09-23T09:38:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95aa7fc700 1 -- 172.21.15.45:0/1624593324 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80048c0 con 0x55e9fca8b860 2023-09-23T09:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95d64f1b80 1 -- 172.21.15.45:0/1624593324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 msgr2=0x55e9fca961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca961d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005f50 tx=0x7f95b800b5b0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95d64f1b80 1 -- 172.21.15.45:0/1624593324 shutdown_connections 2023-09-23T09:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca98b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/1624593324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca961d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.209+0000 7f95d64f1b80 1 -- 172.21.15.45:0/1624593324 >> 172.21.15.45:0/1624593324 conn(0x55e9fca00200 msgr2=0x55e9fca873c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.210+0000 7f95d64f1b80 1 -- 172.21.15.45:0/1624593324 shutdown_connections 2023-09-23T09:38:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.210+0000 7f95d64f1b80 1 -- 172.21.15.45:0/1624593324 wait complete. 2023-09-23T09:38:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.210+0000 7f95d64f1b80 1 -- start start 2023-09-23T09:38:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.210+0000 7f95d64f1b80 1 Processor -- start 2023-09-23T09:38:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca8b860 0x55e9fca932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca96710 0x55e9fca93840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fc9dedd0 con 0x55e9fca8b860 2023-09-23T09:38:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fc97b6b0 con 0x55e9fca96710 2023-09-23T09:38:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fc97b350 con 0x55e9fca8af40 2023-09-23T09:38:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95d64f1b80 10 client.0 _ll_register_callbacks cb 0x55e9fc7d87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca96090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37450/0 (socket says 172.21.15.45:37450) 2023-09-23T09:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 -- 172.21.15.45:0/3233486244 learned_addr learned my addr 172.21.15.45:0/3233486244 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c88a4700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca96710 0x55e9fca93840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c37fe700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca8b860 0x55e9fca932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 -- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca96710 msgr2=0x55e9fca93840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca96710 0x55e9fca93840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 -- 172.21.15.45:0/3233486244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca8b860 msgr2=0x55e9fca932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca8b860 0x55e9fca932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 -- 172.21.15.45:0/3233486244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41aa7c0 con 0x55e9fca8af40 2023-09-23T09:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c37fe700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca8b860 0x55e9fca932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95c3fff700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca96090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95b4003930 tx=0x7f95b40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bd1c8ce04672c63 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.211+0000 7f95a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95a37fe700 1 -- 172.21.15.45:0/3233486244 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4012650 con 0x55e9fca8af40 2023-09-23T09:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95a37fe700 1 -- 172.21.15.45:0/3233486244 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95b4018070 con 0x55e9fca8af40 2023-09-23T09:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9fca7f090 con 0x55e9fca8af40 2023-09-23T09:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95a37fe700 1 -- 172.21.15.45:0/3233486244 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95b4004570 con 0x55e9fca8af40 2023-09-23T09:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 10 client.26116 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f95c410e650 con 0x55e9fca8af40 2023-09-23T09:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95a37fe700 1 -- 172.21.15.45:0/3233486244 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9fca7f090 con 0x55e9fca8af40 2023-09-23T09:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95a37fe700 1 -- 172.21.15.45:0/3233486244 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f95b40218e0 con 0x55e9fca8af40 2023-09-23T09:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 20 client.26116 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcaa6220 0x55e9fcaa8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 4 client.26116 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.212+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9fc9d7db0 con 0x55e9fcaa6220 2023-09-23T09:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.213+0000 7f95c37fe700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcaa6220 0x55e9fcaa8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.213+0000 7f95a37fe700 10 client.26116 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.213+0000 7f95c37fe700 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcaa6220 0x55e9fcaa8640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8500504d9b3132df server_cookie=1c78d07f8d9de236 in_seq=0 out_seq=0 2023-09-23T09:38:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.214+0000 7f95a37fe700 1 -- 172.21.15.45:0/3233486244 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9fc9d7db0 con 0x55e9fcaa6220 2023-09-23T09:38:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.214+0000 7f95a37fe700 10 client.26116 handle_command_reply: tid=0 2023-09-23T09:38:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.214+0000 7f95d64f1b80 1 client.26116 shutdown 2023-09-23T09:38:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.214+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 msgr2=0x55e9fca96090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.214+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca96090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95b4003930 tx=0x7f95b40047c0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 shutdown_connections 2023-09-23T09:38:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcaa6220 0x55e9fcaa8640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca96710 0x55e9fca93840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca96090 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95a37fe700 0 client.26116 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3233486244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca8b860 0x55e9fca932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 >> 172.21.15.45:0/3233486244 conn(0x55e9fca00200 msgr2=0x55e9fca86fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 shutdown_connections 2023-09-23T09:38:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3233486244 wait complete. 2023-09-23T09:38:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.218+0000 7f95d64f1b80 20 client.26116 trim_cache size 0 max 16384 2023-09-23T09:38:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 Processor -- start 2023-09-23T09:38:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 -- start start 2023-09-23T09:38:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca9d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca95bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fc85f8c0 con 0x55e9fca96710 2023-09-23T09:38:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fca9d7f0 con 0x55e9fca8b860 2023-09-23T09:38:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fca94730 con 0x55e9fca8af40 2023-09-23T09:38:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95c88a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95c88a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35356/0 (socket says 172.21.15.45:35356) 2023-09-23T09:38:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95c88a4700 1 -- 172.21.15.45:0/3362297384 learned_addr learned my addr 172.21.15.45:0/3362297384 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.234+0000 7f95c37fe700 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca95bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95c3fff700 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca9d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95c88a4700 1 -- 172.21.15.45:0/3362297384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 msgr2=0x55e9fca9d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95c88a4700 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca9d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95c88a4700 1 -- 172.21.15.45:0/3362297384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 msgr2=0x55e9fca95bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95c88a4700 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca95bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95c88a4700 1 -- 172.21.15.45:0/3362297384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9fc809d80 con 0x55e9fca96710 2023-09-23T09:38:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95c88a4700 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8e630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0ca600 tx=0x7f95bc0a0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc1a858bf22b073 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95aaffd700 1 -- 172.21.15.45:0/3362297384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc09c040 con 0x55e9fca96710 2023-09-23T09:38:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95aaffd700 1 -- 172.21.15.45:0/3362297384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95bc09a500 con 0x55e9fca96710 2023-09-23T09:38:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.235+0000 7f95aaffd700 1 -- 172.21.15.45:0/3362297384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00cbb0 con 0x55e9fca96710 2023-09-23T09:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3362297384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 msgr2=0x55e9fca8e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8e630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0ca600 tx=0x7f95bc0a0040 comp rx=0 tx=0).stop 2023-09-23T09:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3362297384 shutdown_connections 2023-09-23T09:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8e630 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca9d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/3362297384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca95bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3362297384 >> 172.21.15.45:0/3362297384 conn(0x55e9fca00200 msgr2=0x55e9fcaa23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3362297384 shutdown_connections 2023-09-23T09:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 -- 172.21.15.45:0/3362297384 wait complete. 2023-09-23T09:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 -- start start 2023-09-23T09:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.236+0000 7f95d64f1b80 1 Processor -- start 2023-09-23T09:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca90f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95d64f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fca8fce0 con 0x55e9fca96710 2023-09-23T09:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fca8fe50 con 0x55e9fca8b860 2023-09-23T09:38:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95d64f1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fca90010 con 0x55e9fca8af40 2023-09-23T09:38:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95d64f1b80 10 client.0 _ll_register_callbacks cb 0x7f95b40043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca90a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c88a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca90f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca90a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37464/0 (socket says 172.21.15.45:37464) 2023-09-23T09:38:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c3fff700 1 -- 172.21.15.45:0/2283101983 learned_addr learned my addr 172.21.15.45:0/2283101983 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c88a4700 1 -- 172.21.15.45:0/2283101983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 msgr2=0x55e9fca90a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c88a4700 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca90a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c88a4700 1 -- 172.21.15.45:0/2283101983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 msgr2=0x55e9fca90f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c88a4700 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca90f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c88a4700 1 -- 172.21.15.45:0/2283101983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc021020 con 0x55e9fca96710 2023-09-23T09:38:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.237+0000 7f95c88a4700 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8f610 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f95bc099d90 tx=0x7f95bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e70dcbbed955967 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:38:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95a37fe700 1 -- 172.21.15.45:0/2283101983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0a9070 con 0x55e9fca96710 2023-09-23T09:38:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95a37fe700 1 -- 172.21.15.45:0/2283101983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95bc09df50 con 0x55e9fca96710 2023-09-23T09:38:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95b801e040 con 0x55e9fca96710 2023-09-23T09:38:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95a37fe700 1 -- 172.21.15.45:0/2283101983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95bc09c050 con 0x55e9fca96710 2023-09-23T09:38:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 10 client.16455 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e9fc7f2c00 con 0x55e9fca96710 2023-09-23T09:38:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95a37fe700 1 -- 172.21.15.45:0/2283101983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95bc07c080 con 0x55e9fca96710 2023-09-23T09:38:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95a37fe700 1 -- 172.21.15.45:0/2283101983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f95bc0ac070 con 0x55e9fca96710 2023-09-23T09:38:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 20 client.16455 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcafadd0 0x55e9fcafd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 4 client.16455 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:38:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.238+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e9fc9d7ad0 con 0x55e9fcafadd0 2023-09-23T09:38:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.239+0000 7f95c3fff700 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcafadd0 0x55e9fcafd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.239+0000 7f95a37fe700 10 client.16455 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.239+0000 7f95c3fff700 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcafadd0 0x55e9fcafd1f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78f5f3ccd313ca5 server_cookie=4a109ed2ea153490 in_seq=0 out_seq=0 2023-09-23T09:38:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95a37fe700 1 -- 172.21.15.45:0/2283101983 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e9fc9d7ad0 con 0x55e9fcafadd0 2023-09-23T09:38:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95a37fe700 10 client.16455 handle_command_reply: tid=0 2023-09-23T09:38:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 client.16455 shutdown 2023-09-23T09:38:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 msgr2=0x55e9fca8f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8f610 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f95bc099d90 tx=0x7f95bc004ad0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 shutdown_connections 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e9fcafadd0 0x55e9fcafd1f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e9fca96710 0x55e9fca8f610 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9fca8af40 0x55e9fca90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 --2- 172.21.15.45:0/2283101983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e9fca8b860 0x55e9fca90f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 >> 172.21.15.45:0/2283101983 conn(0x55e9fca00200 msgr2=0x55e9fca8e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95a37fe700 0 client.16455 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 shutdown_connections 2023-09-23T09:38:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 1 -- 172.21.15.45:0/2283101983 wait complete. 2023-09-23T09:38:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.262+0000 7f95d64f1b80 20 client.16455 trim_cache size 0 max 16384 2023-09-23T09:38:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:38:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:38:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:38:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:38:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95ac0a1860 msgr2=0x7f95ac0a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95ac0a1860 0x7f95ac0a3d20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009cb0 tx=0x7f95b4009260 comp rx=0 tx=0).stop 2023-09-23T09:38:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41012b0 msgr2=0x7f95c41a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41012b0 0x7f95c41a9920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00efb0 tx=0x7f95bc00c3a0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 shutdown_connections 2023-09-23T09:38:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41012b0 0x7f95c41a9920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c40ff8a0 0x7f95c4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95ac0a1860 0x7f95ac0a3d20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 --2- 172.21.15.45:0/4293955328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95c41062c0 0x7f95c4100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 >> 172.21.15.45:0/4293955328 conn(0x7f95c4076140 msgr2=0x7f95c4104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 shutdown_connections 2023-09-23T09:38:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:01.263+0000 7f95ca307700 1 -- 172.21.15.45:0/4293955328 wait complete. 2023-09-23T09:38:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 495.0 s, 1555.1 tps, lat 61.783 ms stddev 73.155 2023-09-23T09:38:02.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:01 smithi098 ceph-mon[102386]: pgmap v265: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 20 MiB/s wr, 1.14k op/s 2023-09-23T09:38:02.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:01 smithi045 ceph-mon[98627]: pgmap v265: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 20 MiB/s wr, 1.14k op/s 2023-09-23T09:38:02.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:01 smithi123 ceph-mon[103577]: pgmap v265: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 20 MiB/s wr, 1.14k op/s 2023-09-23T09:38:02.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:38:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.500+0000 7f03d11b5700 1 -- 172.21.15.45:0/1772614125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03cc10a0e0 msgr2=0x7f03cc112830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.500+0000 7f03d11b5700 1 --2- 172.21.15.45:0/1772614125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03cc10a0e0 0x7f03cc112830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0098f0 tx=0x7f03bc009c00 comp rx=0 tx=0).stop 2023-09-23T09:38:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.500+0000 7f03d11b5700 1 -- 172.21.15.45:0/1772614125 shutdown_connections 2023-09-23T09:38:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.500+0000 7f03d11b5700 1 --2- 172.21.15.45:0/1772614125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03cc112d70 0x7f03cc115160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.500+0000 7f03d11b5700 1 --2- 172.21.15.45:0/1772614125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 0x7f03cc109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.500+0000 7f03d11b5700 1 --2- 172.21.15.45:0/1772614125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03cc10a0e0 0x7f03cc112830 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.500+0000 7f03d11b5700 1 -- 172.21.15.45:0/1772614125 >> 172.21.15.45:0/1772614125 conn(0x7f03cc06d340 msgr2=0x7f03cc06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.502+0000 7f03d11b5700 1 -- 172.21.15.45:0/1772614125 shutdown_connections 2023-09-23T09:38:02.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.506+0000 7f03d11b5700 1 -- 172.21.15.45:0/1772614125 wait complete. 2023-09-23T09:38:02.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.507+0000 7f03d11b5700 1 Processor -- start 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.514+0000 7f03d11b5700 1 -- start start 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.514+0000 7f03d11b5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 0x7f03cc1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.514+0000 7f03d11b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03cc10a0e0 0x7f03cc1a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.514+0000 7f03d11b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03cc112d70 0x7f03cc1aa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.514+0000 7f03d11b5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc1aad60 con 0x7f03cc112d70 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.514+0000 7f03d11b5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc1aaed0 con 0x7f03cc1097c0 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.514+0000 7f03d11b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc1ab090 con 0x7f03cc10a0e0 2023-09-23T09:38:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 0x7f03cc1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 0x7f03cc1a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56722/0 (socket says 172.21.15.45:56722) 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 -- 172.21.15.45:0/3728962258 learned_addr learned my addr 172.21.15.45:0/3728962258 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 -- 172.21.15.45:0/3728962258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03cc10a0e0 msgr2=0x7f03cc1a62c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03cc10a0e0 0x7f03cc1a62c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 -- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03cc112d70 msgr2=0x7f03cc1aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03cc112d70 0x7f03cc1aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 -- 172.21.15.45:0/3728962258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc009520 con 0x7f03cc1097c0 2023-09-23T09:38:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.515+0000 7f03cad9d700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 0x7f03cc1a5d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ba50 tx=0x7f03c400be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.518+0000 7f03b3fff700 1 -- 172.21.15.45:0/3728962258 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400cab0 con 0x7f03cc1097c0 2023-09-23T09:38:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.518+0000 7f03b3fff700 1 -- 172.21.15.45:0/3728962258 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c400cc10 con 0x7f03cc1097c0 2023-09-23T09:38:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.518+0000 7f03b3fff700 1 -- 172.21.15.45:0/3728962258 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4007660 con 0x7f03cc1097c0 2023-09-23T09:38:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.519+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc1ab370 con 0x7f03cc1097c0 2023-09-23T09:38:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.519+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc19d4a0 con 0x7f03cc1097c0 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.524+0000 7f03dd39fb80 1 Processor -- start 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.527+0000 7f03dd39fb80 1 -- start start 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e8937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893d20 0x562a6e89e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e7fc460 con 0x562a6e893400 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e7edc20 con 0x562a6e89ebd0 2023-09-23T09:38:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e7eb540 con 0x562a6e893d20 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03cb59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e8937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03cb59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e8937e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35376/0 (socket says 172.21.15.45:35376) 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.528+0000 7f03cb59e700 1 -- 172.21.15.45:0/3367194172 learned_addr learned my addr 172.21.15.45:0/3367194172 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.529+0000 7f03cb59e700 1 -- 172.21.15.45:0/3367194172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893d20 msgr2=0x562a6e89e690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.529+0000 7f03cb59e700 1 --2- 172.21.15.45:0/3367194172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893d20 0x562a6e89e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.529+0000 7f03cb59e700 1 -- 172.21.15.45:0/3367194172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 msgr2=0x562a6e8a0fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.529+0000 7f03cb59e700 1 --2- 172.21.15.45:0/3367194172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a0fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.529+0000 7f03cb59e700 1 -- 172.21.15.45:0/3367194172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6e8498d0 con 0x562a6e893400 2023-09-23T09:38:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.529+0000 7f03cb59e700 1 --2- 172.21.15.45:0/3367194172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e8937e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009bc0 tx=0x7f03c0009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5c9c6876d004a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.530+0000 7f03b17fa700 1 -- 172.21.15.45:0/3367194172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0030af0 con 0x562a6e893400 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.530+0000 7f03b17fa700 1 -- 172.21.15.45:0/3367194172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c0004cf0 con 0x562a6e893400 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.530+0000 7f03b17fa700 1 -- 172.21.15.45:0/3367194172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00055a0 con 0x562a6e893400 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.531+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3367194172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 msgr2=0x562a6e8937e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.531+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3367194172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e8937e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009bc0 tx=0x7f03c0009ed0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.532+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3367194172 shutdown_connections 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.532+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3367194172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.532+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3367194172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e8937e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.532+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3367194172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893d20 0x562a6e89e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.532+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3367194172 >> 172.21.15.45:0/3367194172 conn(0x562a6e808260 msgr2=0x562a6e88f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.534+0000 7f03b3fff700 1 -- 172.21.15.45:0/3728962258 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f03c4014990 con 0x7f03cc1097c0 2023-09-23T09:38:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.535+0000 7f03b3fff700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f03b40a18a0 0x7f03b40a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.535+0000 7f03b3fff700 1 -- 172.21.15.45:0/3728962258 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f03c4006e40 con 0x7f03cc1097c0 2023-09-23T09:38:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.537+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3367194172 shutdown_connections 2023-09-23T09:38:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.538+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3367194172 wait complete. 2023-09-23T09:38:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.539+0000 7f03ca59c700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f03b40a18a0 0x7f03b40a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.539+0000 7f03dd39fb80 1 -- start start 2023-09-23T09:38:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.539+0000 7f03dd39fb80 1 Processor -- start 2023-09-23T09:38:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.540+0000 7f03ca59c700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f03b40a18a0 0x7f03b40a3d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009230 tx=0x7f03bc0058e0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e89b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e893d20 0x562a6e89be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e89ebd0 0x562a6e89c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e7e6e30 con 0x562a6e893400 2023-09-23T09:38:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e783710 con 0x562a6e893d20 2023-09-23T09:38:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e7833b0 con 0x562a6e89ebd0 2023-09-23T09:38:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03dd39fb80 10 client.0 _ll_register_callbacks cb 0x562a6e5e0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03ca59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e893d20 0x562a6e89be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03ca59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e893d20 0x562a6e89be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56730/0 (socket says 172.21.15.45:56730) 2023-09-23T09:38:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.543+0000 7f03ca59c700 1 -- 172.21.15.45:0/2014470394 learned_addr learned my addr 172.21.15.45:0/2014470394 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.544+0000 7f03cb59e700 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e89ebd0 0x562a6e89c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.544+0000 7f03ca59c700 1 -- 172.21.15.45:0/2014470394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e89ebd0 msgr2=0x562a6e89c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.544+0000 7f03ca59c700 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e89ebd0 0x562a6e89c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.544+0000 7f03ca59c700 1 -- 172.21.15.45:0/2014470394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 msgr2=0x562a6e89b8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:38:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.544+0000 7f03ca59c700 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e89b8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.544+0000 7f03ca59c700 1 -- 172.21.15.45:0/2014470394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c00098a0 con 0x562a6e893d20 2023-09-23T09:38:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.544+0000 7f03ca59c700 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e893d20 0x562a6e89be00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0098f0 tx=0x7f03bc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9f53c6d5ed2dfcd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.548+0000 7f03aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:38:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.548+0000 7f03aeff5700 1 -- 172.21.15.45:0/2014470394 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01c070 con 0x562a6e893d20 2023-09-23T09:38:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.548+0000 7f03aeff5700 1 -- 172.21.15.45:0/2014470394 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f03bc005040 con 0x562a6e893d20 2023-09-23T09:38:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.548+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a6e887250 con 0x562a6e893d20 2023-09-23T09:38:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.550+0000 7f03aeff5700 1 -- 172.21.15.45:0/2014470394 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03bc005200 con 0x562a6e893d20 2023-09-23T09:38:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.552+0000 7f03dd39fb80 10 client.26147 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.552+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x562a6e611de0 con 0x562a6e893d20 2023-09-23T09:38:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.552+0000 7f03aeff5700 1 -- 172.21.15.45:0/2014470394 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a6e887250 con 0x562a6e893d20 2023-09-23T09:38:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03aeff5700 1 -- 172.21.15.45:0/2014470394 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f03bc02d440 con 0x562a6e893d20 2023-09-23T09:38:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03dd39fb80 10 client.26147 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03dd39fb80 10 client.26147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03dd39fb80 20 client.26147 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e8ae6e0 0x562a6e8b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03dd39fb80 4 client.26147 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:38:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a6e7dfe10 con 0x562a6e8ae6e0 2023-09-23T09:38:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03cad9d700 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e8ae6e0 0x562a6e8b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.553+0000 7f03cad9d700 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e8ae6e0 0x562a6e8b0b00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4adef701bcda7e54 server_cookie=95077d6f695cb524 in_seq=0 out_seq=0 2023-09-23T09:38:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.554+0000 7f03aeff5700 10 client.26147 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.554+0000 7f03aeff5700 1 -- 172.21.15.45:0/2014470394 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562a6e7dfe10 con 0x562a6e8ae6e0 2023-09-23T09:38:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.554+0000 7f03aeff5700 10 client.26147 handle_command_reply: tid=0 2023-09-23T09:38:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.554+0000 7f03dd39fb80 1 client.26147 shutdown 2023-09-23T09:38:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.555+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e893d20 msgr2=0x562a6e89be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.555+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e893d20 0x562a6e89be00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0098f0 tx=0x7f03bc009c00 comp rx=0 tx=0).stop 2023-09-23T09:38:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 shutdown_connections 2023-09-23T09:38:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e8ae6e0 0x562a6e8b0b00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e89ebd0 0x562a6e89c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893400 0x562a6e89b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/2014470394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e893d20 0x562a6e89be00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 >> 172.21.15.45:0/2014470394 conn(0x562a6e808260 msgr2=0x562a6e88f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03aeff5700 0 client.26147 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 shutdown_connections 2023-09-23T09:38:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.558+0000 7f03dd39fb80 1 -- 172.21.15.45:0/2014470394 wait complete. 2023-09-23T09:38:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.559+0000 7f03dd39fb80 20 client.26147 trim_cache size 0 max 16384 2023-09-23T09:38:02.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 Processor -- start 2023-09-23T09:38:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 -- start start 2023-09-23T09:38:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e89c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e667920 con 0x562a6e893d20 2023-09-23T09:38:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e8a5cb0 con 0x562a6e89ebd0 2023-09-23T09:38:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.593+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e8977e0 con 0x562a6e893400 2023-09-23T09:38:02.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.595+0000 7f03cad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.595+0000 7f03cad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37494/0 (socket says 172.21.15.45:37494) 2023-09-23T09:38:02.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.595+0000 7f03cad9d700 1 -- 172.21.15.45:0/3386782716 learned_addr learned my addr 172.21.15.45:0/3386782716 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:02.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.595+0000 7f03cad9d700 1 -- 172.21.15.45:0/3386782716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 msgr2=0x562a6e89c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:02.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.595+0000 7f03cad9d700 1 --2- 172.21.15.45:0/3386782716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e89c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.595+0000 7f03cad9d700 1 -- 172.21.15.45:0/3386782716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 msgr2=0x562a6e8a5710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:38:02.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.595+0000 7f03cad9d700 1 --2- 172.21.15.45:0/3386782716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a5710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.596+0000 7f03cad9d700 1 -- 172.21.15.45:0/3386782716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6e5fac60 con 0x562a6e893400 2023-09-23T09:38:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.596+0000 7f03cad9d700 1 --2- 172.21.15.45:0/3386782716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8962a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f03c400bfd0 tx=0x7f03c4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35dd7bac684fbfea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.596+0000 7f03b1ffb700 1 -- 172.21.15.45:0/3386782716 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4097db0 con 0x562a6e893400 2023-09-23T09:38:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.596+0000 7f03b1ffb700 1 -- 172.21.15.45:0/3386782716 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c400cd80 con 0x562a6e893400 2023-09-23T09:38:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.596+0000 7f03b1ffb700 1 -- 172.21.15.45:0/3386782716 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4099160 con 0x562a6e893400 2023-09-23T09:38:02.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3386782716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 msgr2=0x562a6e8962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3386782716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8962a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f03c400bfd0 tx=0x7f03c4000f40 comp rx=0 tx=0).stop 2023-09-23T09:38:02.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3386782716 shutdown_connections 2023-09-23T09:38:02.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3386782716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e89c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3386782716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8962a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/3386782716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3386782716 >> 172.21.15.45:0/3386782716 conn(0x562a6e808260 msgr2=0x562a6e896c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:02.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3386782716 shutdown_connections 2023-09-23T09:38:02.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.597+0000 7f03dd39fb80 1 -- 172.21.15.45:0/3386782716 wait complete. 2023-09-23T09:38:02.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 -- start start 2023-09-23T09:38:02.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 Processor -- start 2023-09-23T09:38:02.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e8a41a0 con 0x562a6e893d20 2023-09-23T09:38:02.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e8a42e0 con 0x562a6e89ebd0 2023-09-23T09:38:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6e8a44a0 con 0x562a6e893400 2023-09-23T09:38:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03dd39fb80 10 client.0 _ll_register_callbacks cb 0x7f03bc00f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.598+0000 7f03ca59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35392/0 (socket says 172.21.15.45:35392) 2023-09-23T09:38:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 -- 172.21.15.45:0/82704887 learned_addr learned my addr 172.21.15.45:0/82704887 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03cad9d700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03cb59e700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 -- 172.21.15.45:0/82704887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 msgr2=0x562a6e8a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 -- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 msgr2=0x562a6e8a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 -- 172.21.15.45:0/82704887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc19d4a0 con 0x562a6e893d20 2023-09-23T09:38:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03cad9d700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03cb59e700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:38:02.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03ca59c700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a3720 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00fab0 tx=0x7f03bc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f8a538386d078a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:02.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.599+0000 7f03aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:38:02.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.601+0000 7f03aeff5700 1 -- 172.21.15.45:0/82704887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01c070 con 0x562a6e893d20 2023-09-23T09:38:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.601+0000 7f03aeff5700 1 -- 172.21.15.45:0/82704887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f03bc0056a0 con 0x562a6e893d20 2023-09-23T09:38:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.606+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03c0004cf0 con 0x562a6e893d20 2023-09-23T09:38:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.606+0000 7f03aeff5700 1 -- 172.21.15.45:0/82704887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03bc005950 con 0x562a6e893d20 2023-09-23T09:38:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.606+0000 7f03dd39fb80 10 client.16464 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.606+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x562a6e638250 con 0x562a6e893d20 2023-09-23T09:38:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03aeff5700 1 -- 172.21.15.45:0/82704887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c0004cf0 con 0x562a6e893d20 2023-09-23T09:38:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03aeff5700 1 -- 172.21.15.45:0/82704887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f03bc023800 con 0x562a6e893d20 2023-09-23T09:38:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03dd39fb80 10 client.16464 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03dd39fb80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03dd39fb80 20 client.16464 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e903770 0x562a6e905b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03dd39fb80 4 client.16464 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:38:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.607+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562a6e7dfb30 con 0x562a6e903770 2023-09-23T09:38:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.613+0000 7f03cad9d700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e903770 0x562a6e905b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.613+0000 7f03cad9d700 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e903770 0x562a6e905b90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c36b40c25f2cfb5 server_cookie=997312813dbe06e1 in_seq=0 out_seq=0 2023-09-23T09:38:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.613+0000 7f03aeff5700 10 client.16464 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03aeff5700 1 -- 172.21.15.45:0/82704887 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562a6e7dfb30 con 0x562a6e903770 2023-09-23T09:38:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03aeff5700 10 client.16464 handle_command_reply: tid=0 2023-09-23T09:38:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 client.16464 shutdown 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 msgr2=0x562a6e8a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a3720 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00fab0 tx=0x7f03bc005270 comp rx=0 tx=0).stop 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 shutdown_connections 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562a6e903770 0x562a6e905b90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562a6e89ebd0 0x562a6e8a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a6e893400 0x562a6e8a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 --2- 172.21.15.45:0/82704887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a6e893d20 0x562a6e8a3720 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 >> 172.21.15.45:0/82704887 conn(0x562a6e808260 msgr2=0x562a6e89beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03aeff5700 0 client.16464 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 shutdown_connections 2023-09-23T09:38:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.614+0000 7f03dd39fb80 1 -- 172.21.15.45:0/82704887 wait complete. 2023-09-23T09:38:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.615+0000 7f03dd39fb80 20 client.16464 trim_cache size 0 max 16384 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:38:02.619 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:38:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f03b40a18a0 msgr2=0x7f03b40a3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f03b40a18a0 0x7f03b40a3d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009230 tx=0x7f03bc0058e0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 msgr2=0x7f03cc1a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 0x7f03cc1a5d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ba50 tx=0x7f03c400be10 comp rx=0 tx=0).stop 2023-09-23T09:38:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 shutdown_connections 2023-09-23T09:38:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f03b40a18a0 0x7f03b40a3d60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03cc112d70 0x7f03cc1aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1097c0 0x7f03cc1a5d80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 --2- 172.21.15.45:0/3728962258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03cc10a0e0 0x7f03cc1a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.616+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 >> 172.21.15.45:0/3728962258 conn(0x7f03cc06d340 msgr2=0x7f03cc070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.618+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 shutdown_connections 2023-09-23T09:38:02.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:02.618+0000 7f03d11b5700 1 -- 172.21.15.45:0/3728962258 wait complete. 2023-09-23T09:38:02.644 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:38:04.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:03 smithi098 ceph-mon[102386]: pgmap v266: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 595 op/s 2023-09-23T09:38:04.086 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:03 smithi045 ceph-mon[98627]: pgmap v266: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 595 op/s 2023-09-23T09:38:04.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:03 smithi123 ceph-mon[103577]: pgmap v266: 97 pgs: 97 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 595 op/s 2023-09-23T09:38:06.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:05 smithi098 ceph-mon[102386]: pgmap v267: 97 pgs: 97 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 29 MiB/s wr, 1.85k op/s 2023-09-23T09:38:06.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:05 smithi098 ceph-mon[102386]: osdmap e92: 12 total, 12 up, 12 in 2023-09-23T09:38:06.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:05 smithi045 ceph-mon[98627]: pgmap v267: 97 pgs: 97 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 29 MiB/s wr, 1.85k op/s 2023-09-23T09:38:06.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:05 smithi045 ceph-mon[98627]: osdmap e92: 12 total, 12 up, 12 in 2023-09-23T09:38:06.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:05 smithi123 ceph-mon[103577]: pgmap v267: 97 pgs: 97 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 29 MiB/s wr, 1.85k op/s 2023-09-23T09:38:06.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:05 smithi123 ceph-mon[103577]: osdmap e92: 12 total, 12 up, 12 in 2023-09-23T09:38:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 500.0 s, 1148.1 tps, lat 90.311 ms stddev 168.424 2023-09-23T09:38:08.047 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:07 smithi098 ceph-mon[102386]: pgmap v269: 97 pgs: 97 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 26 MiB/s wr, 1.91k op/s 2023-09-23T09:38:08.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:07 smithi045 ceph-mon[98627]: pgmap v269: 97 pgs: 97 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 26 MiB/s wr, 1.91k op/s 2023-09-23T09:38:08.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:07 smithi123 ceph-mon[103577]: pgmap v269: 97 pgs: 97 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 26 MiB/s wr, 1.91k op/s 2023-09-23T09:38:10.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:09 smithi098 ceph-mon[102386]: pgmap v270: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2023-09-23T09:38:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:09 smithi045 ceph-mon[98627]: pgmap v270: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2023-09-23T09:38:10.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:09 smithi123 ceph-mon[103577]: pgmap v270: 97 pgs: 97 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2023-09-23T09:38:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 505.0 s, 1793.4 tps, lat 55.776 ms stddev 62.663 2023-09-23T09:38:12.048 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:11 smithi098 ceph-mon[102386]: pgmap v271: 97 pgs: 97 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-09-23T09:38:12.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:11 smithi045 ceph-mon[98627]: pgmap v271: 97 pgs: 97 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-09-23T09:38:12.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:11 smithi123 ceph-mon[103577]: pgmap v271: 97 pgs: 97 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-09-23T09:38:14.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:13 smithi098 ceph-mon[102386]: pgmap v272: 97 pgs: 97 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-09-23T09:38:14.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:13 smithi045 ceph-mon[98627]: pgmap v272: 97 pgs: 97 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-09-23T09:38:14.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:13 smithi123 ceph-mon[103577]: pgmap v272: 97 pgs: 97 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-09-23T09:38:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 -- 172.21.15.45:0/2873125166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db80fff20 msgr2=0x7f2db8100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 --2- 172.21.15.45:0/2873125166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db80fff20 0x7f2db8100300 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009990 tx=0x7f2dac009ca0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 -- 172.21.15.45:0/2873125166 shutdown_connections 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 --2- 172.21.15.45:0/2873125166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db8105040 0x7f2db810cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 --2- 172.21.15.45:0/2873125166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db80fff20 0x7f2db8100300 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 --2- 172.21.15.45:0/2873125166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db81008d0 0x7f2db8104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 -- 172.21.15.45:0/2873125166 >> 172.21.15.45:0/2873125166 conn(0x7f2db80fb780 msgr2=0x7f2db80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 -- 172.21.15.45:0/2873125166 shutdown_connections 2023-09-23T09:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.399+0000 7f2dbea70700 1 -- 172.21.15.45:0/2873125166 wait complete. 2023-09-23T09:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 Processor -- start 2023-09-23T09:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 -- start start 2023-09-23T09:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db80fff20 0x7f2db8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db8105040 0x7f2db8199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db819dd70 0x7f2db819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db819e7f0 con 0x7f2db819dd70 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db819e960 con 0x7f2db80fff20 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbea70700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db819eb20 con 0x7f2db8105040 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbd00d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db819dd70 0x7f2db819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2db7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db8105040 0x7f2db8199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbd00d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db819dd70 0x7f2db819e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35410/0 (socket says 172.21.15.45:35410) 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2db7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db8105040 0x7f2db8199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37514/0 (socket says 172.21.15.45:37514) 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbd00d700 1 -- 172.21.15.45:0/3087419077 learned_addr learned my addr 172.21.15.45:0/3087419077 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.400+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db80fff20 0x7f2db8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbd00d700 1 -- 172.21.15.45:0/3087419077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db8105040 msgr2=0x7f2db8199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db8105040 0x7f2db8199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbd00d700 1 -- 172.21.15.45:0/3087419077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db80fff20 msgr2=0x7f2db8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db80fff20 0x7f2db8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbd00d700 1 -- 172.21.15.45:0/3087419077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac0095f0 con 0x7f2db819dd70 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2db7fff700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db8105040 0x7f2db8199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db819dd70 0x7f2db819e1f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2db000ef30 tx=0x7f2db000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2db5ffb700 1 -- 172.21.15.45:0/3087419077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0018070 con 0x7f2db819dd70 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2db5ffb700 1 -- 172.21.15.45:0/3087419077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db000f040 con 0x7f2db819dd70 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db81a6020 con 0x7f2db819dd70 2023-09-23T09:38:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2db5ffb700 1 -- 172.21.15.45:0/3087419077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000cd00 con 0x7f2db819dd70 2023-09-23T09:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.401+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db81a64b0 con 0x7f2db819dd70 2023-09-23T09:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.407+0000 7f2dcac5ab80 1 Processor -- start 2023-09-23T09:38:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.421+0000 7f2db5ffb700 1 -- 172.21.15.45:0/3087419077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f2db0003a00 con 0x7f2db819dd70 2023-09-23T09:38:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.422+0000 7f2db5ffb700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2da80a1790 0x7f2da80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.422+0000 7f2db5ffb700 1 -- 172.21.15.45:0/3087419077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2db00cceb0 con 0x7f2db819dd70 2023-09-23T09:38:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.423+0000 7f2dcac5ab80 1 -- start start 2023-09-23T09:38:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.423+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.423+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.423+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.423+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436880e400 con 0x5643688c9f30 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.423+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643687ffbc0 con 0x5643688ae070 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.423+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643687fd4e0 con 0x5643688ad750 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.429+0000 7f2dbc80c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688adb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37520/0 (socket says 172.21.15.45:37520) 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 -- 172.21.15.45:0/3628894289 learned_addr learned my addr 172.21.15.45:0/3628894289 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 -- 172.21.15.45:0/3628894289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 msgr2=0x5643688b89e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/3628894289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688b89e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 -- 172.21.15.45:0/3628894289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 msgr2=0x5643688cc320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:38:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/3628894289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688cc320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 -- 172.21.15.45:0/3628894289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56436885b870 con 0x5643688ad750 2023-09-23T09:38:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.430+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/3628894289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688adb30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2dac005f20 tx=0x7f2dac00b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c872955a938fd9c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.434+0000 7f2db7fff700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2da80a1790 0x7f2da80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.436+0000 7f2da2ffd700 1 -- 172.21.15.45:0/3628894289 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac01d070 con 0x5643688ad750 2023-09-23T09:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.437+0000 7f2da2ffd700 1 -- 172.21.15.45:0/3628894289 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dac01e040 con 0x5643688ad750 2023-09-23T09:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.437+0000 7f2da2ffd700 1 -- 172.21.15.45:0/3628894289 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0049a0 con 0x5643688ad750 2023-09-23T09:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.437+0000 7f2db7fff700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2da80a1790 0x7f2da80a3c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2db819a9e0 tx=0x7f2da4009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.438+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/3628894289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 msgr2=0x5643688adb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.438+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/3628894289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688adb30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2dac005f20 tx=0x7f2dac00b5b0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.449+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/3628894289 shutdown_connections 2023-09-23T09:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.449+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/3628894289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.449+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/3628894289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688adb30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.449+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/3628894289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.449+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/3628894289 >> 172.21.15.45:0/3628894289 conn(0x56436881a200 msgr2=0x5643688a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.449+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/3628894289 shutdown_connections 2023-09-23T09:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.449+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/3628894289 wait complete. 2023-09-23T09:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 -- start start 2023-09-23T09:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 Processor -- start 2023-09-23T09:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643687f8dd0 con 0x5643688c9f30 2023-09-23T09:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643687956b0 con 0x5643688ae070 2023-09-23T09:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368795350 con 0x5643688ad750 2023-09-23T09:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.450+0000 7f2dcac5ab80 10 client.0 _ll_register_callbacks cb 0x5643685f27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35416/0 (socket says 172.21.15.45:35416) 2023-09-23T09:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 -- 172.21.15.45:0/4070572016 learned_addr learned my addr 172.21.15.45:0/4070572016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 -- 172.21.15.45:0/4070572016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 msgr2=0x5643688b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 -- 172.21.15.45:0/4070572016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 msgr2=0x5643688b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 -- 172.21.15.45:0/4070572016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db810d430 con 0x5643688c9f30 2023-09-23T09:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688b6700 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2db00a1bb0 tx=0x7f2db0099740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bc3d4f29ca7f3f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.451+0000 7f2d97fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.454+0000 7f2d97fff700 1 -- 172.21.15.45:0/4070572016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db001b320 con 0x5643688c9f30 2023-09-23T09:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.454+0000 7f2d97fff700 1 -- 172.21.15.45:0/4070572016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2db009a4e0 con 0x5643688c9f30 2023-09-23T09:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.454+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564368899090 con 0x5643688c9f30 2023-09-23T09:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.454+0000 7f2d97fff700 1 -- 172.21.15.45:0/4070572016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db000ce60 con 0x5643688c9f30 2023-09-23T09:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 10 client.16473 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x564368623d80 con 0x5643688c9f30 2023-09-23T09:38:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2d97fff700 1 -- 172.21.15.45:0/4070572016 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db00b6070 con 0x5643688c9f30 2023-09-23T09:38:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2d97fff700 1 -- 172.21.15.45:0/4070572016 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2db00992d0 con 0x5643688c9f30 2023-09-23T09:38:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 10 client.16473 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 20 client.16473 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643688d99f0 0x5643688dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 4 client.16473 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:38:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.460+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643687f1db0 con 0x5643688d99f0 2023-09-23T09:38:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.466+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643688d99f0 0x5643688dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.467+0000 7f2d97fff700 10 client.16473 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.468+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643688d99f0 0x5643688dbe10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9436da76e9a86ef9 server_cookie=f4018e55544cb1a4 in_seq=0 out_seq=0 2023-09-23T09:38:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2d97fff700 1 -- 172.21.15.45:0/4070572016 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5643687f1db0 con 0x5643688d99f0 2023-09-23T09:38:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2d97fff700 10 client.16473 handle_command_reply: tid=0 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 client.16473 shutdown 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 msgr2=0x5643688b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688b6700 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2db00a1bb0 tx=0x7f2db0099740 comp rx=0 tx=0).stop 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 shutdown_connections 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643688d99f0 0x5643688dbe10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688c9f30 0x5643688b6700 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688ad750 0x5643688b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/4070572016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 >> 172.21.15.45:0/4070572016 conn(0x56436881a200 msgr2=0x5643688a0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.469+0000 7f2d97fff700 0 client.16473 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.470+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 shutdown_connections 2023-09-23T09:38:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.470+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/4070572016 wait complete. 2023-09-23T09:38:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.471+0000 7f2dcac5ab80 20 client.16473 trim_cache size 0 max 16384 2023-09-23T09:38:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.494+0000 7f2dcac5ab80 1 Processor -- start 2023-09-23T09:38:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.498+0000 7f2dcac5ab80 1 -- start start 2023-09-23T09:38:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.498+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ad750 0x5643688a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.498+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ae070 0x5643688a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.498+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.498+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643686798c0 con 0x5643688ae070 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.498+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436889dbd0 con 0x5643688ad750 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.498+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643688b8db0 con 0x5643688c9f30 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ae070 0x5643688a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ae070 0x5643688a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35426/0 (socket says 172.21.15.45:35426) 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 -- 172.21.15.45:0/554904288 learned_addr learned my addr 172.21.15.45:0/554904288 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ad750 0x5643688a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 -- 172.21.15.45:0/554904288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 msgr2=0x5643688b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 -- 172.21.15.45:0/554904288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ad750 msgr2=0x5643688a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ad750 0x5643688a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 -- 172.21.15.45:0/554904288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56436860cc00 con 0x5643688ae070 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ad750 0x5643688a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.499+0000 7f2db7fff700 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ae070 0x5643688a1220 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2da40095f0 tx=0x7f2da4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8de3d71c9ceacd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.501+0000 7f2da37fe700 1 -- 172.21.15.45:0/554904288 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4003a90 con 0x5643688ae070 2023-09-23T09:38:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.501+0000 7f2da37fe700 1 -- 172.21.15.45:0/554904288 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da4003bf0 con 0x5643688ae070 2023-09-23T09:38:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.501+0000 7f2da37fe700 1 -- 172.21.15.45:0/554904288 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da401e650 con 0x5643688ae070 2023-09-23T09:38:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/554904288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ae070 msgr2=0x5643688a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ae070 0x5643688a1220 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2da40095f0 tx=0x7f2da4013040 comp rx=0 tx=0).stop 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/554904288 shutdown_connections 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ad750 0x5643688a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/554904288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ae070 0x5643688a1220 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/554904288 >> 172.21.15.45:0/554904288 conn(0x56436881a200 msgr2=0x5643688b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/554904288 shutdown_connections 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/554904288 wait complete. 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 -- start start 2023-09-23T09:38:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.502+0000 7f2dcac5ab80 1 Processor -- start 2023-09-23T09:38:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ad750 0x5643688d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dcac5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643688cf660 con 0x5643688ad750 2023-09-23T09:38:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643688cf7d0 con 0x5643688ae070 2023-09-23T09:38:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dcac5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643688cf990 con 0x5643688c9f30 2023-09-23T09:38:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dcac5ab80 10 client.0 _ll_register_callbacks cb 0x7f2db0096210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ad750 0x5643688d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ad750 0x5643688d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35430/0 (socket says 172.21.15.45:35430) 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 -- 172.21.15.45:0/2120781144 learned_addr learned my addr 172.21.15.45:0/2120781144 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2db7fff700 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbd00d700 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 -- 172.21.15.45:0/2120781144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 msgr2=0x5643688cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 -- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 msgr2=0x5643688cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 -- 172.21.15.45:0/2120781144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56436885b870 con 0x5643688ad750 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2dbc80c700 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ad750 0x5643688d0880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2dac006010 tx=0x7f2dac0094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84289e797cd84eba server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.503+0000 7f2d97fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:38:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2d97fff700 1 -- 172.21.15.45:0/2120781144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac032040 con 0x5643688ad750 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2d97fff700 1 -- 172.21.15.45:0/2120781144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2dac00bcc0 con 0x5643688ad750 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dac01e040 con 0x5643688ad750 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2d97fff700 1 -- 172.21.15.45:0/2120781144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dac01d040 con 0x5643688ad750 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 10 client.16479 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f2da4017070 con 0x5643688ad750 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2d97fff700 1 -- 172.21.15.45:0/2120781144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dac043080 con 0x5643688ad750 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2d97fff700 1 -- 172.21.15.45:0/2120781144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2dac03f510 con 0x5643688ad750 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 10 client.16479 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 20 client.16479 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56436892e860 0x564368930c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 4 client.16479 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:38:14.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.504+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5643687f1ad0 con 0x56436892e860 2023-09-23T09:38:14.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.505+0000 7f2db7fff700 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56436892e860 0x564368930c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:14.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.506+0000 7f2db7fff700 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56436892e860 0x564368930c80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45af0f0ab977d35 server_cookie=bc631d9c2cd0732b in_seq=0 out_seq=0 2023-09-23T09:38:14.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.506+0000 7f2d97fff700 10 client.16479 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2d97fff700 1 -- 172.21.15.45:0/2120781144 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5643687f1ad0 con 0x56436892e860 2023-09-23T09:38:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2d97fff700 10 client.16479 handle_command_reply: tid=0 2023-09-23T09:38:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 client.16479 shutdown 2023-09-23T09:38:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ad750 msgr2=0x5643688d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ad750 0x5643688d0880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2dac006010 tx=0x7f2dac0094b0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 shutdown_connections 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56436892e860 0x564368930c80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643688c9f30 0x5643688cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2d97fff700 0 client.16479 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643688ad750 0x5643688d0880 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 --2- 172.21.15.45:0/2120781144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643688ae070 0x5643688cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.527+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 >> 172.21.15.45:0/2120781144 conn(0x56436881a200 msgr2=0x5643688a0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 shutdown_connections 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dcac5ab80 1 -- 172.21.15.45:0/2120781144 wait complete. 2023-09-23T09:38:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dcac5ab80 20 client.16479 trim_cache size 0 max 16384 2023-09-23T09:38:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:38:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:38:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:38:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:38:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2da80a1790 msgr2=0x7f2da80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2da80a1790 0x7f2da80a3c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2db819a9e0 tx=0x7f2da4009260 comp rx=0 tx=0).stop 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db819dd70 msgr2=0x7f2db819e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db819dd70 0x7f2db819e1f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2db000ef30 tx=0x7f2db000c3a0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 shutdown_connections 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2da80a1790 0x7f2da80a3c50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db819dd70 0x7f2db819e1f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db80fff20 0x7f2db8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 --2- 172.21.15.45:0/3087419077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2db8105040 0x7f2db8199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 >> 172.21.15.45:0/3087419077 conn(0x7f2db80fb780 msgr2=0x7f2db81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.528+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 shutdown_connections 2023-09-23T09:38:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:14.529+0000 7f2dbea70700 1 -- 172.21.15.45:0/3087419077 wait complete. 2023-09-23T09:38:16.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:15 smithi098 ceph-mon[102386]: pgmap v273: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 647 op/s 2023-09-23T09:38:16.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:15 smithi098 ceph-mon[102386]: osdmap e93: 12 total, 12 up, 12 in 2023-09-23T09:38:16.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:15 smithi045 ceph-mon[98627]: pgmap v273: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 647 op/s 2023-09-23T09:38:16.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:15 smithi045 ceph-mon[98627]: osdmap e93: 12 total, 12 up, 12 in 2023-09-23T09:38:16.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:15 smithi123 ceph-mon[103577]: pgmap v273: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 647 op/s 2023-09-23T09:38:16.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:15 smithi123 ceph-mon[103577]: osdmap e93: 12 total, 12 up, 12 in 2023-09-23T09:38:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 510.0 s, 1568.6 tps, lat 63.441 ms stddev 69.945 2023-09-23T09:38:18.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:17 smithi098 ceph-mon[102386]: pgmap v275: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 646 op/s 2023-09-23T09:38:18.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:17 smithi045 ceph-mon[98627]: pgmap v275: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 646 op/s 2023-09-23T09:38:18.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:17 smithi123 ceph-mon[103577]: pgmap v275: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 646 op/s 2023-09-23T09:38:20.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:19 smithi098 ceph-mon[102386]: pgmap v276: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 644 op/s 2023-09-23T09:38:20.086 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:19 smithi045 ceph-mon[98627]: pgmap v276: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 644 op/s 2023-09-23T09:38:20.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:19 smithi123 ceph-mon[103577]: pgmap v276: 97 pgs: 97 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 644 op/s 2023-09-23T09:38:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 515.0 s, 777.0 tps, lat 126.990 ms stddev 194.157 2023-09-23T09:38:22.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:21 smithi098 ceph-mon[102386]: pgmap v277: 97 pgs: 97 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 681 op/s 2023-09-23T09:38:22.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:21 smithi045 ceph-mon[98627]: pgmap v277: 97 pgs: 97 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 681 op/s 2023-09-23T09:38:22.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:21 smithi123 ceph-mon[103577]: pgmap v277: 97 pgs: 97 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 681 op/s 2023-09-23T09:38:24.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:23 smithi098 ceph-mon[102386]: pgmap v278: 97 pgs: 97 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 682 op/s 2023-09-23T09:38:24.088 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:23 smithi045 ceph-mon[98627]: pgmap v278: 97 pgs: 97 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 682 op/s 2023-09-23T09:38:24.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:23 smithi123 ceph-mon[103577]: pgmap v278: 97 pgs: 97 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 682 op/s 2023-09-23T09:38:26.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:25 smithi098 ceph-mon[102386]: pgmap v279: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 30 MiB/s wr, 1.95k op/s 2023-09-23T09:38:26.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:25 smithi045 ceph-mon[98627]: pgmap v279: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 30 MiB/s wr, 1.95k op/s 2023-09-23T09:38:26.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:25 smithi123 ceph-mon[103577]: pgmap v279: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 30 MiB/s wr, 1.95k op/s 2023-09-23T09:38:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 520.0 s, 1316.3 tps, lat 76.716 ms stddev 89.098 2023-09-23T09:38:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 -- 172.21.15.45:0/702868072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100b10 msgr2=0x7f9e70104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 --2- 172.21.15.45:0/702868072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100b10 0x7f9e70104920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9e640099c0 tx=0x7f9e64009cd0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 -- 172.21.15.45:0/702868072 shutdown_connections 2023-09-23T09:38:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 --2- 172.21.15.45:0/702868072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e70105050 0x7f9e7010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 --2- 172.21.15.45:0/702868072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e70100160 0x7f9e70100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 --2- 172.21.15.45:0/702868072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100b10 0x7f9e70104920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 -- 172.21.15.45:0/702868072 >> 172.21.15.45:0/702868072 conn(0x7f9e70076840 msgr2=0x7f9e70076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 -- 172.21.15.45:0/702868072 shutdown_connections 2023-09-23T09:38:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.665+0000 7f9e75371700 1 -- 172.21.15.45:0/702868072 wait complete. 2023-09-23T09:38:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 Processor -- start 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 -- start start 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100160 0x7f9e70197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e70105050 0x7f9e70198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e701a0d90 0x7f9e701a3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701a3750 con 0x7f9e701a0d90 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701a3890 con 0x7f9e70105050 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e75371700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701a3a50 con 0x7f9e70100160 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e701a0d90 0x7f9e701a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e6effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100160 0x7f9e70197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e6e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e70105050 0x7f9e70198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e701a0d90 0x7f9e701a3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35448/0 (socket says 172.21.15.45:35448) 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.666+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/437472306 learned_addr learned my addr 172.21.15.45:0/437472306 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/437472306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100160 msgr2=0x7f9e70197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100160 0x7f9e70197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/437472306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e70105050 msgr2=0x7f9e70198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e70105050 0x7f9e70198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/437472306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640095f0 con 0x7f9e701a0d90 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e6effd700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100160 0x7f9e70197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e701a0d90 0x7f9e701a3180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800ef30 tx=0x7f9e6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e57fff700 1 -- 172.21.15.45:0/437472306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68018070 con 0x7f9e701a0d90 2023-09-23T09:38:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e57fff700 1 -- 172.21.15.45:0/437472306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e6800f040 con 0x7f9e701a0d90 2023-09-23T09:38:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e701a3cd0 con 0x7f9e701a0d90 2023-09-23T09:38:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e57fff700 1 -- 172.21.15.45:0/437472306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800cd00 con 0x7f9e701a0d90 2023-09-23T09:38:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.667+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e701a4160 con 0x7f9e701a0d90 2023-09-23T09:38:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.669+0000 7f9e57fff700 1 -- 172.21.15.45:0/437472306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f9e68003a00 con 0x7f9e701a0d90 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.670+0000 7f9e57fff700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9e580a1710 0x7f9e580a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.670+0000 7f9e6effd700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9e580a1710 0x7f9e580a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.670+0000 7f9e57fff700 1 -- 172.21.15.45:0/437472306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e680cce90 con 0x7f9e701a0d90 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.670+0000 7f9e6effd700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9e580a1710 0x7f9e580a3bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9e60011490 tx=0x7f9e60009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 Processor -- start 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 -- start start 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 0x557c4cab0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cabb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cabdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4ca19400 con 0x557c4cabb710 2023-09-23T09:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4ca0abc0 con 0x557c4caaff40 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4ca084e0 con 0x557c4cab0860 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cabdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 0x557c4cab0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cabdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35456/0 (socket says 172.21.15.45:35456) 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3519708362 learned_addr learned my addr 172.21.15.45:0/3519708362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6e7fc700 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cabb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3519708362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 msgr2=0x557c4cabb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cabb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3519708362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 msgr2=0x557c4cab0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 0x557c4cab0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.671+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3519708362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c4ca66870 con 0x557c4cabb710 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cabdb00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9e680a1bc0 tx=0x7f9e68099930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a634282b3ba4551a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e557fa700 1 -- 172.21.15.45:0/3519708362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e680a0070 con 0x557c4cabb710 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e557fa700 1 -- 172.21.15.45:0/3519708362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e68004ad0 con 0x557c4cabb710 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e557fa700 1 -- 172.21.15.45:0/3519708362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e680a20f0 con 0x557c4cabb710 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3519708362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 msgr2=0x557c4cabdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cabdb00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9e680a1bc0 tx=0x7f9e68099930 comp rx=0 tx=0).stop 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3519708362 shutdown_connections 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cabdb00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 0x557c4cab0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3519708362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cabb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3519708362 >> 172.21.15.45:0/3519708362 conn(0x557c4ca25200 msgr2=0x557c4caac3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3519708362 shutdown_connections 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.672+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3519708362 wait complete. 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 -- start start 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 Processor -- start 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4caaff40 0x557c4cab84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cab0860 0x557c4cab8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cab8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4ca03dd0 con 0x557c4cabb710 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4c9a06b0 con 0x557c4cab0860 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4c9a0350 con 0x557c4caaff40 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e8155bb80 10 client.0 _ll_register_callbacks cb 0x557c4c7fd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cab8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e6e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cab0860 0x557c4cab8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e6effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4caaff40 0x557c4cab84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cab8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35462/0 (socket says 172.21.15.45:35462) 2023-09-23T09:38:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.673+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3987275993 learned_addr learned my addr 172.21.15.45:0/3987275993 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.674+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3987275993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4caaff40 msgr2=0x557c4cab84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.674+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4caaff40 0x557c4cab84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.674+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3987275993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cab0860 msgr2=0x557c4cab8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.674+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cab0860 0x557c4cab8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.674+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/3987275993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e701a4160 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.674+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cab8f50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9e68099d80 tx=0x7f9e680b2440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a08f6e8bcf20db7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.675+0000 7f9e52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.675+0000 7f9e52ff5700 1 -- 172.21.15.45:0/3987275993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6801b320 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.675+0000 7f9e52ff5700 1 -- 172.21.15.45:0/3987275993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e6809a4f0 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.675+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c4caa4090 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.675+0000 7f9e52ff5700 1 -- 172.21.15.45:0/3987275993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e680a20f0 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.675+0000 7f9e8155bb80 10 client.16491 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.675+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557c4c82ed80 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e52ff5700 1 -- 172.21.15.45:0/3987275993 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c4caa4090 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e52ff5700 1 -- 172.21.15.45:0/3987275993 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e680a1c60 con 0x557c4cabb710 2023-09-23T09:38:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e8155bb80 10 client.16491 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e8155bb80 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e8155bb80 20 client.16491 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cacade0 0x557c4cacd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e8155bb80 4 client.16491 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.676+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c4c9fcdb0 con 0x557c4cacade0 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.677+0000 7f9e6effd700 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cacade0 0x557c4cacd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.677+0000 7f9e6effd700 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cacade0 0x557c4cacd1d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50605a6e8a4de26 server_cookie=f4001d5797100e36 in_seq=0 out_seq=0 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.677+0000 7f9e52ff5700 10 client.16491 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e52ff5700 1 -- 172.21.15.45:0/3987275993 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c4c9fcdb0 con 0x557c4cacade0 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e52ff5700 10 client.16491 handle_command_reply: tid=0 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 client.16491 shutdown 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 msgr2=0x557c4cab8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cab8f50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9e68099d80 tx=0x7f9e680b2440 comp rx=0 tx=0).stop 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 shutdown_connections 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cacade0 0x557c4cacd1d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cabb710 0x557c4cab8f50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e52ff5700 0 client.16491 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4caaff40 0x557c4cab84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/3987275993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cab0860 0x557c4cab8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 >> 172.21.15.45:0/3987275993 conn(0x557c4ca25200 msgr2=0x557c4caabf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 shutdown_connections 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.678+0000 7f9e8155bb80 1 -- 172.21.15.45:0/3987275993 wait complete. 2023-09-23T09:38:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.679+0000 7f9e8155bb80 20 client.16491 trim_cache size 0 max 16384 2023-09-23T09:38:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 Processor -- start 2023-09-23T09:38:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 -- start start 2023-09-23T09:38:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4caaff40 0x557c4cab2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cab8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cabb710 0x557c4cab73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4c8848c0 con 0x557c4caaff40 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4cab6a70 con 0x557c4cabb710 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4cab4380 con 0x557c4cab0860 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4caaff40 0x557c4cab2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cabb710 0x557c4cab73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4caaff40 0x557c4cab2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35466/0 (socket says 172.21.15.45:35466) 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6effd700 1 -- 172.21.15.45:0/652882675 learned_addr learned my addr 172.21.15.45:0/652882675 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6e7fc700 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cab8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6effd700 1 -- 172.21.15.45:0/652882675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 msgr2=0x557c4cab8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6effd700 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cab8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6effd700 1 -- 172.21.15.45:0/652882675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cabb710 msgr2=0x557c4cab73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.696+0000 7f9e6effd700 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cabb710 0x557c4cab73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.697+0000 7f9e6effd700 1 -- 172.21.15.45:0/652882675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c4c817c00 con 0x557c4caaff40 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.697+0000 7f9e6effd700 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4caaff40 0x557c4cab2f80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9e60013010 tx=0x7f9e60014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2d0e82c9755386 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.697+0000 7f9e55ffb700 1 -- 172.21.15.45:0/652882675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60024040 con 0x557c4caaff40 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.697+0000 7f9e55ffb700 1 -- 172.21.15.45:0/652882675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e600295b0 con 0x557c4caaff40 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.697+0000 7f9e55ffb700 1 -- 172.21.15.45:0/652882675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6001e890 con 0x557c4caaff40 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.698+0000 7f9e8155bb80 1 -- 172.21.15.45:0/652882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4caaff40 msgr2=0x557c4cab2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.698+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4caaff40 0x557c4cab2f80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9e60013010 tx=0x7f9e60014000 comp rx=0 tx=0).stop 2023-09-23T09:38:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.698+0000 7f9e8155bb80 1 -- 172.21.15.45:0/652882675 shutdown_connections 2023-09-23T09:38:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.698+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4cabb710 0x557c4cab73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.698+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4caaff40 0x557c4cab2f80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.698+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/652882675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cab0860 0x557c4cab8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.698+0000 7f9e8155bb80 1 -- 172.21.15.45:0/652882675 >> 172.21.15.45:0/652882675 conn(0x557c4ca25200 msgr2=0x557c4cab6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.718+0000 7f9e8155bb80 1 -- 172.21.15.45:0/652882675 shutdown_connections 2023-09-23T09:38:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.729+0000 7f9e8155bb80 1 -- 172.21.15.45:0/652882675 wait complete. 2023-09-23T09:38:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.729+0000 7f9e8155bb80 1 -- start start 2023-09-23T09:38:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.729+0000 7f9e8155bb80 1 Processor -- start 2023-09-23T09:38:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.742+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 0x557c4cabfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.742+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cab0860 0x557c4cac03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.742+0000 7f9e8155bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cabb710 0x557c4cb17bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.742+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4cac0b20 con 0x557c4cab0860 2023-09-23T09:38:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.742+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4cb180f0 con 0x557c4caaff40 2023-09-23T09:38:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.743+0000 7f9e8155bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4cb182b0 con 0x557c4cabb710 2023-09-23T09:38:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.743+0000 7f9e8155bb80 10 client.0 _ll_register_callbacks cb 0x7f9e60004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cabb710 0x557c4cb17bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cabb710 0x557c4cb17bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37582/0 (socket says 172.21.15.45:37582) 2023-09-23T09:38:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/999220351 learned_addr learned my addr 172.21.15.45:0/999220351 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6e7fc700 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cab0860 0x557c4cac03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/999220351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 msgr2=0x557c4cabfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 0x557c4cabfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/999220351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cab0860 msgr2=0x557c4cac03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cab0860 0x557c4cac03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.750+0000 7f9e6f7fe700 1 -- 172.21.15.45:0/999220351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e60017070 con 0x557c4cabb710 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.751+0000 7f9e6f7fe700 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cabb710 0x557c4cb17bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9e68096810 tx=0x7f9e68099610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80c63e8eab535053 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.751+0000 7f9e52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.752+0000 7f9e6e7fc700 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cab0860 0x557c4cac03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.756+0000 7f9e52ff5700 1 -- 172.21.15.45:0/999220351 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68099930 con 0x557c4cabb710 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.756+0000 7f9e52ff5700 1 -- 172.21.15.45:0/999220351 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e6809a4f0 con 0x557c4cabb710 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.756+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c4c9fcad0 con 0x557c4cabb710 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.757+0000 7f9e52ff5700 1 -- 172.21.15.45:0/999220351 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e68099750 con 0x557c4cabb710 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.759+0000 7f9e8155bb80 10 client.26167 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.759+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557c4c8551f0 con 0x557c4cabb710 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.759+0000 7f9e52ff5700 1 -- 172.21.15.45:0/999220351 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c4c9fcad0 con 0x557c4cabb710 2023-09-23T09:38:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.759+0000 7f9e52ff5700 1 -- 172.21.15.45:0/999220351 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e68095be0 con 0x557c4cabb710 2023-09-23T09:38:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.765+0000 7f9e8155bb80 10 client.26167 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.765+0000 7f9e8155bb80 10 client.26167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.765+0000 7f9e8155bb80 20 client.26167 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.765+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cb19520 0x557c4cb1b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.765+0000 7f9e8155bb80 4 client.26167 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.765+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c4c84ff10 con 0x557c4cb19520 2023-09-23T09:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.765+0000 7f9e6effd700 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cb19520 0x557c4cb1b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.766+0000 7f9e52ff5700 10 client.26167 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.766+0000 7f9e6effd700 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cb19520 0x557c4cb1b940 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e6d8d3fdb07baf server_cookie=51d839acce058d8f in_seq=0 out_seq=0 2023-09-23T09:38:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.793+0000 7f9e52ff5700 1 -- 172.21.15.45:0/999220351 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c4c84ff10 con 0x557c4cb19520 2023-09-23T09:38:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.793+0000 7f9e52ff5700 10 client.26167 handle_command_reply: tid=0 2023-09-23T09:38:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.793+0000 7f9e8155bb80 1 client.26167 shutdown 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cabb710 msgr2=0x557c4cb17bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cabb710 0x557c4cb17bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9e68096810 tx=0x7f9e68099610 comp rx=0 tx=0).stop 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 shutdown_connections 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557c4cb19520 0x557c4cb1b940 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c4cabb710 0x557c4cb17bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c4caaff40 0x557c4cabfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 --2- 172.21.15.45:0/999220351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c4cab0860 0x557c4cac03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e52ff5700 0 client.26167 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 >> 172.21.15.45:0/999220351 conn(0x557c4ca25200 msgr2=0x557c4cab8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 shutdown_connections 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 1 -- 172.21.15.45:0/999220351 wait complete. 2023-09-23T09:38:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e8155bb80 20 client.26167 trim_cache size 0 max 16384 2023-09-23T09:38:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:38:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:38:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:38:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9e580a1710 msgr2=0x7f9e580a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9e580a1710 0x7f9e580a3bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9e60011490 tx=0x7f9e60009260 comp rx=0 tx=0).stop 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e701a0d90 msgr2=0x7f9e701a3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e701a0d90 0x7f9e701a3180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800ef30 tx=0x7f9e6800c3a0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 shutdown_connections 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f9e580a1710 0x7f9e580a3bd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e701a0d90 0x7f9e701a3180 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e70100160 0x7f9e70197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 --2- 172.21.15.45:0/437472306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e70105050 0x7f9e70198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.794+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 >> 172.21.15.45:0/437472306 conn(0x7f9e70076840 msgr2=0x7f9e700fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.795+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 shutdown_connections 2023-09-23T09:38:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:27.795+0000 7f9e75371700 1 -- 172.21.15.45:0/437472306 wait complete. 2023-09-23T09:38:28.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:27 smithi098 ceph-mon[102386]: pgmap v280: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.86k op/s 2023-09-23T09:38:28.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:27 smithi045 ceph-mon[98627]: pgmap v280: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.86k op/s 2023-09-23T09:38:28.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:27 smithi123 ceph-mon[103577]: pgmap v280: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.86k op/s 2023-09-23T09:38:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:29 smithi045 ceph-mon[98627]: pgmap v281: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 26 MiB/s wr, 1.70k op/s 2023-09-23T09:38:30.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:29 smithi123 ceph-mon[103577]: pgmap v281: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 26 MiB/s wr, 1.70k op/s 2023-09-23T09:38:30.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:29 smithi098 ceph-mon[102386]: pgmap v281: 97 pgs: 97 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 26 MiB/s wr, 1.70k op/s 2023-09-23T09:38:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 525.0 s, 1261.0 tps, lat 80.014 ms stddev 101.859 2023-09-23T09:38:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:31 smithi045 ceph-mon[98627]: pgmap v282: 97 pgs: 97 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2023-09-23T09:38:32.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:31 smithi123 ceph-mon[103577]: pgmap v282: 97 pgs: 97 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2023-09-23T09:38:32.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:31 smithi098 ceph-mon[102386]: pgmap v282: 97 pgs: 97 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2023-09-23T09:38:32.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:38:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.859+0000 7f95de665700 1 -- 172.21.15.45:0/3288780292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95d80ff220 msgr2=0x7f95d80ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.859+0000 7f95de665700 1 --2- 172.21.15.45:0/3288780292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95d80ff220 0x7f95d80ff6a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009990 tx=0x7f95c8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.859+0000 7f95de665700 1 -- 172.21.15.45:0/3288780292 shutdown_connections 2023-09-23T09:38:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.859+0000 7f95de665700 1 --2- 172.21.15.45:0/3288780292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d80ffbe0 0x7f95d81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.859+0000 7f95de665700 1 --2- 172.21.15.45:0/3288780292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d8105cf0 0x7f95d81060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.859+0000 7f95de665700 1 --2- 172.21.15.45:0/3288780292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95d80ff220 0x7f95d80ff6a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.859+0000 7f95de665700 1 -- 172.21.15.45:0/3288780292 >> 172.21.15.45:0/3288780292 conn(0x7f95d8077290 msgr2=0x7f95d80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.861+0000 7f95de665700 1 -- 172.21.15.45:0/3288780292 shutdown_connections 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.861+0000 7f95de665700 1 -- 172.21.15.45:0/3288780292 wait complete. 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 Processor -- start 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 -- start start 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d80ff220 0x7f95d80723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95d80ffbe0 0x7f95d80728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d806b350 0x7f95d806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d806bee0 con 0x7f95d806b350 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d806c050 con 0x7f95d80ff220 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.863+0000 7f95de665700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d806c210 con 0x7f95d80ffbe0 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d80ff220 0x7f95d80723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d80ff220 0x7f95d80723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56836/0 (socket says 172.21.15.45:56836) 2023-09-23T09:38:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 -- 172.21.15.45:0/3455907287 learned_addr learned my addr 172.21.15.45:0/3455907287 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 -- 172.21.15.45:0/3455907287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95d80ffbe0 msgr2=0x7f95d80728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95d80ffbe0 0x7f95d80728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 -- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d806b350 msgr2=0x7f95d806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d806b350 0x7f95d806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.864+0000 7f95d7fff700 1 -- 172.21.15.45:0/3455907287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80095f0 con 0x7f95d80ff220 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.865+0000 7f95d7fff700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d80ff220 0x7f95d80723b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b990 tx=0x7f95d000bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.865+0000 7f95d57fa700 1 -- 172.21.15.45:0/3455907287 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000c9b0 con 0x7f95d80ff220 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.865+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d8070070 con 0x7f95d80ff220 2023-09-23T09:38:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.865+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d8070580 con 0x7f95d80ff220 2023-09-23T09:38:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.869+0000 7f95d57fa700 1 -- 172.21.15.45:0/3455907287 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d0014440 con 0x7f95d80ff220 2023-09-23T09:38:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.869+0000 7f95d57fa700 1 -- 172.21.15.45:0/3455907287 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0012580 con 0x7f95d80ff220 2023-09-23T09:38:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.869+0000 7f95d57fa700 1 -- 172.21.15.45:0/3455907287 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f95d00127a0 con 0x7f95d80ff220 2023-09-23T09:38:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.870+0000 7f95d57fa700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95c00a18d0 0x7f95c00a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.870+0000 7f95d57fa700 1 -- 172.21.15.45:0/3455907287 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95d0006e40 con 0x7f95d80ff220 2023-09-23T09:38:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.870+0000 7f95ea84fb80 1 Processor -- start 2023-09-23T09:38:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.870+0000 7f95d77fe700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95c00a18d0 0x7f95c00a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95ea84fb80 1 -- start start 2023-09-23T09:38:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95d77fe700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95c00a18d0 0x7f95c00a3d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95c800b2e0 tx=0x7f95c8005fb0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c0708497e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849d20 0x55c070854690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070854bd0 0x55c070856fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0707b2460 con 0x55c070849d20 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0707a3c20 con 0x55c070849400 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.871+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0707a1540 con 0x55c070854bd0 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c0708497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c0708497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56844/0 (socket says 172.21.15.45:56844) 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d7fff700 1 -- 172.21.15.45:0/2531504686 learned_addr learned my addr 172.21.15.45:0/2531504686 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d77fe700 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849d20 0x55c070854690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95dcc02700 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070854bd0 0x55c070856fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d77fe700 1 -- 172.21.15.45:0/2531504686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070854bd0 msgr2=0x55c070856fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d77fe700 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070854bd0 0x55c070856fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d77fe700 1 -- 172.21.15.45:0/2531504686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 msgr2=0x55c0708497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d77fe700 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c0708497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d77fe700 1 -- 172.21.15.45:0/2531504686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0707ff8d0 con 0x55c070849d20 2023-09-23T09:38:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.872+0000 7f95d77fe700 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849d20 0x55c070854690 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098e0 tx=0x7f95c801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42652954f5977c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.873+0000 7f95be7fc700 1 -- 172.21.15.45:0/2531504686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c801d070 con 0x55c070849d20 2023-09-23T09:38:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.873+0000 7f95be7fc700 1 -- 172.21.15.45:0/2531504686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c800bdf0 con 0x55c070849d20 2023-09-23T09:38:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.873+0000 7f95be7fc700 1 -- 172.21.15.45:0/2531504686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8025b00 con 0x55c070849d20 2023-09-23T09:38:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.874+0000 7f95ea84fb80 1 -- 172.21.15.45:0/2531504686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849d20 msgr2=0x55c070854690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.874+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849d20 0x55c070854690 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098e0 tx=0x7f95c801d000 comp rx=0 tx=0).stop 2023-09-23T09:38:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.875+0000 7f95ea84fb80 1 -- 172.21.15.45:0/2531504686 shutdown_connections 2023-09-23T09:38:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.875+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070854bd0 0x55c070856fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.875+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c0708497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.875+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/2531504686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849d20 0x55c070854690 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.875+0000 7f95ea84fb80 1 -- 172.21.15.45:0/2531504686 >> 172.21.15.45:0/2531504686 conn(0x55c0707be260 msgr2=0x55c070845790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.875+0000 7f95ea84fb80 1 -- 172.21.15.45:0/2531504686 shutdown_connections 2023-09-23T09:38:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.875+0000 7f95ea84fb80 1 -- 172.21.15.45:0/2531504686 wait complete. 2023-09-23T09:38:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.876+0000 7f95ea84fb80 1 -- start start 2023-09-23T09:38:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.876+0000 7f95ea84fb80 1 Processor -- start 2023-09-23T09:38:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.876+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c070854640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070849d20 0x55c0708518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070854bd0 0x55c070851fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07079ce30 con 0x55c070854bd0 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c070739710 con 0x55c070849400 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0707393b0 con 0x55c070849d20 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95ea84fb80 10 client.0 _ll_register_callbacks cb 0x55c070596800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070849d20 0x55c0708518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070849d20 0x55c0708518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37608/0 (socket says 172.21.15.45:37608) 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 -- 172.21.15.45:0/3942785473 learned_addr learned my addr 172.21.15.45:0/3942785473 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d7fff700 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c070854640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 -- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 msgr2=0x55c070854640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c070854640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 -- 172.21.15.45:0/3942785473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070854bd0 msgr2=0x55c070851fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070854bd0 0x55c070851fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 -- 172.21.15.45:0/3942785473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d8105410 con 0x55c070849d20 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95d77fe700 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070849d20 0x55c0708518c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f95c800fdc0 tx=0x7f95c800fdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d68f6cbd6c9eac server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.877+0000 7f95b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.878+0000 7f95b77fe700 1 -- 172.21.15.45:0/3942785473 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8032070 con 0x55c070849d20 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.878+0000 7f95b77fe700 1 -- 172.21.15.45:0/3942785473 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95c8025630 con 0x55c070849d20 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.878+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c07083d250 con 0x55c070849d20 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.878+0000 7f95b77fe700 1 -- 172.21.15.45:0/3942785473 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c80257f0 con 0x55c070849d20 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.878+0000 7f95ea84fb80 10 client.26176 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.878+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c0705c7de0 con 0x55c070849d20 2023-09-23T09:38:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.879+0000 7f95b77fe700 1 -- 172.21.15.45:0/3942785473 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c07083d250 con 0x55c070849d20 2023-09-23T09:38:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95b77fe700 1 -- 172.21.15.45:0/3942785473 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f95c803f460 con 0x55c070849d20 2023-09-23T09:38:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95ea84fb80 10 client.26176 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95ea84fb80 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95ea84fb80 20 client.26176 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c070864110 0x55c070866500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95ea84fb80 4 client.26176 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:38:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c070795e10 con 0x55c070864110 2023-09-23T09:38:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.880+0000 7f95d7fff700 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c070864110 0x55c070866500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.881+0000 7f95b77fe700 10 client.26176 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.881+0000 7f95d7fff700 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c070864110 0x55c070866500 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a19c22989cd8bc8b server_cookie=93230c9af7975846 in_seq=0 out_seq=0 2023-09-23T09:38:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95b77fe700 1 -- 172.21.15.45:0/3942785473 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c070795e10 con 0x55c070864110 2023-09-23T09:38:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95b77fe700 10 client.26176 handle_command_reply: tid=0 2023-09-23T09:38:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 client.26176 shutdown 2023-09-23T09:38:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070849d20 msgr2=0x55c0708518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070849d20 0x55c0708518c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f95c800fdc0 tx=0x7f95c800fdf0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 shutdown_connections 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c070864110 0x55c070866500 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070854bd0 0x55c070851fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95b77fe700 0 client.26176 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070849400 0x55c070854640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/3942785473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070849d20 0x55c0708518c0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f95c800fdc0 tx=0x7f95c800fdf0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 >> 172.21.15.45:0/3942785473 conn(0x55c0707be260 msgr2=0x55c070845320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 shutdown_connections 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.882+0000 7f95ea84fb80 1 -- 172.21.15.45:0/3942785473 wait complete. 2023-09-23T09:38:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.883+0000 7f95ea84fb80 20 client.26176 trim_cache size 0 max 16384 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 Processor -- start 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 -- start start 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07084cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07084d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070851080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07061d920 con 0x55c070849400 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c070845d30 con 0x55c070854bd0 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c070860ff0 con 0x55c070852110 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07084cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07084cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35508/0 (socket says 172.21.15.45:35508) 2023-09-23T09:38:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95d77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07084d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95d7fff700 1 -- 172.21.15.45:0/1779836083 learned_addr learned my addr 172.21.15.45:0/1779836083 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.914+0000 7f95dcc02700 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070851080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95d7fff700 1 -- 172.21.15.45:0/1779836083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 msgr2=0x55c070851080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95d7fff700 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070851080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95d7fff700 1 -- 172.21.15.45:0/1779836083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 msgr2=0x55c07084d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95d7fff700 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07084d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95d7fff700 1 -- 172.21.15.45:0/1779836083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0705b0c60 con 0x55c070849400 2023-09-23T09:38:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95d7fff700 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07084cbf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f95d0004c20 tx=0x7f95d00126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd66cf9efc0c3ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95beffd700 1 -- 172.21.15.45:0/1779836083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d009eb70 con 0x55c070849400 2023-09-23T09:38:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95beffd700 1 -- 172.21.15.45:0/1779836083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d00a2d20 con 0x55c070849400 2023-09-23T09:38:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.915+0000 7f95beffd700 1 -- 172.21.15.45:0/1779836083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d00cb540 con 0x55c070849400 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1779836083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 msgr2=0x55c07084cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07084cbf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f95d0004c20 tx=0x7f95d00126e0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1779836083 shutdown_connections 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070851080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07084cbf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1779836083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07084d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1779836083 >> 172.21.15.45:0/1779836083 conn(0x55c0707be260 msgr2=0x55c07085fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1779836083 shutdown_connections 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.917+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1779836083 wait complete. 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 -- start start 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 Processor -- start 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07085b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070859ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07085a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07085a720 con 0x55c070849400 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07085a890 con 0x55c070854bd0 2023-09-23T09:38:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0708b7ea0 con 0x55c070852110 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.918+0000 7f95ea84fb80 10 client.0 _ll_register_callbacks cb 0x7f95c800fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07085a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95d77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070859ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07085a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56868/0 (socket says 172.21.15.45:56868) 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 -- 172.21.15.45:0/1321735532 learned_addr learned my addr 172.21.15.45:0/1321735532 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 -- 172.21.15.45:0/1321735532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 msgr2=0x55c070859ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070859ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 -- 172.21.15.45:0/1321735532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 msgr2=0x55c07085b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07085b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 -- 172.21.15.45:0/1321735532 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d8070580 con 0x55c070854bd0 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95dcc02700 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07085a050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f95cc013720 tx=0x7f95cc013a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf6b870b5b53d21 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:38:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.919+0000 7f95b77fe700 1 -- 172.21.15.45:0/1321735532 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004460 con 0x55c070854bd0 2023-09-23T09:38:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95c800bdf0 con 0x55c070854bd0 2023-09-23T09:38:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95b77fe700 1 -- 172.21.15.45:0/1321735532 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95cc003680 con 0x55c070854bd0 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95b77fe700 1 -- 172.21.15.45:0/1321735532 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95cc003840 con 0x55c070854bd0 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 10 client.26192 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c0705ee250 con 0x55c070854bd0 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95b77fe700 1 -- 172.21.15.45:0/1321735532 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c800bdf0 con 0x55c070854bd0 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95b77fe700 1 -- 172.21.15.45:0/1321735532 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f95cc00fb50 con 0x55c070854bd0 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 10 client.26192 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 10 client.26192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 20 client.26192 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c0708b96c0 0x55c0708bbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 4 client.26192 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.920+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c070795b30 con 0x55c0708b96c0 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95d7fff700 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c0708b96c0 0x55c0708bbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95b77fe700 10 client.26192 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95d7fff700 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c0708b96c0 0x55c0708bbae0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552ec5a90eabfbd3 server_cookie=988bfb0939eed544 in_seq=0 out_seq=0 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95b77fe700 1 -- 172.21.15.45:0/1321735532 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c070795b30 con 0x55c0708b96c0 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95b77fe700 10 client.26192 handle_command_reply: tid=0 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95ea84fb80 1 client.26192 shutdown 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 msgr2=0x55c07085a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.921+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07085a050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f95cc013720 tx=0x7f95cc013a30 comp rx=0 tx=0).stop 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 shutdown_connections 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55c0708b96c0 0x55c0708bbae0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c070854bd0 0x55c07085a050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95b77fe700 0 client.26192 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c070849400 0x55c07085b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 --2- 172.21.15.45:0/1321735532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c070852110 0x55c070859ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 >> 172.21.15.45:0/1321735532 conn(0x55c0707be260 msgr2=0x55c07084cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 shutdown_connections 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 1 -- 172.21.15.45:0/1321735532 wait complete. 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95ea84fb80 20 client.26192 trim_cache size 0 max 16384 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:38:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:38:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:38:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:38:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:38:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:38:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:38:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:38:32.928 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95c00a18d0 msgr2=0x7f95c00a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95de665700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95c00a18d0 0x7f95c00a3d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95c800b2e0 tx=0x7f95c8005fb0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d80ff220 msgr2=0x7f95d80723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.922+0000 7f95de665700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d80ff220 0x7f95d80723b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b990 tx=0x7f95d000bd50 comp rx=0 tx=0).stop 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 shutdown_connections 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95c00a18d0 0x7f95c00a3d90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d806b350 0x7f95d806b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95d80ff220 0x7f95d80723b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 --2- 172.21.15.45:0/3455907287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95d80ffbe0 0x7f95d80728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 >> 172.21.15.45:0/3455907287 conn(0x7f95d8077290 msgr2=0x7f95d8102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 shutdown_connections 2023-09-23T09:38:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:38:32.923+0000 7f95de665700 1 -- 172.21.15.45:0/3455907287 wait complete. 2023-09-23T09:38:32.951 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:38:34.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:33 smithi098 ceph-mon[102386]: pgmap v283: 97 pgs: 97 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 27 MiB/s wr, 1.74k op/s 2023-09-23T09:38:34.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:33 smithi045 ceph-mon[98627]: pgmap v283: 97 pgs: 97 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 27 MiB/s wr, 1.74k op/s 2023-09-23T09:38:34.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:33 smithi123 ceph-mon[103577]: pgmap v283: 97 pgs: 97 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 27 MiB/s wr, 1.74k op/s 2023-09-23T09:38:36.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:35 smithi098 ceph-mon[102386]: pgmap v284: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 39 MiB/s wr, 2.54k op/s 2023-09-23T09:38:36.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:35 smithi045 ceph-mon[98627]: pgmap v284: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 39 MiB/s wr, 2.54k op/s 2023-09-23T09:38:36.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:35 smithi123 ceph-mon[103577]: pgmap v284: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 39 MiB/s wr, 2.54k op/s 2023-09-23T09:38:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 530.0 s, 1174.2 tps, lat 84.859 ms stddev 97.739 2023-09-23T09:38:37.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:36 smithi098 ceph-mon[102386]: pgmap v285: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2023-09-23T09:38:37.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:36 smithi045 ceph-mon[98627]: pgmap v285: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2023-09-23T09:38:37.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:36 smithi123 ceph-mon[103577]: pgmap v285: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2023-09-23T09:38:39.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:39 smithi123 ceph-mon[103577]: pgmap v286: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 MiB/s wr, 1.29k op/s 2023-09-23T09:38:39.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:39 smithi098 ceph-mon[102386]: pgmap v286: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 MiB/s wr, 1.29k op/s 2023-09-23T09:38:39.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:39 smithi045 ceph-mon[98627]: pgmap v286: 97 pgs: 97 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 MiB/s wr, 1.29k op/s 2023-09-23T09:38:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/942739992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8106120 msgr2=0x7f4fc810dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/942739992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8106120 0x7f4fc810dca0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098f0 tx=0x7f4fb8009c00 comp rx=0 tx=0).stop 2023-09-23T09:38:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/942739992 shutdown_connections 2023-09-23T09:38:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/942739992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8106120 0x7f4fc810dca0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/942739992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8101000 0x7f4fc81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/942739992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fc81019b0 0x7f4fc81059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/942739992 >> 172.21.15.45:0/942739992 conn(0x7f4fc80fc880 msgr2=0x7f4fc80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.973+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/942739992 shutdown_connections 2023-09-23T09:38:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.974+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/942739992 wait complete. 2023-09-23T09:38:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 Processor -- start 2023-09-23T09:38:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 -- start start 2023-09-23T09:38:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8101000 0x7f4fc810e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8106120 0x7f4fc810e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fc810ed30 0x7f4fc81a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81a9d70 con 0x7f4fc810ed30 2023-09-23T09:38:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81a9ee0 con 0x7f4fc8106120 2023-09-23T09:38:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fcdf6a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81aa0a0 con 0x7f4fc8101000 2023-09-23T09:38:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8101000 0x7f4fc810e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fc6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8106120 0x7f4fc810e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8101000 0x7f4fc810e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37636/0 (socket says 172.21.15.45:37636) 2023-09-23T09:38:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.975+0000 7f4fc77fe700 1 -- 172.21.15.45:0/1911458893 learned_addr learned my addr 172.21.15.45:0/1911458893 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.976+0000 7f4fc7fff700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fc810ed30 0x7f4fc81a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.976+0000 7f4fc6ffd700 1 -- 172.21.15.45:0/1911458893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8101000 msgr2=0x7f4fc810e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.976+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8101000 0x7f4fc810e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.976+0000 7f4fc6ffd700 1 -- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fc810ed30 msgr2=0x7f4fc81a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.976+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fc810ed30 0x7f4fc81a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.976+0000 7f4fc6ffd700 1 -- 172.21.15.45:0/1911458893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8009520 con 0x7f4fc8106120 2023-09-23T09:38:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.976+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8106120 0x7f4fc810e7f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005fd0 tx=0x7f4fb000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.977+0000 7f4fc4ff9700 1 -- 172.21.15.45:0/1911458893 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0010900 con 0x7f4fc8106120 2023-09-23T09:38:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.977+0000 7f4fc4ff9700 1 -- 172.21.15.45:0/1911458893 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb0018420 con 0x7f4fc8106120 2023-09-23T09:38:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.977+0000 7f4fc4ff9700 1 -- 172.21.15.45:0/1911458893 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000f680 con 0x7f4fc8106120 2023-09-23T09:38:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.977+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc81aa380 con 0x7f4fc8106120 2023-09-23T09:38:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.977+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc81aa840 con 0x7f4fc8106120 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.983+0000 7f4fda154b80 1 Processor -- start 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.984+0000 7f4fda154b80 1 -- start start 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.984+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872d1f40 0x5652872d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.984+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 0x5652872ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.984+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872ee720 0x5652872f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.984+0000 7f4fda154b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528723b400 con 0x5652872ee720 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.984+0000 7f4fda154b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528722cbc0 con 0x5652872d2860 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.984+0000 7f4fda154b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528722a4e0 con 0x5652872d1f40 2023-09-23T09:38:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.985+0000 7f4fc4ff9700 1 -- 172.21.15.45:0/1911458893 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f4fb0018590 con 0x7f4fc8106120 2023-09-23T09:38:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.986+0000 7f4fc4ff9700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4fb40a18e0 0x7f4fb40a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.986+0000 7f4fc4ff9700 1 -- 172.21.15.45:0/1911458893 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4fb0014070 con 0x7f4fc8106120 2023-09-23T09:38:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.987+0000 7f4fc6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872ee720 0x5652872f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.987+0000 7f4fc6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872ee720 0x5652872f0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35536/0 (socket says 172.21.15.45:35536) 2023-09-23T09:38:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.987+0000 7f4fc6ffd700 1 -- 172.21.15.45:0/97396643 learned_addr learned my addr 172.21.15.45:0/97396643 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.991+0000 7f4fc7fff700 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872d1f40 0x5652872d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.993+0000 7f4fc6ffd700 1 -- 172.21.15.45:0/97396643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872d1f40 msgr2=0x5652872d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.993+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872d1f40 0x5652872d2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.993+0000 7f4fc6ffd700 1 -- 172.21.15.45:0/97396643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 msgr2=0x5652872ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.993+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 0x5652872ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.993+0000 7f4fc6ffd700 1 -- 172.21.15.45:0/97396643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565287288870 con 0x5652872ee720 2023-09-23T09:38:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.994+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872ee720 0x5652872f0b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00a1fb0 tx=0x7f4fb0099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6750fbd34e477636 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.994+0000 7f4fbdffb700 1 -- 172.21.15.45:0/97396643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb009e2a0 con 0x5652872ee720 2023-09-23T09:38:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.994+0000 7f4fbdffb700 1 -- 172.21.15.45:0/97396643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb00a24e0 con 0x5652872ee720 2023-09-23T09:38:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.994+0000 7f4fbdffb700 1 -- 172.21.15.45:0/97396643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb009a3b0 con 0x5652872ee720 2023-09-23T09:38:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.995+0000 7f4fda154b80 1 -- 172.21.15.45:0/97396643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872ee720 msgr2=0x5652872f0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.995+0000 7f4fda154b80 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872ee720 0x5652872f0b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00a1fb0 tx=0x7f4fb0099d90 comp rx=0 tx=0).stop 2023-09-23T09:38:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.999+0000 7f4fc7fff700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4fb40a18e0 0x7f4fb40a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:40.999+0000 7f4fda154b80 1 -- 172.21.15.45:0/97396643 shutdown_connections 2023-09-23T09:38:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.000+0000 7f4fda154b80 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872ee720 0x5652872f0b10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.000+0000 7f4fda154b80 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872d1f40 0x5652872d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.000+0000 7f4fda154b80 1 --2- 172.21.15.45:0/97396643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 0x5652872ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.000+0000 7f4fda154b80 1 -- 172.21.15.45:0/97396643 >> 172.21.15.45:0/97396643 conn(0x565287247200 msgr2=0x5652872ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.000+0000 7f4fc7fff700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4fb40a18e0 0x7f4fb40a3da0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009fd0 tx=0x7f4fb8005de0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.006+0000 7f4fda154b80 1 -- 172.21.15.45:0/97396643 shutdown_connections 2023-09-23T09:38:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.009+0000 7f4fda154b80 1 -- 172.21.15.45:0/97396643 wait complete. 2023-09-23T09:38:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.009+0000 7f4fda154b80 1 -- start start 2023-09-23T09:38:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.009+0000 7f4fda154b80 1 Processor -- start 2023-09-23T09:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 0x5652872eb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872eba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fda154b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x565287225dd0 con 0x5652872d2860 2023-09-23T09:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fda154b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652871c26b0 con 0x5652872d1f40 2023-09-23T09:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fda154b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652871c2350 con 0x5652872ee720 2023-09-23T09:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fda154b80 10 client.0 _ll_register_callbacks cb 0x56528701f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872ee130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56892/0 (socket says 172.21.15.45:56892) 2023-09-23T09:38:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.010+0000 7f4fc77fe700 1 -- 172.21.15.45:0/555503356 learned_addr learned my addr 172.21.15.45:0/555503356 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fc77fe700 1 -- 172.21.15.45:0/555503356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 msgr2=0x5652872eba50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872eba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fc77fe700 1 -- 172.21.15.45:0/555503356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 msgr2=0x5652872eb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 0x5652872eb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fc77fe700 1 -- 172.21.15.45:0/555503356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc81aa840 con 0x5652872d1f40 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872ee130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0004b10 tx=0x7f4fc000f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89ba8dae4699a72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.011+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/555503356 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0019070 con 0x5652872d1f40 2023-09-23T09:38:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.012+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/555503356 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4fc0004bb0 con 0x5652872d1f40 2023-09-23T09:38:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.012+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652872c6090 con 0x5652872d1f40 2023-09-23T09:38:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.013+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/555503356 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc0004d70 con 0x5652872d1f40 2023-09-23T09:38:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.013+0000 7f4fda154b80 10 client.26198 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.013+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x565287050d80 con 0x5652872d1f40 2023-09-23T09:38:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/555503356 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652872c6090 con 0x5652872d1f40 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/555503356 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4fc001c020 con 0x5652872d1f40 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fda154b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fda154b80 10 client.26198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fda154b80 20 client.26198 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fda154b80 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5652872fe310 0x565287300730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fda154b80 4 client.26198 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56528721edb0 con 0x5652872fe310 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.014+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5652872fe310 0x565287300730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.015+0000 7f4fa6ffd700 10 client.26198 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.015+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5652872fe310 0x565287300730 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2088682534b9ae5d server_cookie=64e36c6c0a3a4af3 in_seq=0 out_seq=0 2023-09-23T09:38:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.017+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/555503356 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56528721edb0 con 0x5652872fe310 2023-09-23T09:38:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.017+0000 7f4fa6ffd700 10 client.26198 handle_command_reply: tid=0 2023-09-23T09:38:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.020+0000 7f4fda154b80 1 client.26198 shutdown 2023-09-23T09:38:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.020+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 msgr2=0x5652872ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.020+0000 7f4fda154b80 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872ee130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0004b10 tx=0x7f4fc000f420 comp rx=0 tx=0).stop 2023-09-23T09:38:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.021+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 shutdown_connections 2023-09-23T09:38:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.021+0000 7f4fda154b80 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5652872fe310 0x565287300730 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.021+0000 7f4fda154b80 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872eba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.021+0000 7f4fda154b80 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872ee130 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.021+0000 7f4fda154b80 1 --2- 172.21.15.45:0/555503356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 0x5652872eb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.021+0000 7f4fa6ffd700 0 client.26198 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.021+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 >> 172.21.15.45:0/555503356 conn(0x565287247200 msgr2=0x5652872cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.024+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 shutdown_connections 2023-09-23T09:38:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.024+0000 7f4fda154b80 1 -- 172.21.15.45:0/555503356 wait complete. 2023-09-23T09:38:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.024+0000 7f4fda154b80 20 client.26198 trim_cache size 0 max 16384 2023-09-23T09:38:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.044+0000 7f4fda154b80 1 Processor -- start 2023-09-23T09:38:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.044+0000 7f4fda154b80 1 -- start start 2023-09-23T09:38:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d1f40 0x5652872f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 0x5652872f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fda154b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652870a68c0 con 0x5652872d1f40 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fda154b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652872f57b0 con 0x5652872d2860 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fda154b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652872cd170 con 0x5652872ee720 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d1f40 0x5652872f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 0x5652872f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d1f40 0x5652872f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35550/0 (socket says 172.21.15.45:35550) 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 -- 172.21.15.45:0/1101177301 learned_addr learned my addr 172.21.15.45:0/1101177301 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 -- 172.21.15.45:0/1101177301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 msgr2=0x5652872e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/1101177301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 -- 172.21.15.45:0/1101177301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 msgr2=0x5652872f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/1101177301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 0x5652872f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 -- 172.21.15.45:0/1101177301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565287039c00 con 0x5652872d1f40 2023-09-23T09:38:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/1101177301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d1f40 0x5652872f5270 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0015db0 tx=0x7f4fc00079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458ee8a33c5bf610 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fbe7fc700 1 -- 172.21.15.45:0/1101177301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0019040 con 0x5652872d1f40 2023-09-23T09:38:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.045+0000 7f4fbe7fc700 1 -- 172.21.15.45:0/1101177301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc0010040 con 0x5652872d1f40 2023-09-23T09:38:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fbe7fc700 1 -- 172.21.15.45:0/1101177301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00158c0 con 0x5652872d1f40 2023-09-23T09:38:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 -- 172.21.15.45:0/1101177301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d1f40 msgr2=0x5652872f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 --2- 172.21.15.45:0/1101177301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d1f40 0x5652872f5270 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0015db0 tx=0x7f4fc00079d0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 -- 172.21.15.45:0/1101177301 shutdown_connections 2023-09-23T09:38:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 --2- 172.21.15.45:0/1101177301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 --2- 172.21.15.45:0/1101177301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d1f40 0x5652872f5270 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 --2- 172.21.15.45:0/1101177301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d2860 0x5652872f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 -- 172.21.15.45:0/1101177301 >> 172.21.15.45:0/1101177301 conn(0x565287247200 msgr2=0x5652872d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 -- 172.21.15.45:0/1101177301 shutdown_connections 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.046+0000 7f4fda154b80 1 -- 172.21.15.45:0/1101177301 wait complete. 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 -- start start 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 Processor -- start 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 0x5652872f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652872f4390 con 0x5652872d2860 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652872f44d0 con 0x5652872d1f40 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565287352170 con 0x5652872ee720 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fda154b80 10 client.0 _ll_register_callbacks cb 0x7f4fc000a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fc6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 0x5652872f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.047+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56906/0 (socket says 172.21.15.45:56906) 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 -- 172.21.15.45:0/3731226164 learned_addr learned my addr 172.21.15.45:0/3731226164 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 -- 172.21.15.45:0/3731226164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 msgr2=0x5652872f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 -- 172.21.15.45:0/3731226164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 msgr2=0x5652872f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 0x5652872f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 -- 172.21.15.45:0/3731226164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565287050d80 con 0x5652872d1f40 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc7fff700 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fc77fe700 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872f5180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0015ef0 tx=0x7f4fc00156b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ca1c1b3ad04281 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/3731226164 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0019040 con 0x5652872d1f40 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fb00a24e0 con 0x5652872d1f40 2023-09-23T09:38:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.048+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/3731226164 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4fc0004c70 con 0x5652872d1f40 2023-09-23T09:38:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.050+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/3731226164 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc0007690 con 0x5652872d1f40 2023-09-23T09:38:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.050+0000 7f4fda154b80 10 client.26204 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.050+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5652870771f0 con 0x5652872d1f40 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/3731226164 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb00a24e0 con 0x5652872d1f40 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/3731226164 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4fc000f140 con 0x5652872d1f40 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fda154b80 10 client.26204 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fda154b80 10 client.26204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fda154b80 20 client.26204 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fda154b80 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565287353610 0x565287355a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fda154b80 4 client.26204 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56528721ead0 con 0x565287353610 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.051+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565287353610 0x565287355a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.052+0000 7f4fa6ffd700 10 client.26204 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.052+0000 7f4fc6ffd700 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565287353610 0x565287355a30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4925ca257b95902 server_cookie=90d9fe5aec2e776e in_seq=0 out_seq=0 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fa6ffd700 1 -- 172.21.15.45:0/3731226164 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56528721ead0 con 0x565287353610 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fa6ffd700 10 client.26204 handle_command_reply: tid=0 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 client.26204 shutdown 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 msgr2=0x5652872f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872f5180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0015ef0 tx=0x7f4fc00156b0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 shutdown_connections 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x565287353610 0x565287355a30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652872ee720 0x5652872f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fa6ffd700 0 client.26204 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652872d1f40 0x5652872f5180 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 --2- 172.21.15.45:0/3731226164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652872d2860 0x5652872f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 >> 172.21.15.45:0/3731226164 conn(0x565287247200 msgr2=0x5652872f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 shutdown_connections 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 1 -- 172.21.15.45:0/3731226164 wait complete. 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.075+0000 7f4fda154b80 20 client.26204 trim_cache size 0 max 16384 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:38:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:38:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:38:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:38:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.077+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4fb40a18e0 msgr2=0x7f4fb40a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.077+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4fb40a18e0 0x7f4fb40a3da0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009fd0 tx=0x7f4fb8005de0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.077+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8106120 msgr2=0x7f4fc810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.077+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8106120 0x7f4fc810e7f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005fd0 tx=0x7f4fb000eca0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 shutdown_connections 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4fb40a18e0 0x7f4fb40a3da0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fc810ed30 0x7f4fc81a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8101000 0x7f4fc810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 --2- 172.21.15.45:0/1911458893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8106120 0x7f4fc810e7f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 >> 172.21.15.45:0/1911458893 conn(0x7f4fc80fc880 msgr2=0x7f4fc80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 shutdown_connections 2023-09-23T09:38:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:41.078+0000 7f4fcdf6a700 1 -- 172.21.15.45:0/1911458893 wait complete. 2023-09-23T09:38:41.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:41 smithi045 ceph-mon[98627]: pgmap v287: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2023-09-23T09:38:41.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:41 smithi123 ceph-mon[103577]: pgmap v287: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2023-09-23T09:38:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 535.0 s, 1465.6 tps, lat 68.396 ms stddev 107.775 2023-09-23T09:38:42.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:41 smithi098 ceph-mon[102386]: pgmap v287: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2023-09-23T09:38:43.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:43 smithi045 ceph-mon[98627]: pgmap v288: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-09-23T09:38:43.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:43 smithi123 ceph-mon[103577]: pgmap v288: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-09-23T09:38:44.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:43 smithi098 ceph-mon[102386]: pgmap v288: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-09-23T09:38:45.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:45 smithi045 ceph-mon[98627]: pgmap v289: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 28 MiB/s wr, 1.98k op/s 2023-09-23T09:38:45.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:45 smithi123 ceph-mon[103577]: pgmap v289: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 28 MiB/s wr, 1.98k op/s 2023-09-23T09:38:46.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:45 smithi098 ceph-mon[102386]: pgmap v289: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 28 MiB/s wr, 1.98k op/s 2023-09-23T09:38:46.888 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 540.0 s, 1438.2 tps, lat 68.929 ms stddev 69.592 2023-09-23T09:38:47.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:47 smithi045 ceph-mon[98627]: pgmap v290: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-09-23T09:38:47.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:47 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:38:47.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:47 smithi123 ceph-mon[103577]: pgmap v290: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-09-23T09:38:47.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:47 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:38:48.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:47 smithi098 ceph-mon[102386]: pgmap v290: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-09-23T09:38:48.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:47 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:38:48.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:48 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:38:48.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:48 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:38:48.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:48 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:38:48.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:48 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:38:48.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:48 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:38:48.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:48 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:38:49.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:48 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:38:49.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:48 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:38:49.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:48 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:38:50.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:49 smithi098 ceph-mon[102386]: pgmap v291: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:38:50.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:49 smithi045 ceph-mon[98627]: pgmap v291: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:38:50.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:49 smithi123 ceph-mon[103577]: pgmap v291: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:38:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 545.0 s, 1385.6 tps, lat 72.827 ms stddev 80.026 2023-09-23T09:38:52.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:51 smithi098 ceph-mon[102386]: pgmap v292: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-09-23T09:38:52.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:51 smithi045 ceph-mon[98627]: pgmap v292: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-09-23T09:38:52.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:51 smithi123 ceph-mon[103577]: pgmap v292: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-09-23T09:38:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.250+0000 7f37bd184700 1 -- 172.21.15.45:0/701406038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8100160 msgr2=0x7f37b8100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.250+0000 7f37bd184700 1 --2- 172.21.15.45:0/701406038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8100160 0x7f37b8100540 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f37a80099c0 tx=0x7f37a8009cd0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.250+0000 7f37bd184700 1 -- 172.21.15.45:0/701406038 shutdown_connections 2023-09-23T09:38:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.250+0000 7f37bd184700 1 --2- 172.21.15.45:0/701406038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b8105050 0x7f37b810cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.250+0000 7f37bd184700 1 --2- 172.21.15.45:0/701406038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8100160 0x7f37b8100540 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.250+0000 7f37bd184700 1 --2- 172.21.15.45:0/701406038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100b10 0x7f37b8104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.251+0000 7f37bd184700 1 -- 172.21.15.45:0/701406038 >> 172.21.15.45:0/701406038 conn(0x7f37b8076840 msgr2=0x7f37b8076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.253+0000 7f37bd184700 1 -- 172.21.15.45:0/701406038 shutdown_connections 2023-09-23T09:38:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.253+0000 7f37bd184700 1 -- 172.21.15.45:0/701406038 wait complete. 2023-09-23T09:38:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 Processor -- start 2023-09-23T09:38:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 -- start start 2023-09-23T09:38:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100160 0x7f37b81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8105050 0x7f37b81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b819dd50 0x7f37b819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b819e7d0 con 0x7f37b8105050 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b819e940 con 0x7f37b819dd50 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37bd184700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b819eb00 con 0x7f37b8100160 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37b659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8105050 0x7f37b81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100160 0x7f37b81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37b759e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b819dd50 0x7f37b819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100160 0x7f37b81991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37676/0 (socket says 172.21.15.45:37676) 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37b659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8105050 0x7f37b81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35572/0 (socket says 172.21.15.45:35572) 2023-09-23T09:38:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37b759e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b819dd50 0x7f37b819e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56924/0 (socket says 172.21.15.45:56924) 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.254+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3911257228 learned_addr learned my addr 172.21.15.45:0/3911257228 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3911257228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b819dd50 msgr2=0x7f37b819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b819dd50 0x7f37b819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8105050 msgr2=0x7f37b81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8105050 0x7f37b81996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3911257228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80095f0 con 0x7f37b8100160 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100160 0x7f37b81991a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37a80098e0 tx=0x7f37a800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f379ffff700 1 -- 172.21.15.45:0/3911257228 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a801d070 con 0x7f37b8100160 2023-09-23T09:38:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b81a6000 con 0x7f37b8100160 2023-09-23T09:38:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f379ffff700 1 -- 172.21.15.45:0/3911257228 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a8022470 con 0x7f37b8100160 2023-09-23T09:38:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f379ffff700 1 -- 172.21.15.45:0/3911257228 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8017650 con 0x7f37b8100160 2023-09-23T09:38:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.255+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b81a6430 con 0x7f37b8100160 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.258+0000 7f379ffff700 1 -- 172.21.15.45:0/3911257228 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f37a8022850 con 0x7f37b8100160 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.259+0000 7f379ffff700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f37a00a1710 0x7f37a00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.259+0000 7f379ffff700 1 -- 172.21.15.45:0/3911257228 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f37a80cdec0 con 0x7f37b8100160 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.259+0000 7f37b659c700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f37a00a1710 0x7f37a00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.259+0000 7f37b659c700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f37a00a1710 0x7f37a00a3bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37b819a9c0 tx=0x7f37ac009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 Processor -- start 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 -- start start 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbeb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbeb860 0x5643acbf61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbf6710 0x5643acbf8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acb54400 con 0x5643acbeb860 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acb45bc0 con 0x5643acbf6710 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37c936eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acb434e0 con 0x5643acbeaf40 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbeb860 0x5643acbf61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbeb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbeb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37684/0 (socket says 172.21.15.45:37684) 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbeb860 0x5643acbf61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35580/0 (socket says 172.21.15.45:35580) 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b759e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbf6710 0x5643acbf8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3215914272 learned_addr learned my addr 172.21.15.45:0/3215914272 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3215914272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbf6710 msgr2=0x5643acbf8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbf6710 0x5643acbf8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3215914272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbeb860 msgr2=0x5643acbf61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbeb860 0x5643acbf61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.260+0000 7f37b6d9d700 1 -- 172.21.15.45:0/3215914272 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643acba1870 con 0x5643acbeaf40 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.261+0000 7f37b659c700 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbeb860 0x5643acbf61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.261+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbeb320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f37a8097490 tx=0x7f37a809aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62cf183e09466985 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.261+0000 7f379d7fa700 1 -- 172.21.15.45:0/3215914272 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a809af90 con 0x5643acbeaf40 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.261+0000 7f379d7fa700 1 -- 172.21.15.45:0/3215914272 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a809f0e0 con 0x5643acbeaf40 2023-09-23T09:38:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.261+0000 7f379d7fa700 1 -- 172.21.15.45:0/3215914272 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a809b450 con 0x5643acbeaf40 2023-09-23T09:38:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 -- 172.21.15.45:0/3215914272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 msgr2=0x5643acbeb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbeb320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f37a8097490 tx=0x7f37a809aba0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 -- 172.21.15.45:0/3215914272 shutdown_connections 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbf6710 0x5643acbf8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbeb320 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 --2- 172.21.15.45:0/3215914272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbeb860 0x5643acbf61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 -- 172.21.15.45:0/3215914272 >> 172.21.15.45:0/3215914272 conn(0x5643acb60200 msgr2=0x5643acbe73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 -- 172.21.15.45:0/3215914272 shutdown_connections 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.262+0000 7f37c936eb80 1 -- 172.21.15.45:0/3215914272 wait complete. 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.263+0000 7f37c936eb80 1 -- start start 2023-09-23T09:38:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.263+0000 7f37c936eb80 1 Processor -- start 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.263+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.263+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acbff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.263+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc0a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37c936eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acb3edd0 con 0x5643acbf6710 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37c936eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acadb6b0 con 0x5643acbeb860 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37c936eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acadb350 con 0x5643acbeaf40 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37c936eb80 10 client.0 _ll_register_callbacks cb 0x5643ac9387a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acbff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acbff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56934/0 (socket says 172.21.15.45:56934) 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b659c700 1 -- 172.21.15.45:0/718348709 learned_addr learned my addr 172.21.15.45:0/718348709 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b759e700 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc0a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b6d9d700 1 -- 172.21.15.45:0/718348709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 msgr2=0x5643acbff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acbff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b6d9d700 1 -- 172.21.15.45:0/718348709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 msgr2=0x5643acc0a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc0a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b6d9d700 1 -- 172.21.15.45:0/718348709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac017070 con 0x5643acbeaf40 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbff300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37a8097490 tx=0x7f37a8022b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c197774477d39cb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.264+0000 7f379aff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f379aff5700 1 -- 172.21.15.45:0/718348709 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a809af90 con 0x5643acbeaf40 2023-09-23T09:38:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f379aff5700 1 -- 172.21.15.45:0/718348709 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f37a809b1f0 con 0x5643acbeaf40 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643acbdf090 con 0x5643acbeaf40 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f379aff5700 1 -- 172.21.15.45:0/718348709 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a809b430 con 0x5643acbeaf40 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 10 client.26209 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f37b81a6430 con 0x5643acbeaf40 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f379aff5700 1 -- 172.21.15.45:0/718348709 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643acbdf090 con 0x5643acbeaf40 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f379aff5700 1 -- 172.21.15.45:0/718348709 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f37a80cbd20 con 0x5643acbeaf40 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 10 client.26209 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 20 client.26209 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acbfa3f0 0x5643acbfc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 4 client.26209 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.265+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643acb37db0 con 0x5643acbfa3f0 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.266+0000 7f37b659c700 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acbfa3f0 0x5643acbfc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.266+0000 7f379aff5700 10 client.26209 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.266+0000 7f37b659c700 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acbfa3f0 0x5643acbfc810 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1034302fe3bed026 server_cookie=b2b0f621351ccd05 in_seq=0 out_seq=0 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f379aff5700 1 -- 172.21.15.45:0/718348709 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5643acb37db0 con 0x5643acbfa3f0 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f379aff5700 10 client.26209 handle_command_reply: tid=0 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 client.26209 shutdown 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 msgr2=0x5643acbff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbff300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37a8097490 tx=0x7f37a8022b00 comp rx=0 tx=0).stop 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 shutdown_connections 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acbfa3f0 0x5643acbfc810 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc0a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f379aff5700 0 client.26209 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acbff300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 --2- 172.21.15.45:0/718348709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acbff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 >> 172.21.15.45:0/718348709 conn(0x5643acb60200 msgr2=0x5643acbe6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 shutdown_connections 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 1 -- 172.21.15.45:0/718348709 wait complete. 2023-09-23T09:38:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.267+0000 7f37c936eb80 20 client.26209 trim_cache size 0 max 16384 2023-09-23T09:38:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 Processor -- start 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 -- start start 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeaf40 0x5643acbff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeb860 0x5643acbeff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ac9bf8c0 con 0x5643acbf6710 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acbf5810 con 0x5643acbeaf40 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37c936eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acbf5d80 con 0x5643acbeb860 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeb860 0x5643acbeff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeb860 0x5643acbeff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37700/0 (socket says 172.21.15.45:37700) 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc09a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35596/0 (socket says 172.21.15.45:35596) 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 -- 172.21.15.45:0/2719904392 learned_addr learned my addr 172.21.15.45:0/2719904392 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeaf40 0x5643acbff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 -- 172.21.15.45:0/2719904392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeaf40 msgr2=0x5643acbff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeaf40 0x5643acbff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 -- 172.21.15.45:0/2719904392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 msgr2=0x5643acc09a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc09a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b659c700 1 -- 172.21.15.45:0/2719904392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643acbf61d0 con 0x5643acbeb860 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.288+0000 7f37b759e700 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc09a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:38:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.289+0000 7f37b659c700 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeb860 0x5643acbeff60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0097c0 tx=0x7f37ac014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7e3b8cb4f97e5c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.289+0000 7f379dffb700 1 -- 172.21.15.45:0/2719904392 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac022070 con 0x5643acbeb860 2023-09-23T09:38:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.289+0000 7f379dffb700 1 -- 172.21.15.45:0/2719904392 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37ac023040 con 0x5643acbeb860 2023-09-23T09:38:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.289+0000 7f379dffb700 1 -- 172.21.15.45:0/2719904392 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac029e20 con 0x5643acbeb860 2023-09-23T09:38:54.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:53 smithi098 ceph-mon[102386]: pgmap v293: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 808 op/s 2023-09-23T09:38:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.289+0000 7f37c936eb80 1 -- 172.21.15.45:0/2719904392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeb860 msgr2=0x5643acbeff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.289+0000 7f37c936eb80 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeb860 0x5643acbeff60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0097c0 tx=0x7f37ac014000 comp rx=0 tx=0).stop 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 -- 172.21.15.45:0/2719904392 shutdown_connections 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc09a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeaf40 0x5643acbff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 --2- 172.21.15.45:0/2719904392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeb860 0x5643acbeff60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 -- 172.21.15.45:0/2719904392 >> 172.21.15.45:0/2719904392 conn(0x5643acb60200 msgr2=0x5643acbff120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 -- 172.21.15.45:0/2719904392 shutdown_connections 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 -- 172.21.15.45:0/2719904392 wait complete. 2023-09-23T09:38:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 -- start start 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.290+0000 7f37c936eb80 1 Processor -- start 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acc024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acc02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37c936eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37c936eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acc63f10 con 0x5643acbf6710 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37c936eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acc64080 con 0x5643acbeb860 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37c936eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643acc64240 con 0x5643acbeaf40 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37c936eb80 10 client.0 _ll_register_callbacks cb 0x7f37ac0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acc02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acc024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acc02a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56950/0 (socket says 172.21.15.45:56950) 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b659c700 1 -- 172.21.15.45:0/263638245 learned_addr learned my addr 172.21.15.45:0/263638245 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b759e700 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b759e700 1 -- 172.21.15.45:0/263638245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 msgr2=0x5643acc024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b759e700 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acc024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b759e700 1 -- 172.21.15.45:0/263638245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 msgr2=0x5643acc02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b759e700 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acc02a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.291+0000 7f37b759e700 1 -- 172.21.15.45:0/263638245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b00091a0 con 0x5643acbf6710 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f37b759e700 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc639d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5643acbf5b90 tx=0x7f37b0004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d7898071ab29bc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:38:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f379aff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f379aff5700 1 -- 172.21.15.45:0/263638245 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b001c070 con 0x5643acbf6710 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f379aff5700 1 -- 172.21.15.45:0/263638245 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f37b0004bb0 con 0x5643acbf6710 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37a809f0e0 con 0x5643acbf6710 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f379aff5700 1 -- 172.21.15.45:0/263638245 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b000e040 con 0x5643acbf6710 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f37c936eb80 10 client.16536 fetch_fsmap learned FSMap version 43 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f37ac004040 con 0x5643acbf6710 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.292+0000 7f379aff5700 1 -- 172.21.15.45:0/263638245 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a809f0e0 con 0x5643acbf6710 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f379aff5700 1 -- 172.21.15.45:0/263638245 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f37b0026070 con 0x5643acbf6710 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37c936eb80 10 client.16536 fetch_fsmap finished waiting for FSMap version 43 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37c936eb80 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:38:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37c936eb80 20 client.16536 populate_metadata read hostname 'smithi045' 2023-09-23T09:38:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37c936eb80 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acc6bb00 0x5643acc6df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:38:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37c936eb80 4 client.16536 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:38:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5643acb37ad0 con 0x5643acc6bb00 2023-09-23T09:38:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acc6bb00 0x5643acc6df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:38:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f379aff5700 10 client.16536 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.293+0000 7f37b6d9d700 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acc6bb00 0x5643acc6df20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44653cd793a1e31 server_cookie=4e19dc784f4cbb0b in_seq=0 out_seq=0 2023-09-23T09:38:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f379aff5700 1 -- 172.21.15.45:0/263638245 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5643acb37ad0 con 0x5643acc6bb00 2023-09-23T09:38:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f379aff5700 10 client.16536 handle_command_reply: tid=0 2023-09-23T09:38:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 client.16536 shutdown 2023-09-23T09:38:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 msgr2=0x5643acc639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc639d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5643acbf5b90 tx=0x7f37b0004910 comp rx=0 tx=0).stop 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 shutdown_connections 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5643acc6bb00 0x5643acc6df20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643acbf6710 0x5643acc639d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643acbeaf40 0x5643acc024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 --2- 172.21.15.45:0/263638245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643acbeb860 0x5643acc02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 >> 172.21.15.45:0/263638245 conn(0x5643acb60200 msgr2=0x5643acbf4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f379aff5700 0 client.16536 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 shutdown_connections 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.311+0000 7f37c936eb80 1 -- 172.21.15.45:0/263638245 wait complete. 2023-09-23T09:38:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37c936eb80 20 client.16536 trim_cache size 0 max 16384 2023-09-23T09:38:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:38:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:38:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:38:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:38:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f37a00a1710 msgr2=0x7f37a00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f37a00a1710 0x7f37a00a3bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37b819a9c0 tx=0x7f37ac009260 comp rx=0 tx=0).stop 2023-09-23T09:38:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100160 msgr2=0x7f37b81991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:38:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100160 0x7f37b81991a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37a80098e0 tx=0x7f37a800f690 comp rx=0 tx=0).stop 2023-09-23T09:38:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 shutdown_connections 2023-09-23T09:38:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b819dd50 0x7f37b819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37b8100160 0x7f37b81991a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f37a00a1710 0x7f37a00a3bd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 --2- 172.21.15.45:0/3911257228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b8105050 0x7f37b81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:38:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 >> 172.21.15.45:0/3911257228 conn(0x7f37b8076840 msgr2=0x7f37b80fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:38:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 shutdown_connections 2023-09-23T09:38:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:38:54.312+0000 7f37bd184700 1 -- 172.21.15.45:0/3911257228 wait complete. 2023-09-23T09:38:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:53 smithi045 ceph-mon[98627]: pgmap v293: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 808 op/s 2023-09-23T09:38:54.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:53 smithi123 ceph-mon[103577]: pgmap v293: 97 pgs: 97 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 808 op/s 2023-09-23T09:38:56.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:55 smithi098 ceph-mon[102386]: pgmap v294: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-09-23T09:38:56.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:55 smithi045 ceph-mon[98627]: pgmap v294: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-09-23T09:38:56.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:55 smithi123 ceph-mon[103577]: pgmap v294: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-09-23T09:38:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 550.0 s, 1571.6 tps, lat 63.778 ms stddev 65.710 2023-09-23T09:38:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:57 smithi098 ceph-mon[102386]: pgmap v295: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 534 op/s 2023-09-23T09:38:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:57 smithi045 ceph-mon[98627]: pgmap v295: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 534 op/s 2023-09-23T09:38:58.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:57 smithi123 ceph-mon[103577]: pgmap v295: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 534 op/s 2023-09-23T09:38:59.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:58 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:38:59.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:58 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:38:59.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:58 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:39:00.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:38:59 smithi098 ceph-mon[102386]: pgmap v296: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 587 op/s 2023-09-23T09:39:00.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:38:59 smithi045 ceph-mon[98627]: pgmap v296: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 587 op/s 2023-09-23T09:39:00.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:38:59 smithi123 ceph-mon[103577]: pgmap v296: 97 pgs: 97 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 587 op/s 2023-09-23T09:39:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 555.0 s, 1391.2 tps, lat 71.130 ms stddev 124.186 2023-09-23T09:39:02.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:01 smithi098 ceph-mon[102386]: pgmap v297: 97 pgs: 97 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 14 MiB/s wr, 839 op/s 2023-09-23T09:39:02.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:01 smithi045 ceph-mon[98627]: pgmap v297: 97 pgs: 97 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 14 MiB/s wr, 839 op/s 2023-09-23T09:39:02.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:01 smithi123 ceph-mon[103577]: pgmap v297: 97 pgs: 97 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 14 MiB/s wr, 839 op/s 2023-09-23T09:39:02.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:39:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.123+0000 7f95eb710700 1 -- 172.21.15.45:0/2970140966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e4100880 msgr2=0x7f95e41048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.123+0000 7f95eb710700 1 --2- 172.21.15.45:0/2970140966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e4100880 0x7f95e41048c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009990 tx=0x7f95d4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.123+0000 7f95eb710700 1 -- 172.21.15.45:0/2970140966 shutdown_connections 2023-09-23T09:39:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.123+0000 7f95eb710700 1 --2- 172.21.15.45:0/2970140966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95e4104ff0 0x7f95e410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.123+0000 7f95eb710700 1 --2- 172.21.15.45:0/2970140966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e40ffed0 0x7f95e41002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.123+0000 7f95eb710700 1 --2- 172.21.15.45:0/2970140966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e4100880 0x7f95e41048c0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009990 tx=0x7f95d4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.123+0000 7f95eb710700 1 -- 172.21.15.45:0/2970140966 >> 172.21.15.45:0/2970140966 conn(0x7f95e40fb770 msgr2=0x7f95e40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.125+0000 7f95eb710700 1 -- 172.21.15.45:0/2970140966 shutdown_connections 2023-09-23T09:39:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.127+0000 7f95eb710700 1 -- 172.21.15.45:0/2970140966 wait complete. 2023-09-23T09:39:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.127+0000 7f95eb710700 1 Processor -- start 2023-09-23T09:39:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95eb710700 1 -- start start 2023-09-23T09:39:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95eb710700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40ffed0 0x7f95e4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95eb710700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95e4104ff0 0x7f95e41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95eb710700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e4198a90 0x7f95e41a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95eb710700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e41a37a0 con 0x7f95e4198a90 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95eb710700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e41a3910 con 0x7f95e4104ff0 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95eb710700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e41a3ad0 con 0x7f95e40ffed0 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e4198a90 0x7f95e41a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95e94ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40ffed0 0x7f95e4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e4198a90 0x7f95e41a2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35614/0 (socket says 172.21.15.45:35614) 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.128+0000 7f95e9cad700 1 -- 172.21.15.45:0/670618919 learned_addr learned my addr 172.21.15.45:0/670618919 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.129+0000 7f95e94ac700 1 -- 172.21.15.45:0/670618919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95e4104ff0 msgr2=0x7f95e41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.129+0000 7f95e94ac700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95e4104ff0 0x7f95e41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.129+0000 7f95e94ac700 1 -- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e4198a90 msgr2=0x7f95e41a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.129+0000 7f95e94ac700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e4198a90 0x7f95e41a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.129+0000 7f95e94ac700 1 -- 172.21.15.45:0/670618919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40095f0 con 0x7f95e40ffed0 2023-09-23T09:39:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.129+0000 7f95e9cad700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e4198a90 0x7f95e41a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.129+0000 7f95e94ac700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40ffed0 0x7f95e4197f70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f95e000e9f0 tx=0x7f95e000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.134+0000 7f95da7fc700 1 -- 172.21.15.45:0/670618919 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e000eee0 con 0x7f95e40ffed0 2023-09-23T09:39:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.134+0000 7f95da7fc700 1 -- 172.21.15.45:0/670618919 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95e000cbd0 con 0x7f95e40ffed0 2023-09-23T09:39:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.134+0000 7f95da7fc700 1 -- 172.21.15.45:0/670618919 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0018680 con 0x7f95e40ffed0 2023-09-23T09:39:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.135+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e4195dc0 con 0x7f95e40ffed0 2023-09-23T09:39:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.135+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e4196250 con 0x7f95e40ffed0 2023-09-23T09:39:03.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.146+0000 7f95f78fab80 1 Processor -- start 2023-09-23T09:39:03.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.147+0000 7f95f78fab80 1 -- start start 2023-09-23T09:39:03.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.147+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2daff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.147+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2d80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.147+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.147+0000 7f95f78fab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d23b460 con 0x561c8d2dac10 2023-09-23T09:39:03.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.147+0000 7f95f78fab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d22cc20 con 0x561c8d2eee50 2023-09-23T09:39:03.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.147+0000 7f95f78fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d22a540 con 0x561c8d2db530 2023-09-23T09:39:03.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.149+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.149+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2daff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35620/0 (socket says 172.21.15.45:35620) 2023-09-23T09:39:03.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.149+0000 7f95e9cad700 1 -- 172.21.15.45:0/1410224203 learned_addr learned my addr 172.21.15.45:0/1410224203 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.149+0000 7f95e94ac700 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2d80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.149+0000 7f95da7fc700 1 -- 172.21.15.45:0/670618919 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f95e000cd40 con 0x7f95e40ffed0 2023-09-23T09:39:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.150+0000 7f95da7fc700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95d00a18d0 0x7f95d00a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.150+0000 7f95da7fc700 1 -- 172.21.15.45:0/670618919 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95e0014070 con 0x7f95e40ffed0 2023-09-23T09:39:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.156+0000 7f95e94ac700 1 -- 172.21.15.45:0/1410224203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 msgr2=0x561c8d2f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.156+0000 7f95e8cab700 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.156+0000 7f95e94ac700 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.156+0000 7f95e94ac700 1 -- 172.21.15.45:0/1410224203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 msgr2=0x561c8d2daff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.156+0000 7f95e94ac700 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2daff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.156+0000 7f95e94ac700 1 -- 172.21.15.45:0/1410224203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c8d2888d0 con 0x561c8d2db530 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.156+0000 7f95e94ac700 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2d80e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f95e00cae30 tx=0x7f95e009a4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d07ae887293c3b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.159+0000 7f95cf7fe700 1 -- 172.21.15.45:0/1410224203 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e009a650 con 0x561c8d2db530 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.159+0000 7f95cf7fe700 1 -- 172.21.15.45:0/1410224203 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95e009e7a0 con 0x561c8d2db530 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.159+0000 7f95cf7fe700 1 -- 172.21.15.45:0/1410224203 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e009ab10 con 0x561c8d2db530 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.160+0000 7f95f78fab80 1 -- 172.21.15.45:0/1410224203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 msgr2=0x561c8d2d80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.160+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2d80e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f95e00cae30 tx=0x7f95e009a4f0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.164+0000 7f95f78fab80 1 -- 172.21.15.45:0/1410224203 shutdown_connections 2023-09-23T09:39:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.164+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.164+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2daff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.164+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1410224203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2d80e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.164+0000 7f95f78fab80 1 -- 172.21.15.45:0/1410224203 >> 172.21.15.45:0/1410224203 conn(0x561c8d247260 msgr2=0x561c8d2ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.164+0000 7f95e9cad700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95d00a18d0 0x7f95d00a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.175+0000 7f95f78fab80 1 -- 172.21.15.45:0/1410224203 shutdown_connections 2023-09-23T09:39:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.175+0000 7f95e9cad700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95d00a18d0 0x7f95d00a3d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0076a0 tx=0x7f95dc009040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.175+0000 7f95f78fab80 1 -- 172.21.15.45:0/1410224203 wait complete. 2023-09-23T09:39:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.176+0000 7f95f78fab80 1 -- start start 2023-09-23T09:39:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.176+0000 7f95f78fab80 1 Processor -- start 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2dac10 0x561c8d2ef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2db530 0x561c8d2ef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95f78fab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d225e30 con 0x561c8d2db530 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95f78fab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d1c2710 con 0x561c8d2eee50 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95f78fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d1c23b0 con 0x561c8d2dac10 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95f78fab80 10 client.0 _ll_register_callbacks cb 0x561c8d01f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95e94ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2dac10 0x561c8d2ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56978/0 (socket says 172.21.15.45:56978) 2023-09-23T09:39:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95e9cad700 1 -- 172.21.15.45:0/1019197990 learned_addr learned my addr 172.21.15.45:0/1019197990 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.177+0000 7f95e94ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2dac10 0x561c8d2ef230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37736/0 (socket says 172.21.15.45:37736) 2023-09-23T09:39:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.178+0000 7f95e94ac700 1 -- 172.21.15.45:0/1019197990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 msgr2=0x561c8d2fa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.178+0000 7f95e94ac700 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2fa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.178+0000 7f95e94ac700 1 -- 172.21.15.45:0/1019197990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2db530 msgr2=0x561c8d2ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.178+0000 7f95e94ac700 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2db530 0x561c8d2ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.178+0000 7f95e94ac700 1 -- 172.21.15.45:0/1019197990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e4196250 con 0x561c8d2dac10 2023-09-23T09:39:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.186+0000 7f95e94ac700 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2dac10 0x561c8d2ef230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x561c8d2dbab0 tx=0x7f95e0099d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d10540fc3c5c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.186+0000 7f95ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:39:03.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.194+0000 7f95ccff9700 1 -- 172.21.15.45:0/1019197990 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00a29e0 con 0x561c8d2dac10 2023-09-23T09:39:03.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.194+0000 7f95ccff9700 1 -- 172.21.15.45:0/1019197990 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95e009a650 con 0x561c8d2dac10 2023-09-23T09:39:03.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.194+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c8d2c6250 con 0x561c8d2dac10 2023-09-23T09:39:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.194+0000 7f95ccff9700 1 -- 172.21.15.45:0/1019197990 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95e009aac0 con 0x561c8d2dac10 2023-09-23T09:39:03.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.199+0000 7f95f78fab80 10 client.26227 fetch_fsmap learned FSMap version 44 2023-09-23T09:39:03.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.199+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x561c8d050de0 con 0x561c8d2dac10 2023-09-23T09:39:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.202+0000 7f95ccff9700 1 -- 172.21.15.45:0/1019197990 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c8d2c6250 con 0x561c8d2dac10 2023-09-23T09:39:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.203+0000 7f95ccff9700 1 -- 172.21.15.45:0/1019197990 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f95e00995b0 con 0x561c8d2dac10 2023-09-23T09:39:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.203+0000 7f95f78fab80 10 client.26227 fetch_fsmap finished waiting for FSMap version 44 2023-09-23T09:39:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.203+0000 7f95f78fab80 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.203+0000 7f95f78fab80 20 client.26227 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.203+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d2fe260 0x561c8d300650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.203+0000 7f95f78fab80 4 client.26227 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:39:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.203+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c8d21ee10 con 0x561c8d2fe260 2023-09-23T09:39:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.204+0000 7f95e8cab700 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d2fe260 0x561c8d300650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.204+0000 7f95e8cab700 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d2fe260 0x561c8d300650 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa4e0500c720c82d server_cookie=9aa3299e31788d7b in_seq=0 out_seq=0 2023-09-23T09:39:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.205+0000 7f95ccff9700 10 client.26227 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.205+0000 7f95ccff9700 1 -- 172.21.15.45:0/1019197990 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c8d21ee10 con 0x561c8d2fe260 2023-09-23T09:39:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.205+0000 7f95ccff9700 10 client.26227 handle_command_reply: tid=0 2023-09-23T09:39:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.205+0000 7f95f78fab80 1 client.26227 shutdown 2023-09-23T09:39:03.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2dac10 msgr2=0x561c8d2ef230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2dac10 0x561c8d2ef230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x561c8d2dbab0 tx=0x7f95e0099d50 comp rx=0 tx=0).stop 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 shutdown_connections 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d2fe260 0x561c8d300650 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2eee50 0x561c8d2fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2dac10 0x561c8d2ef230 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 --2- 172.21.15.45:0/1019197990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2db530 0x561c8d2ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 >> 172.21.15.45:0/1019197990 conn(0x561c8d247260 msgr2=0x561c8d2ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.212+0000 7f95ccff9700 0 client.26227 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.213+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 shutdown_connections 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.213+0000 7f95f78fab80 1 -- 172.21.15.45:0/1019197990 wait complete. 2023-09-23T09:39:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.213+0000 7f95f78fab80 20 client.26227 trim_cache size 0 max 16384 2023-09-23T09:39:03.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.235+0000 7f95f78fab80 1 Processor -- start 2023-09-23T09:39:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.235+0000 7f95f78fab80 1 -- start start 2023-09-23T09:39:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2db530 0x561c8d2d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2eee50 0x561c8d2f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95f78fab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d0a6920 con 0x561c8d2dac10 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95f78fab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d2cf5b0 con 0x561c8d2db530 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95f78fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d2d5cf0 con 0x561c8d2eee50 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2eee50 0x561c8d2f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2eee50 0x561c8d2f99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37740/0 (socket says 172.21.15.45:37740) 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95e9cad700 1 -- 172.21.15.45:0/125471100 learned_addr learned my addr 172.21.15.45:0/125471100 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95e8cab700 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2db530 0x561c8d2d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.236+0000 7f95e94ac700 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95e9cad700 1 -- 172.21.15.45:0/125471100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2db530 msgr2=0x561c8d2d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95e9cad700 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2db530 0x561c8d2d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95e9cad700 1 -- 172.21.15.45:0/125471100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 msgr2=0x561c8d2d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95e9cad700 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95e9cad700 1 -- 172.21.15.45:0/125471100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c8d2d2360 con 0x561c8d2eee50 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95e9cad700 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2eee50 0x561c8d2f99d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00e520 tx=0x7f95dc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4395f98fc80fdde7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95cffff700 1 -- 172.21.15.45:0/125471100 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc003c00 con 0x561c8d2eee50 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95cffff700 1 -- 172.21.15.45:0/125471100 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95dc00f040 con 0x561c8d2eee50 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.237+0000 7f95cffff700 1 -- 172.21.15.45:0/125471100 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc004280 con 0x561c8d2eee50 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 -- 172.21.15.45:0/125471100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2eee50 msgr2=0x561c8d2f99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2eee50 0x561c8d2f99d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00e520 tx=0x7f95dc004b40 comp rx=0 tx=0).stop 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 -- 172.21.15.45:0/125471100 shutdown_connections 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2eee50 0x561c8d2f99d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2dac10 0x561c8d2d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 --2- 172.21.15.45:0/125471100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2db530 0x561c8d2d2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 -- 172.21.15.45:0/125471100 >> 172.21.15.45:0/125471100 conn(0x561c8d247260 msgr2=0x561c8d2ef230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 -- 172.21.15.45:0/125471100 shutdown_connections 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 -- 172.21.15.45:0/125471100 wait complete. 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.238+0000 7f95f78fab80 1 -- start start 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 1 Processor -- start 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2dac10 0x561c8d2f4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2eee50 0x561c8d2f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d2f3180 con 0x561c8d2eee50 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d2f32f0 con 0x561c8d2dac10 2023-09-23T09:39:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8d34c040 con 0x561c8d2db530 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95f78fab80 10 client.0 _ll_register_callbacks cb 0x561c8d01f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95e94ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2dac10 0x561c8d2f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95e8cab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95e9cad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2eee50 0x561c8d2f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95e94ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2dac10 0x561c8d2f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:56990/0 (socket says 172.21.15.45:56990) 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95e94ac700 1 -- 172.21.15.45:0/3023728502 learned_addr learned my addr 172.21.15.45:0/3023728502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.239+0000 7f95e8cab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37746/0 (socket says 172.21.15.45:37746) 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95e9cad700 1 -- 172.21.15.45:0/3023728502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 msgr2=0x561c8d2f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95e9cad700 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95e9cad700 1 -- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2dac10 msgr2=0x561c8d2f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95e9cad700 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2dac10 0x561c8d2f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95e9cad700 1 -- 172.21.15.45:0/3023728502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc008ce0 con 0x561c8d2eee50 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95e94ac700 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2dac10 0x561c8d2f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95e9cad700 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2eee50 0x561c8d2f2aa0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00e4f0 tx=0x7f95dc009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e64dcdae644642 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95ccff9700 1 -- 172.21.15.45:0/3023728502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc023c60 con 0x561c8d2eee50 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c8d2fb000 con 0x561c8d2eee50 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95ccff9700 1 -- 172.21.15.45:0/3023728502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95dc02f070 con 0x561c8d2eee50 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95ccff9700 1 -- 172.21.15.45:0/3023728502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95dc00ee90 con 0x561c8d2eee50 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95f78fab80 10 client.16551 fetch_fsmap learned FSMap version 44 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.240+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x561c8d039c60 con 0x561c8d2eee50 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95ccff9700 1 -- 172.21.15.45:0/3023728502 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95dc040080 con 0x561c8d2eee50 2023-09-23T09:39:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95ccff9700 1 -- 172.21.15.45:0/3023728502 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f95dc029020 con 0x561c8d2eee50 2023-09-23T09:39:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95f78fab80 10 client.16551 fetch_fsmap finished waiting for FSMap version 44 2023-09-23T09:39:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95f78fab80 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95f78fab80 20 client.16551 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95f78fab80 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d34d490 0x561c8d34d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95f78fab80 4 client.16551 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:39:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f95e009e7a0 con 0x561c8d34d490 2023-09-23T09:39:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.241+0000 7f95e94ac700 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d34d490 0x561c8d34d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:03.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95ccff9700 10 client.16551 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:03.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95e94ac700 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d34d490 0x561c8d34d8a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4bec322bd02cc3e server_cookie=2bed67a22ea88b2f in_seq=0 out_seq=0 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95ccff9700 1 -- 172.21.15.45:0/3023728502 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x7f95e009e7a0 con 0x561c8d34d490 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95ccff9700 10 client.16551 handle_command_reply: tid=0 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 client.16551 shutdown 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2eee50 msgr2=0x561c8d2f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2eee50 0x561c8d2f2aa0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00e4f0 tx=0x7f95dc009e90 comp rx=0 tx=0).stop 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 shutdown_connections 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x561c8d34d490 0x561c8d34d8a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c8d2eee50 0x561c8d2f2aa0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561c8d2dac10 0x561c8d2f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95ccff9700 0 client.16551 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 --2- 172.21.15.45:0/3023728502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c8d2db530 0x561c8d2f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 >> 172.21.15.45:0/3023728502 conn(0x561c8d247260 msgr2=0x561c8d2ef230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 shutdown_connections 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.244+0000 7f95f78fab80 1 -- 172.21.15.45:0/3023728502 wait complete. 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95f78fab80 20 client.16551 trim_cache size 0 max 16384 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:39:03.250 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:39:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:39:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:39:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:39:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:39:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:39:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:39:03.251 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:39:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95d00a18d0 msgr2=0x7f95d00a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95d00a18d0 0x7f95d00a3d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0076a0 tx=0x7f95dc009040 comp rx=0 tx=0).stop 2023-09-23T09:39:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40ffed0 msgr2=0x7f95e4197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40ffed0 0x7f95e4197f70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f95e000e9f0 tx=0x7f95e000ed00 comp rx=0 tx=0).stop 2023-09-23T09:39:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 shutdown_connections 2023-09-23T09:39:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95d00a18d0 0x7f95d00a3d90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e4198a90 0x7f95e41a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40ffed0 0x7f95e4197f70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 --2- 172.21.15.45:0/670618919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95e4104ff0 0x7f95e41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 >> 172.21.15.45:0/670618919 conn(0x7f95e40fb770 msgr2=0x7f95e40fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 shutdown_connections 2023-09-23T09:39:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:03.245+0000 7f95eb710700 1 -- 172.21.15.45:0/670618919 wait complete. 2023-09-23T09:39:03.264 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:39:03.264 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:02 smithi045 ceph-mon[98627]: pgmap v298: 97 pgs: 97 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.7 MiB/s wr, 591 op/s 2023-09-23T09:39:03.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:02 smithi098 ceph-mon[102386]: pgmap v298: 97 pgs: 97 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.7 MiB/s wr, 591 op/s 2023-09-23T09:39:03.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:02 smithi123 ceph-mon[103577]: pgmap v298: 97 pgs: 97 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.7 MiB/s wr, 591 op/s 2023-09-23T09:39:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:05 smithi098 ceph-mon[102386]: pgmap v299: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 23 MiB/s wr, 1.81k op/s 2023-09-23T09:39:05.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:05 smithi045 ceph-mon[98627]: pgmap v299: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 23 MiB/s wr, 1.81k op/s 2023-09-23T09:39:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:05 smithi123 ceph-mon[103577]: pgmap v299: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 23 MiB/s wr, 1.81k op/s 2023-09-23T09:39:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 560.0 s, 1671.0 tps, lat 59.693 ms stddev 70.653 2023-09-23T09:39:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 -- 172.21.15.45:0/1208800090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98106130 msgr2=0x7f6e9810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 --2- 172.21.15.45:0/1208800090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98106130 0x7f6e9810dcb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000b0a0 tx=0x7f6e9000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 -- 172.21.15.45:0/1208800090 shutdown_connections 2023-09-23T09:39:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 --2- 172.21.15.45:0/1208800090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98106130 0x7f6e9810dcb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 --2- 172.21.15.45:0/1208800090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98101010 0x7f6e981013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 --2- 172.21.15.45:0/1208800090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e981019c0 0x7f6e98105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 -- 172.21.15.45:0/1208800090 >> 172.21.15.45:0/1208800090 conn(0x7f6e980fc8d0 msgr2=0x7f6e980fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 -- 172.21.15.45:0/1208800090 shutdown_connections 2023-09-23T09:39:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.487+0000 7f6e9f669700 1 -- 172.21.15.45:0/1208800090 wait complete. 2023-09-23T09:39:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 Processor -- start 2023-09-23T09:39:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 -- start start 2023-09-23T09:39:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e98101010 0x7f6e98197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98106130 0x7f6e98197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9819bc00 0x7f6e9819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819c680 con 0x7f6e9819bc00 2023-09-23T09:39:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819c7f0 con 0x7f6e98101010 2023-09-23T09:39:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9f669700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819c9b0 con 0x7f6e98106130 2023-09-23T09:39:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9cc04700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98106130 0x7f6e98197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9cc04700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98106130 0x7f6e98197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37762/0 (socket says 172.21.15.45:37762) 2023-09-23T09:39:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9cc04700 1 -- 172.21.15.45:0/3593571794 learned_addr learned my addr 172.21.15.45:0/3593571794 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9d405700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e98101010 0x7f6e98197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.488+0000 7f6e9dc06700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9819bc00 0x7f6e9819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9cc04700 1 -- 172.21.15.45:0/3593571794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e98101010 msgr2=0x7f6e98197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e98101010 0x7f6e98197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9cc04700 1 -- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9819bc00 msgr2=0x7f6e9819c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9819bc00 0x7f6e9819c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9cc04700 1 -- 172.21.15.45:0/3593571794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e94009520 con 0x7f6e98106130 2023-09-23T09:39:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9d405700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e98101010 0x7f6e98197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98106130 0x7f6e98197590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800e9f0 tx=0x7f6e8800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e8e7fc700 1 -- 172.21.15.45:0/3593571794 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88009750 con 0x7f6e98106130 2023-09-23T09:39:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e90009d00 con 0x7f6e98106130 2023-09-23T09:39:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e8e7fc700 1 -- 172.21.15.45:0/3593571794 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e8800c7c0 con 0x7f6e98106130 2023-09-23T09:39:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e8e7fc700 1 -- 172.21.15.45:0/3593571794 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880054d0 con 0x7f6e98106130 2023-09-23T09:39:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.489+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e98073430 con 0x7f6e98106130 2023-09-23T09:39:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.492+0000 7f6e8e7fc700 1 -- 172.21.15.45:0/3593571794 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f6e8800c930 con 0x7f6e98106130 2023-09-23T09:39:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.493+0000 7f6e8e7fc700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6e840a1710 0x7f6e840a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.494+0000 7f6e8e7fc700 1 -- 172.21.15.45:0/3593571794 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6e88014070 con 0x7f6e98106130 2023-09-23T09:39:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.494+0000 7f6e9d405700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6e840a1710 0x7f6e840a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.495+0000 7f6e9d405700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6e840a1710 0x7f6e840a3bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 Processor -- start 2023-09-23T09:39:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 -- start start 2023-09-23T09:39:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8a2e400 con 0x560dc8ac5860 2023-09-23T09:39:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8a1fbc0 con 0x560dc8ad0710 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6eab853b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8a1d4e0 con 0x560dc8ac4f40 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9d405700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35664/0 (socket says 172.21.15.45:35664) 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9d405700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37768/0 (socket says 172.21.15.45:37768) 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 -- 172.21.15.45:0/803245447 learned_addr learned my addr 172.21.15.45:0/803245447 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9dc06700 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 -- 172.21.15.45:0/803245447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 msgr2=0x560dc8ac5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 -- 172.21.15.45:0/803245447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 msgr2=0x560dc8ad2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9cc04700 1 -- 172.21.15.45:0/803245447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dc8a7b870 con 0x560dc8ac5860 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9d405700 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.497+0000 7f6e9dc06700 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.498+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad01d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6e8809a0d0 tx=0x7f6e88099db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c69b1e0ccae150 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.498+0000 7f6e837fe700 1 -- 172.21.15.45:0/803245447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8809e520 con 0x560dc8ac5860 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.498+0000 7f6e837fe700 1 -- 172.21.15.45:0/803245447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e880a2760 con 0x560dc8ac5860 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.498+0000 7f6e837fe700 1 -- 172.21.15.45:0/803245447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8809a630 con 0x560dc8ac5860 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 -- 172.21.15.45:0/803245447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 msgr2=0x560dc8ad01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad01d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6e8809a0d0 tx=0x7f6e88099db0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 -- 172.21.15.45:0/803245447 shutdown_connections 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 --2- 172.21.15.45:0/803245447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad01d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 -- 172.21.15.45:0/803245447 >> 172.21.15.45:0/803245447 conn(0x560dc8a3a200 msgr2=0x560dc8ac13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 -- 172.21.15.45:0/803245447 shutdown_connections 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 -- 172.21.15.45:0/803245447 wait complete. 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 -- start start 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.499+0000 7f6eab853b80 1 Processor -- start 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac4f40 0x560dc8acd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ac5860 0x560dc8acd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ad0710 0x560dc8acdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6eab853b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8a18dd0 con 0x560dc8ac4f40 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6eab853b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc89b56b0 con 0x560dc8ac5860 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6eab853b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc89b5350 con 0x560dc8ad0710 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6eab853b80 10 client.0 _ll_register_callbacks cb 0x560dc88127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6e9dc06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ad0710 0x560dc8acdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6e9d405700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac4f40 0x560dc8acd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6e9dc06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ad0710 0x560dc8acdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37778/0 (socket says 172.21.15.45:37778) 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6e9dc06700 1 -- 172.21.15.45:0/533642296 learned_addr learned my addr 172.21.15.45:0/533642296 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ac5860 0x560dc8acd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.500+0000 7f6e9d405700 1 -- 172.21.15.45:0/533642296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ad0710 msgr2=0x560dc8acdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e9d405700 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ad0710 0x560dc8acdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e9d405700 1 -- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ac5860 msgr2=0x560dc8acd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e9d405700 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ac5860 0x560dc8acd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e9d405700 1 -- 172.21.15.45:0/533642296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9810e520 con 0x560dc8ac4f40 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e9d405700 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac4f40 0x560dc8acd4b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6e94011d50 tx=0x7f6e94013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8058e28e15a2763e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e80ff9700 1 -- 172.21.15.45:0/533642296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94021040 con 0x560dc8ac4f40 2023-09-23T09:39:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e80ff9700 1 -- 172.21.15.45:0/533642296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6e9401bca0 con 0x560dc8ac4f40 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dc8ab9090 con 0x560dc8ac4f40 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6e80ff9700 1 -- 172.21.15.45:0/533642296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e9401be60 con 0x560dc8ac4f40 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6eab853b80 10 client.16563 fetch_fsmap learned FSMap version 44 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.501+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f6e98073430 con 0x560dc8ac4f40 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6e80ff9700 1 -- 172.21.15.45:0/533642296 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dc8ab9090 con 0x560dc8ac4f40 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6e80ff9700 1 -- 172.21.15.45:0/533642296 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6e9402f4c0 con 0x560dc8ac4f40 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6eab853b80 10 client.16563 fetch_fsmap finished waiting for FSMap version 44 2023-09-23T09:39:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6eab853b80 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6eab853b80 20 client.16563 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6eab853b80 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8ae03b0 0x560dc8ae27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6eab853b80 4 client.16563 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dc8a11db0 con 0x560dc8ae03b0 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.502+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8ae03b0 0x560dc8ae27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.503+0000 7f6e80ff9700 10 client.16563 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.503+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8ae03b0 0x560dc8ae27d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b5d4441c2f8732 server_cookie=bd3e54aa995ac9bd in_seq=0 out_seq=0 2023-09-23T09:39:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.503+0000 7f6e80ff9700 1 -- 172.21.15.45:0/533642296 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560dc8a11db0 con 0x560dc8ae03b0 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.503+0000 7f6e80ff9700 10 client.16563 handle_command_reply: tid=0 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.503+0000 7f6eab853b80 1 client.16563 shutdown 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac4f40 msgr2=0x560dc8acd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac4f40 0x560dc8acd4b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6e94011d50 tx=0x7f6e94013040 comp rx=0 tx=0).stop 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 shutdown_connections 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8ae03b0 0x560dc8ae27d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ad0710 0x560dc8acdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac4f40 0x560dc8acd4b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 --2- 172.21.15.45:0/533642296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ac5860 0x560dc8acd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 >> 172.21.15.45:0/533642296 conn(0x560dc8a3a200 msgr2=0x560dc8ac0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6e80ff9700 0 client.16563 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 shutdown_connections 2023-09-23T09:39:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 1 -- 172.21.15.45:0/533642296 wait complete. 2023-09-23T09:39:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.504+0000 7f6eab853b80 20 client.16563 trim_cache size 0 max 16384 2023-09-23T09:39:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 Processor -- start 2023-09-23T09:39:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 -- start start 2023-09-23T09:39:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8accc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ac8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc88998c0 con 0x560dc8ac5860 2023-09-23T09:39:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8abdba0 con 0x560dc8ad0710 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6eab853b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8ac55a0 con 0x560dc8ac4f40 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.526+0000 7f6e9d405700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37782/0 (socket says 172.21.15.45:37782) 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 -- 172.21.15.45:0/393221658 learned_addr learned my addr 172.21.15.45:0/393221658 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9dc06700 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ac8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8accc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 -- 172.21.15.45:0/393221658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 msgr2=0x560dc8ac8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ac8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 -- 172.21.15.45:0/393221658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 msgr2=0x560dc8accc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8accc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 -- 172.21.15.45:0/393221658 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dc8843d80 con 0x560dc8ac4f40 2023-09-23T09:39:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.527+0000 7f6e9d405700 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac7ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009f60 tx=0x7f6e940057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdf38492853f2079 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.528+0000 7f6e83fff700 1 -- 172.21.15.45:0/393221658 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94021040 con 0x560dc8ac4f40 2023-09-23T09:39:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.528+0000 7f6e83fff700 1 -- 172.21.15.45:0/393221658 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e940092d0 con 0x560dc8ac4f40 2023-09-23T09:39:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.528+0000 7f6e83fff700 1 -- 172.21.15.45:0/393221658 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e940046d0 con 0x560dc8ac4f40 2023-09-23T09:39:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 -- 172.21.15.45:0/393221658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 msgr2=0x560dc8ac7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac7ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009f60 tx=0x7f6e940057a0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 -- 172.21.15.45:0/393221658 shutdown_connections 2023-09-23T09:39:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ac8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8ac4f40 0x560dc8ac7ee0 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009f60 tx=0x7f6e940057a0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 --2- 172.21.15.45:0/393221658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8accc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 -- 172.21.15.45:0/393221658 >> 172.21.15.45:0/393221658 conn(0x560dc8a3a200 msgr2=0x560dc8adc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 -- 172.21.15.45:0/393221658 shutdown_connections 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.529+0000 7f6eab853b80 1 -- 172.21.15.45:0/393221658 wait complete. 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 -- start start 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 Processor -- start 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8b421a0 0x560dc8b44590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8b44ad0 con 0x560dc8ac5860 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8b44c10 con 0x560dc8ad0710 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8b44dd0 con 0x560dc8b421a0 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.530+0000 7f6eab853b80 10 client.0 _ll_register_callbacks cb 0x7f6e9400e970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57032/0 (socket says 172.21.15.45:57032) 2023-09-23T09:39:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 -- 172.21.15.45:0/163089617 learned_addr learned my addr 172.21.15.45:0/163089617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9dc06700 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8b421a0 0x560dc8b44590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9d405700 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 -- 172.21.15.45:0/163089617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8b421a0 msgr2=0x560dc8b44590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8b421a0 0x560dc8b44590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 -- 172.21.15.45:0/163089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 msgr2=0x560dc8ad4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 -- 172.21.15.45:0/163089617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e98073430 con 0x560dc8ad0710 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e9cc04700 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad5530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e880cac60 tx=0x7f6e88009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50e85023c42aefff server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:39:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e80ff9700 1 -- 172.21.15.45:0/163089617 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8807d070 con 0x560dc8ad0710 2023-09-23T09:39:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6e80ff9700 1 -- 172.21.15.45:0/163089617 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6e88098770 con 0x560dc8ad0710 2023-09-23T09:39:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.531+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e880a2760 con 0x560dc8ad0710 2023-09-23T09:39:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.532+0000 7f6e80ff9700 1 -- 172.21.15.45:0/163089617 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e880b8040 con 0x560dc8ad0710 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.532+0000 7f6eab853b80 10 client.26252 fetch_fsmap learned FSMap version 44 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.532+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560dc882cc00 con 0x560dc8ad0710 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6e80ff9700 1 -- 172.21.15.45:0/163089617 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e880a2760 con 0x560dc8ad0710 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6e80ff9700 1 -- 172.21.15.45:0/163089617 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6e88065350 con 0x560dc8ad0710 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6eab853b80 10 client.26252 fetch_fsmap finished waiting for FSMap version 44 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6eab853b80 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6eab853b80 20 client.26252 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6eab853b80 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8b46270 0x560dc8b48690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6eab853b80 4 client.26252 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:39:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560dc8a11ad0 con 0x560dc8b46270 2023-09-23T09:39:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.533+0000 7f6e9d405700 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8b46270 0x560dc8b48690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.534+0000 7f6e80ff9700 10 client.26252 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.534+0000 7f6e9d405700 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8b46270 0x560dc8b48690 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ddb2ed5aa75356 server_cookie=82d2aa809e275c61 in_seq=0 out_seq=0 2023-09-23T09:39:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.552+0000 7f6e80ff9700 1 -- 172.21.15.45:0/163089617 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560dc8a11ad0 con 0x560dc8b46270 2023-09-23T09:39:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.552+0000 7f6e80ff9700 10 client.26252 handle_command_reply: tid=0 2023-09-23T09:39:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.552+0000 7f6eab853b80 1 client.26252 shutdown 2023-09-23T09:39:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.552+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 msgr2=0x560dc8ad5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.552+0000 7f6eab853b80 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad5530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e880cac60 tx=0x7f6e88009f70 comp rx=0 tx=0).stop 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 shutdown_connections 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x560dc8b46270 0x560dc8b48690 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560dc8b421a0 0x560dc8b44590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dc8ac5860 0x560dc8ad4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 --2- 172.21.15.45:0/163089617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dc8ad0710 0x560dc8ad5530 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6e80ff9700 0 client.26252 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 >> 172.21.15.45:0/163089617 conn(0x560dc8a3a200 msgr2=0x560dc8acdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 shutdown_connections 2023-09-23T09:39:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 1 -- 172.21.15.45:0/163089617 wait complete. 2023-09-23T09:39:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.554+0000 7f6eab853b80 20 client.26252 trim_cache size 0 max 16384 2023-09-23T09:39:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:39:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:39:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:39:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:39:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6e840a1710 msgr2=0x7f6e840a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6e840a1710 0x7f6e840a3bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94009260 comp rx=0 tx=0).stop 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98106130 msgr2=0x7f6e98197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98106130 0x7f6e98197590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800e9f0 tx=0x7f6e8800ed00 comp rx=0 tx=0).stop 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 shutdown_connections 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6e840a1710 0x7f6e840a3bd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9819bc00 0x7f6e9819c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e98101010 0x7f6e98197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 --2- 172.21.15.45:0/3593571794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e98106130 0x7f6e98197590 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 >> 172.21.15.45:0/3593571794 conn(0x7f6e980fc8d0 msgr2=0x7f6e980fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 shutdown_connections 2023-09-23T09:39:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:07.555+0000 7f6e9f669700 1 -- 172.21.15.45:0/3593571794 wait complete. 2023-09-23T09:39:08.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:07 smithi098 ceph-mon[102386]: pgmap v300: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-09-23T09:39:08.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:07 smithi045 ceph-mon[98627]: pgmap v300: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-09-23T09:39:08.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:07 smithi123 ceph-mon[103577]: pgmap v300: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-09-23T09:39:10.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:09 smithi098 ceph-mon[102386]: pgmap v301: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2023-09-23T09:39:10.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:09 smithi045 ceph-mon[98627]: pgmap v301: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2023-09-23T09:39:10.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:09 smithi123 ceph-mon[103577]: pgmap v301: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2023-09-23T09:39:11.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:10 smithi098 ceph-mon[102386]: osdmap e94: 12 total, 12 up, 12 in 2023-09-23T09:39:11.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:10 smithi045 ceph-mon[98627]: osdmap e94: 12 total, 12 up, 12 in 2023-09-23T09:39:11.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:10 smithi123 ceph-mon[103577]: osdmap e94: 12 total, 12 up, 12 in 2023-09-23T09:39:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 565.0 s, 1234.0 tps, lat 79.865 ms stddev 112.468 2023-09-23T09:39:12.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:11 smithi098 ceph-mon[102386]: pgmap v303: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-09-23T09:39:12.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:11 smithi045 ceph-mon[98627]: pgmap v303: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-09-23T09:39:12.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:11 smithi123 ceph-mon[103577]: pgmap v303: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-09-23T09:39:14.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:13 smithi098 ceph-mon[102386]: pgmap v304: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-09-23T09:39:14.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:13 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:39:14.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:13 smithi045 ceph-mon[98627]: pgmap v304: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-09-23T09:39:14.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:13 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:39:14.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:13 smithi123 ceph-mon[103577]: pgmap v304: 97 pgs: 97 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-09-23T09:39:14.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:13 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:39:16.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:15 smithi098 ceph-mon[102386]: pgmap v305: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.5 MiB/s wr, 619 op/s 2023-09-23T09:39:16.050 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:15 smithi098 ceph-mon[102386]: osdmap e95: 12 total, 12 up, 12 in 2023-09-23T09:39:16.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:15 smithi045 ceph-mon[98627]: pgmap v305: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.5 MiB/s wr, 619 op/s 2023-09-23T09:39:16.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:15 smithi045 ceph-mon[98627]: osdmap e95: 12 total, 12 up, 12 in 2023-09-23T09:39:16.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:15 smithi123 ceph-mon[103577]: pgmap v305: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.5 MiB/s wr, 619 op/s 2023-09-23T09:39:16.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:15 smithi123 ceph-mon[103577]: osdmap e95: 12 total, 12 up, 12 in 2023-09-23T09:39:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 570.0 s, 1202.8 tps, lat 60.611 ms stddev 74.666 2023-09-23T09:39:18.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:17 smithi098 ceph-mon[102386]: pgmap v307: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.1 MiB/s wr, 691 op/s 2023-09-23T09:39:18.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:17 smithi045 ceph-mon[98627]: pgmap v307: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.1 MiB/s wr, 691 op/s 2023-09-23T09:39:18.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:17 smithi123 ceph-mon[103577]: pgmap v307: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.1 MiB/s wr, 691 op/s 2023-09-23T09:39:20.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:19 smithi098 ceph-mon[102386]: pgmap v308: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.6 MiB/s wr, 393 op/s 2023-09-23T09:39:20.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:19 smithi045 ceph-mon[98627]: pgmap v308: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.6 MiB/s wr, 393 op/s 2023-09-23T09:39:20.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:19 smithi123 ceph-mon[103577]: pgmap v308: 97 pgs: 97 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.6 MiB/s wr, 393 op/s 2023-09-23T09:39:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.741+0000 7f7fd4794700 1 -- 172.21.15.45:0/3764729391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100ad0 msgr2=0x7f7fcc104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.741+0000 7f7fd4794700 1 --2- 172.21.15.45:0/3764729391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100ad0 0x7f7fcc104950 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc009c00 comp rx=0 tx=0).stop 2023-09-23T09:39:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 -- 172.21.15.45:0/3764729391 shutdown_connections 2023-09-23T09:39:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 --2- 172.21.15.45:0/3764729391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fcc105080 0x7f7fcc10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 --2- 172.21.15.45:0/3764729391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fcc100120 0x7f7fcc100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 --2- 172.21.15.45:0/3764729391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100ad0 0x7f7fcc104950 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 -- 172.21.15.45:0/3764729391 >> 172.21.15.45:0/3764729391 conn(0x7f7fcc077c50 msgr2=0x7f7fcc078060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 -- 172.21.15.45:0/3764729391 shutdown_connections 2023-09-23T09:39:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 -- 172.21.15.45:0/3764729391 wait complete. 2023-09-23T09:39:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 Processor -- start 2023-09-23T09:39:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.742+0000 7f7fd4794700 1 -- start start 2023-09-23T09:39:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd4794700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100120 0x7f7fcc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd4794700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fcc105080 0x7f7fcc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd4794700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fcc1a0ec0 0x7f7fcc1a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd4794700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1a37f0 con 0x7f7fcc105080 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd4794700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1a3960 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd4794700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1a3b20 con 0x7f7fcc100120 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd2530700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100120 0x7f7fcc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd2530700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100120 0x7f7fcc198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37802/0 (socket says 172.21.15.45:37802) 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd2530700 1 -- 172.21.15.45:0/803118237 learned_addr learned my addr 172.21.15.45:0/803118237 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fcc105080 0x7f7fcc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.743+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fcc1a0ec0 0x7f7fcc1a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fd2d31700 1 -- 172.21.15.45:0/803118237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100120 msgr2=0x7f7fcc198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100120 0x7f7fcc198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fd2d31700 1 -- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fcc105080 msgr2=0x7f7fcc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fcc105080 0x7f7fcc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fd2d31700 1 -- 172.21.15.45:0/803118237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80095f0 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fcc105080 0x7f7fcc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fcc1a0ec0 0x7f7fcc1a32b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400efc0 tx=0x7f7fc400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fc37fe700 1 -- 172.21.15.45:0/803118237 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4018070 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fc37fe700 1 -- 172.21.15.45:0/803118237 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc400f040 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.744+0000 7f7fc37fe700 1 -- 172.21.15.45:0/803118237 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400cc80 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.745+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc009520 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.745+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc195f40 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.747+0000 7f7fc37fe700 1 -- 172.21.15.45:0/803118237 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f7fc4003a00 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.747+0000 7f7fc37fe700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f7fb80a5b20 0x7f7fb80a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.748+0000 7f7fc37fe700 1 -- 172.21.15.45:0/803118237 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7fc40ccec0 con 0x7f7fcc1a0ec0 2023-09-23T09:39:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.748+0000 7f7fd2530700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f7fb80a5b20 0x7f7fb80a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.748+0000 7f7fd2530700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f7fb80a5b20 0x7f7fb80a7fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005d10 tx=0x7f7fc8009310 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.749+0000 7f7fe097eb80 1 Processor -- start 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fe097eb80 1 -- start start 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba76b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba7761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba778b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba6c3400 con 0x55bfba76af50 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba6b4bc0 con 0x55bfba76b870 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba6b24e0 con 0x55bfba776720 2023-09-23T09:39:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fd2530700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba76b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fd1d2f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba7761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fd1d2f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba7761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57056/0 (socket says 172.21.15.45:57056) 2023-09-23T09:39:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fd2d31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba778b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fd2530700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba76b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35704/0 (socket says 172.21.15.45:35704) 2023-09-23T09:39:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.750+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/2519509553 learned_addr learned my addr 172.21.15.45:0/2519509553 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/2519509553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 msgr2=0x55bfba778b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba778b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/2519509553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 msgr2=0x55bfba76b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba76b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/2519509553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfba710870 con 0x55bfba76b870 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba778b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd2530700 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba76b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba7761e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005e90 tx=0x7f7fbc005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29fb1e40a3126e81 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fc0ff9700 1 -- 172.21.15.45:0/2519509553 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc01c050 con 0x55bfba76b870 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fc0ff9700 1 -- 172.21.15.45:0/2519509553 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fbc01d040 con 0x55bfba76b870 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.751+0000 7f7fc0ff9700 1 -- 172.21.15.45:0/2519509553 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0052a0 con 0x55bfba76b870 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2519509553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 msgr2=0x55bfba7761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba7761e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005e90 tx=0x7f7fbc005dc0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2519509553 shutdown_connections 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba778b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba76b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2519509553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba7761e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2519509553 >> 172.21.15.45:0/2519509553 conn(0x55bfba6cf200 msgr2=0x55bfba756330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.752+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2519509553 shutdown_connections 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.753+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2519509553 wait complete. 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.753+0000 7f7fe097eb80 1 -- start start 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.753+0000 7f7fe097eb80 1 Processor -- start 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba776160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba7733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba773b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba6addd0 con 0x55bfba76b870 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba64a6b0 con 0x55bfba76af50 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba64a350 con 0x55bfba776720 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fe097eb80 10 client.0 _ll_register_callbacks cb 0x55bfba4a77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba773b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd1d2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba7733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba773b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37816/0 (socket says 172.21.15.45:37816) 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 -- 172.21.15.45:0/2173201749 learned_addr learned my addr 172.21.15.45:0/2173201749 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2530700 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba776160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 -- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 msgr2=0x55bfba776160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba776160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 -- 172.21.15.45:0/2173201749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 msgr2=0x55bfba7733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba7733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.754+0000 7f7fd2d31700 1 -- 172.21.15.45:0/2173201749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc195f40 con 0x55bfba776720 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba773b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c6f0 tx=0x7f7fc40997c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d759ab05228c10b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/2173201749 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc409a4e0 con 0x55bfba776720 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/2173201749 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7fc4099de0 con 0x55bfba776720 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfba74e090 con 0x55bfba776720 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/2173201749 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fc409df30 con 0x55bfba776720 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 10 client.26263 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f7fcc10d470 con 0x55bfba776720 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/2173201749 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc4063080 con 0x55bfba776720 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/2173201749 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7fc40992d0 con 0x55bfba776720 2023-09-23T09:39:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 10 client.26263 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 20 client.26263 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba786090 0x55bfba7884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 4 client.26263 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.755+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfba6a6db0 con 0x55bfba786090 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.756+0000 7f7fd2530700 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba786090 0x55bfba7884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.756+0000 7f7fb5ffb700 10 client.26263 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.756+0000 7f7fd2530700 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba786090 0x55bfba7884b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab819ef0e7f8864 server_cookie=3fd5add04bb6b48c in_seq=0 out_seq=0 2023-09-23T09:39:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.757+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/2173201749 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bfba6a6db0 con 0x55bfba786090 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.757+0000 7f7fb5ffb700 10 client.26263 handle_command_reply: tid=0 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.757+0000 7f7fe097eb80 1 client.26263 shutdown 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 msgr2=0x55bfba773b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba773b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c6f0 tx=0x7f7fc40997c0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 shutdown_connections 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba786090 0x55bfba7884b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba773b20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba776160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/2173201749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba7733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 >> 172.21.15.45:0/2173201749 conn(0x55bfba6cf200 msgr2=0x55bfba755f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fb5ffb700 0 client.26263 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.758+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 shutdown_connections 2023-09-23T09:39:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.759+0000 7f7fe097eb80 1 -- 172.21.15.45:0/2173201749 wait complete. 2023-09-23T09:39:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.759+0000 7f7fe097eb80 20 client.26263 trim_cache size 0 max 16384 2023-09-23T09:39:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 Processor -- start 2023-09-23T09:39:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 -- start start 2023-09-23T09:39:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba773aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba772c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba782400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba52e8c0 con 0x55bfba76b870 2023-09-23T09:39:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba7749c0 con 0x55bfba76af50 2023-09-23T09:39:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.777+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba76e2a0 con 0x55bfba776720 2023-09-23T09:39:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba772c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba772c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35720/0 (socket says 172.21.15.45:35720) 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/208780856 learned_addr learned my addr 172.21.15.45:0/208780856 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd2530700 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba773aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/208780856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 msgr2=0x55bfba782400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba782400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/208780856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 msgr2=0x55bfba773aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba773aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.778+0000 7f7fd1d2f700 1 -- 172.21.15.45:0/208780856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfba4d8d80 con 0x55bfba76b870 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.779+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba772c40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009ff0 tx=0x7f7fbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf7ec2cdd891137 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.779+0000 7f7fc17fa700 1 -- 172.21.15.45:0/208780856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00f660 con 0x55bfba76b870 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.779+0000 7f7fc17fa700 1 -- 172.21.15.45:0/208780856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fbc00f7c0 con 0x55bfba76b870 2023-09-23T09:39:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.779+0000 7f7fc17fa700 1 -- 172.21.15.45:0/208780856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00bb40 con 0x55bfba76b870 2023-09-23T09:39:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 -- 172.21.15.45:0/208780856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 msgr2=0x55bfba772c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba772c40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009ff0 tx=0x7f7fbc009c00 comp rx=0 tx=0).stop 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 -- 172.21.15.45:0/208780856 shutdown_connections 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba782400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76af50 0x55bfba773aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/208780856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76b870 0x55bfba772c40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 -- 172.21.15.45:0/208780856 >> 172.21.15.45:0/208780856 conn(0x55bfba6cf200 msgr2=0x55bfba754af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 -- 172.21.15.45:0/208780856 shutdown_connections 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 -- 172.21.15.45:0/208780856 wait complete. 2023-09-23T09:39:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.780+0000 7f7fe097eb80 1 -- start start 2023-09-23T09:39:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 1 Processor -- start 2023-09-23T09:39:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba77d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba77b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba77b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba77be20 con 0x55bfba76af50 2023-09-23T09:39:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba77bf90 con 0x55bfba76b870 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfba7d8ec0 con 0x55bfba776720 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fe097eb80 10 client.0 _ll_register_callbacks cb 0x7f7fc4099bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba77d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2d31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba77b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba77d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35724/0 (socket says 172.21.15.45:35724) 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 -- 172.21.15.45:0/3076061813 learned_addr learned my addr 172.21.15.45:0/3076061813 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2d31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba77b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37834/0 (socket says 172.21.15.45:37834) 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba77b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 -- 172.21.15.45:0/3076061813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 msgr2=0x55bfba77b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba77b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 -- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 msgr2=0x55bfba77b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba77b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2530700 1 -- 172.21.15.45:0/3076061813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfba710870 con 0x55bfba76af50 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.781+0000 7f7fd2d31700 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba77b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fd2530700 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba77d210 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8011b60 tx=0x7f7fc80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb009deb5f00fc2d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/3076061813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8022040 con 0x55bfba76af50 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fbc01d040 con 0x55bfba76af50 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/3076061813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7fc802b2e0 con 0x55bfba76af50 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/3076061813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fc800eea0 con 0x55bfba76af50 2023-09-23T09:39:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 10 client.16587 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55bfba4c1c00 con 0x55bfba76af50 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/3076061813 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc8063080 con 0x55bfba76af50 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/3076061813 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7fc80573f0 con 0x55bfba76af50 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 10 client.16587 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 20 client.16587 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba7584a0 0x55bfba75a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 4 client.16587 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.782+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bfba6a6ad0 con 0x55bfba7584a0 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.783+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba7584a0 0x55bfba75a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.783+0000 7f7fd1d2f700 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba7584a0 0x55bfba75a8c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76553956d0fbd9c6 server_cookie=ede7749db5981431 in_seq=0 out_seq=0 2023-09-23T09:39:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.783+0000 7f7fb5ffb700 10 client.16587 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fb5ffb700 1 -- 172.21.15.45:0/3076061813 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bfba6a6ad0 con 0x55bfba7584a0 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fb5ffb700 10 client.16587 handle_command_reply: tid=0 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 client.16587 shutdown 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 msgr2=0x55bfba77d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba77d210 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8011b60 tx=0x7f7fc80047c0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 shutdown_connections 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55bfba7584a0 0x55bfba75a8c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfba776720 0x55bfba77b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfba76af50 0x55bfba77d210 secure :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8011b60 tx=0x7f7fc80047c0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 --2- 172.21.15.45:0/3076061813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfba76b870 0x55bfba77b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 >> 172.21.15.45:0/3076061813 conn(0x55bfba6cf200 msgr2=0x55bfba782400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fb5ffb700 0 client.16587 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.849+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 shutdown_connections 2023-09-23T09:39:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.850+0000 7f7fe097eb80 1 -- 172.21.15.45:0/3076061813 wait complete. 2023-09-23T09:39:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.850+0000 7f7fe097eb80 20 client.16587 trim_cache size 0 max 16384 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.850+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f7fb80a5b20 msgr2=0x7f7fb80a7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.850+0000 7f7fd4794700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f7fb80a5b20 0x7f7fb80a7fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005d10 tx=0x7f7fc8009310 comp rx=0 tx=0).stop 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.850+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fcc1a0ec0 msgr2=0x7f7fcc1a32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.850+0000 7f7fd4794700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fcc1a0ec0 0x7f7fcc1a32b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400efc0 tx=0x7f7fc400c3a0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 shutdown_connections 2023-09-23T09:39:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fcc1a0ec0 0x7f7fcc1a32b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fcc100120 0x7f7fcc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f7fb80a5b20 0x7f7fb80a7fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 --2- 172.21.15.45:0/803118237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fcc105080 0x7f7fcc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 >> 172.21.15.45:0/803118237 conn(0x7f7fcc077c50 msgr2=0x7f7fcc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 shutdown_connections 2023-09-23T09:39:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:20.853+0000 7f7fd4794700 1 -- 172.21.15.45:0/803118237 wait complete. 2023-09-23T09:39:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 575.0 s, 768.6 tps, lat 166.208 ms stddev 346.036 2023-09-23T09:39:22.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:21 smithi098 ceph-mon[102386]: pgmap v309: 97 pgs: 97 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2023-09-23T09:39:22.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:21 smithi045 ceph-mon[98627]: pgmap v309: 97 pgs: 97 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2023-09-23T09:39:22.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:21 smithi123 ceph-mon[103577]: pgmap v309: 97 pgs: 97 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2023-09-23T09:39:24.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:23 smithi098 ceph-mon[102386]: pgmap v310: 97 pgs: 97 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2023-09-23T09:39:24.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:23 smithi045 ceph-mon[98627]: pgmap v310: 97 pgs: 97 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2023-09-23T09:39:24.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:23 smithi123 ceph-mon[103577]: pgmap v310: 97 pgs: 97 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2023-09-23T09:39:26.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:25 smithi098 ceph-mon[102386]: pgmap v311: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-09-23T09:39:26.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:25 smithi045 ceph-mon[98627]: pgmap v311: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-09-23T09:39:26.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:25 smithi123 ceph-mon[103577]: pgmap v311: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-09-23T09:39:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 580.0 s, 1081.2 tps, lat 92.310 ms stddev 100.149 2023-09-23T09:39:28.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:27 smithi098 ceph-mon[102386]: pgmap v312: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-09-23T09:39:28.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:27 smithi045 ceph-mon[98627]: pgmap v312: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-09-23T09:39:28.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:27 smithi123 ceph-mon[103577]: pgmap v312: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-09-23T09:39:30.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:29 smithi098 ceph-mon[102386]: pgmap v313: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-23T09:39:30.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:29 smithi045 ceph-mon[98627]: pgmap v313: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-23T09:39:30.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:29 smithi123 ceph-mon[103577]: pgmap v313: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-23T09:39:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 585.0 s, 1461.6 tps, lat 69.360 ms stddev 83.825 2023-09-23T09:39:32.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:31 smithi098 ceph-mon[102386]: pgmap v314: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2023-09-23T09:39:32.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:31 smithi045 ceph-mon[98627]: pgmap v314: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2023-09-23T09:39:32.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:31 smithi123 ceph-mon[103577]: pgmap v314: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2023-09-23T09:39:33.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:39:33.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.507+0000 7f6ecca35700 1 -- 172.21.15.45:0/1759718919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec80ff600 msgr2=0x7f6ec80ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.507+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1759718919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec80ff600 0x7f6ec80ffa80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009990 tx=0x7f6eb8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.508+0000 7f6ecca35700 1 -- 172.21.15.45:0/1759718919 shutdown_connections 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.508+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1759718919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80fffc0 0x7f6ec810c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.508+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1759718919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec81054c0 0x7f6ec81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.508+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1759718919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec80ff600 0x7f6ec80ffa80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.508+0000 7f6ecca35700 1 -- 172.21.15.45:0/1759718919 >> 172.21.15.45:0/1759718919 conn(0x7f6ec8074f20 msgr2=0x7f6ec8075330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.508+0000 7f6ecca35700 1 -- 172.21.15.45:0/1759718919 shutdown_connections 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.508+0000 7f6ecca35700 1 -- 172.21.15.45:0/1759718919 wait complete. 2023-09-23T09:39:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.509+0000 7f6ecca35700 1 Processor -- start 2023-09-23T09:39:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.509+0000 7f6ecca35700 1 -- start start 2023-09-23T09:39:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ecca35700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80ff600 0x7f6ec8107690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ecca35700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec80fffc0 0x7f6ec8107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ecca35700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec81054c0 0x7f6ec8108110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ecca35700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec81a97a0 con 0x7f6ec80fffc0 2023-09-23T09:39:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ecca35700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec806a870 con 0x7f6ec80ff600 2023-09-23T09:39:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ecca35700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec806aa30 con 0x7f6ec81054c0 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ec659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80ff600 0x7f6ec8107690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ec659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80ff600 0x7f6ec8107690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57088/0 (socket says 172.21.15.45:57088) 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ec659c700 1 -- 172.21.15.45:0/1212880604 learned_addr learned my addr 172.21.15.45:0/1212880604 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec80fffc0 0x7f6ec8107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.510+0000 7f6ec6d9d700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec81054c0 0x7f6ec8108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6ec659c700 1 -- 172.21.15.45:0/1212880604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec81054c0 msgr2=0x7f6ec8108110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6ec659c700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec81054c0 0x7f6ec8108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6ec659c700 1 -- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec80fffc0 msgr2=0x7f6ec8107bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6ec659c700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec80fffc0 0x7f6ec8107bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6ec659c700 1 -- 172.21.15.45:0/1212880604 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80095f0 con 0x7f6ec80ff600 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6ec659c700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80ff600 0x7f6ec8107690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000ba50 tx=0x7f6ec000be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6eb77fe700 1 -- 172.21.15.45:0/1212880604 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000cab0 con 0x7f6ec80ff600 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6eb77fe700 1 -- 172.21.15.45:0/1212880604 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ec000cc10 con 0x7f6ec80ff600 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6eb77fe700 1 -- 172.21.15.45:0/1212880604 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00075d0 con 0x7f6ec80ff600 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.511+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec806ad10 con 0x7f6ec80ff600 2023-09-23T09:39:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.512+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec806b1d0 con 0x7f6ec80ff600 2023-09-23T09:39:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.515+0000 7f6ed8c1fb80 1 Processor -- start 2023-09-23T09:39:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.516+0000 7f6ed8c1fb80 1 -- start start 2023-09-23T09:39:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.516+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba137307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.516+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba1373b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.516+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba1373dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.516+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba13699460 con 0x55ba13730d20 2023-09-23T09:39:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.516+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1368ac20 con 0x55ba13730400 2023-09-23T09:39:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.516+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba13688540 con 0x55ba1373bbd0 2023-09-23T09:39:33.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6eb77fe700 1 -- 172.21.15.45:0/1212880604 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f6ec0014440 con 0x7f6ec80ff600 2023-09-23T09:39:33.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba1373dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba1373dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37850/0 (socket says 172.21.15.45:37850) 2023-09-23T09:39:33.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/1602693042 learned_addr learned my addr 172.21.15.45:0/1602693042 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:33.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec659c700 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba1373b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/1602693042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 msgr2=0x55ba137307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba137307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/1602693042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 msgr2=0x55ba1373b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba1373b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/1602693042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba136e68d0 con 0x55ba1373bbd0 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.517+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba1373dfc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009990 tx=0x7f6eb8009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae011bdeddabebc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.518+0000 7f6eb4ff9700 1 -- 172.21.15.45:0/1602693042 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800b9d0 con 0x55ba1373bbd0 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.518+0000 7f6eb4ff9700 1 -- 172.21.15.45:0/1602693042 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb801e040 con 0x55ba1373bbd0 2023-09-23T09:39:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.518+0000 7f6eb4ff9700 1 -- 172.21.15.45:0/1602693042 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800f460 con 0x55ba1373bbd0 2023-09-23T09:39:33.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.518+0000 7f6eb77fe700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6eb00a18e0 0x7f6eb00a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.518+0000 7f6eb77fe700 1 -- 172.21.15.45:0/1212880604 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6ec0006e40 con 0x7f6ec80ff600 2023-09-23T09:39:33.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.519+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/1602693042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 msgr2=0x55ba1373dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.519+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba1373dfc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009990 tx=0x7f6eb8009ee0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/1602693042 shutdown_connections 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba1373dfc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba137307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/1602693042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba1373b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/1602693042 >> 172.21.15.45:0/1602693042 conn(0x55ba136a5260 msgr2=0x55ba1372c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ec6d9d700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6eb00a18e0 0x7f6eb00a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/1602693042 shutdown_connections 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/1602693042 wait complete. 2023-09-23T09:39:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.521+0000 7f6ec6d9d700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6eb00a18e0 0x7f6eb00a3da0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005d90 tx=0x7f6ebc005d20 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 -- start start 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 Processor -- start 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba1373b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba13738860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba13738e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba13683e30 con 0x55ba13730d20 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba13620710 con 0x55ba13730400 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba136203b0 con 0x55ba1373bbd0 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba13738860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ed8c1fb80 10 client.0 _ll_register_callbacks cb 0x55ba1347d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba13738860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35754/0 (socket says 172.21.15.45:35754) 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ec659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba1373b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/3965362889 learned_addr learned my addr 172.21.15.45:0/3965362889 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.522+0000 7f6ec6d9d700 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba13738e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/3965362889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 msgr2=0x55ba13738e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba13738e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/3965362889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 msgr2=0x55ba1373b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba1373b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/3965362889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec810cbf0 con 0x55ba13730d20 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ec6d9d700 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba13738e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba13738860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8005920 tx=0x7f6eb8005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69628461e03a1eac server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6eadffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6eadffb700 1 -- 172.21.15.45:0/3965362889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8031070 con 0x55ba13730d20 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6eadffb700 1 -- 172.21.15.45:0/3965362889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6eb800ba00 con 0x55ba13730d20 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba13724250 con 0x55ba13730d20 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6eadffb700 1 -- 172.21.15.45:0/3965362889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb8003890 con 0x55ba13730d20 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ed8c1fb80 10 client.16596 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.523+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ba134aede0 con 0x55ba13730d20 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6eadffb700 1 -- 172.21.15.45:0/3965362889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba13724250 con 0x55ba13730d20 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6eadffb700 1 -- 172.21.15.45:0/3965362889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6eb8034030 con 0x55ba13730d20 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6ed8c1fb80 10 client.16596 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6ed8c1fb80 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6ed8c1fb80 20 client.16596 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1374b180 0x55ba1374d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6ed8c1fb80 4 client.16596 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:39:33.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba1367ce10 con 0x55ba1374b180 2023-09-23T09:39:33.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.524+0000 7f6ec659c700 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1374b180 0x55ba1374d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.525+0000 7f6ec659c700 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1374b180 0x55ba1374d5a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af001a2f23705f48 server_cookie=dc51f33f9f2d82fc in_seq=0 out_seq=0 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.525+0000 7f6eadffb700 10 client.16596 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.525+0000 7f6eadffb700 1 -- 172.21.15.45:0/3965362889 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ba1367ce10 con 0x55ba1374b180 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.525+0000 7f6eadffb700 10 client.16596 handle_command_reply: tid=0 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.525+0000 7f6ed8c1fb80 1 client.16596 shutdown 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.526+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 msgr2=0x55ba13738860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.526+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba13738860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8005920 tx=0x7f6eb8005a90 comp rx=0 tx=0).stop 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 shutdown_connections 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1374b180 0x55ba1374d5a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba1373bbd0 0x55ba13738e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6eadffb700 0 client.16596 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:33.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba1373b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/3965362889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730d20 0x55ba13738860 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 >> 172.21.15.45:0/3965362889 conn(0x55ba136a5260 msgr2=0x55ba1372c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:33.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 shutdown_connections 2023-09-23T09:39:33.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/3965362889 wait complete. 2023-09-23T09:39:33.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.527+0000 7f6ed8c1fb80 20 client.16596 trim_cache size 0 max 16384 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.548+0000 7f6ed8c1fb80 1 Processor -- start 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.548+0000 7f6ed8c1fb80 1 -- start start 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.548+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730400 0x55ba13733350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.548+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13733b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.548+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba1373bbd0 0x55ba137340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.548+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba13504920 con 0x55ba13730400 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba13731ec0 con 0x55ba1373bbd0 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1372b580 con 0x55ba13730d20 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13733b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13733b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37868/0 (socket says 172.21.15.45:37868) 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/2354193581 learned_addr learned my addr 172.21.15.45:0/2354193581 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec659c700 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730400 0x55ba13733350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec6d9d700 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba1373bbd0 0x55ba137340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/2354193581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba1373bbd0 msgr2=0x55ba137340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba1373bbd0 0x55ba137340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/2354193581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730400 msgr2=0x55ba13733350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730400 0x55ba13733350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/2354193581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba13497c60 con 0x55ba13730d20 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13733b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80098e0 tx=0x7f6eb800bfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e923353168697616 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6eb57fa700 1 -- 172.21.15.45:0/2354193581 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8031070 con 0x55ba13730d20 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6eb57fa700 1 -- 172.21.15.45:0/2354193581 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb800f920 con 0x55ba13730d20 2023-09-23T09:39:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.549+0000 7f6eb57fa700 1 -- 172.21.15.45:0/2354193581 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8003af0 con 0x55ba13730d20 2023-09-23T09:39:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/2354193581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 msgr2=0x55ba13733b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13733b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80098e0 tx=0x7f6eb800bfb0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/2354193581 shutdown_connections 2023-09-23T09:39:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba1373bbd0 0x55ba137340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba13730400 0x55ba13733350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/2354193581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13733b90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/2354193581 >> 172.21.15.45:0/2354193581 conn(0x55ba136a5260 msgr2=0x55ba13746e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/2354193581 shutdown_connections 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.550+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/2354193581 wait complete. 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 -- start start 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 Processor -- start 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba137406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13740be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba1373bbd0 0x55ba13741120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1379df70 con 0x55ba1373bbd0 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1379e0b0 con 0x55ba13730400 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1379e1f0 con 0x55ba13730d20 2023-09-23T09:39:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ed8c1fb80 10 client.0 _ll_register_callbacks cb 0x7f6ec00cfe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba1373bbd0 0x55ba13741120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13740be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba1373bbd0 0x55ba13741120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35770/0 (socket says 172.21.15.45:35770) 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec659c700 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba137406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec6d9d700 1 -- 172.21.15.45:0/4244040801 learned_addr learned my addr 172.21.15.45:0/4244040801 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 msgr2=0x55ba137406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba137406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/4244040801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba1373bbd0 msgr2=0x55ba13741120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba1373bbd0 0x55ba13741120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec5d9b700 1 -- 172.21.15.45:0/4244040801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba134aede0 con 0x55ba13730d20 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.551+0000 7f6ec6d9d700 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba1373bbd0 0x55ba13741120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.552+0000 7f6ec5d9b700 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13740be0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ba13739350 tx=0x7f6eb8009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80458e802c09001e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.552+0000 7f6eadffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.552+0000 7f6eadffb700 1 -- 172.21.15.45:0/4244040801 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8031070 con 0x55ba13730d20 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.552+0000 7f6eadffb700 1 -- 172.21.15.45:0/4244040801 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6eb8000e40 con 0x55ba13730d20 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.552+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eb801e040 con 0x55ba13730d20 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.552+0000 7f6eadffb700 1 -- 172.21.15.45:0/4244040801 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb8009290 con 0x55ba13730d20 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 10 client.26281 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ba134d5250 con 0x55ba13730d20 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6eadffb700 1 -- 172.21.15.45:0/4244040801 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb801e040 con 0x55ba13730d20 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6eadffb700 1 -- 172.21.15.45:0/4244040801 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6eb8044070 con 0x55ba13730d20 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 10 client.26281 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 20 client.26281 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1379f540 0x55ba137a1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 4 client.26281 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:39:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.553+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba1367cb30 con 0x55ba1379f540 2023-09-23T09:39:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6ec659c700 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1379f540 0x55ba137a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6eadffb700 10 client.26281 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6ec659c700 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1379f540 0x55ba137a1960 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91fc4bce4def46d server_cookie=3db62e9d90851646 in_seq=0 out_seq=0 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6eadffb700 1 -- 172.21.15.45:0/4244040801 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ba1367cb30 con 0x55ba1379f540 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6eadffb700 10 client.26281 handle_command_reply: tid=0 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6ed8c1fb80 1 client.26281 shutdown 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 msgr2=0x55ba13740be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.554+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13740be0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ba13739350 tx=0x7f6eb8009f50 comp rx=0 tx=0).stop 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 shutdown_connections 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55ba1379f540 0x55ba137a1960 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba1373bbd0 0x55ba13741120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba13730400 0x55ba137406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6eadffb700 0 client.26281 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 --2- 172.21.15.45:0/4244040801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba13730d20 0x55ba13740be0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 >> 172.21.15.45:0/4244040801 conn(0x55ba136a5260 msgr2=0x55ba13733b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 shutdown_connections 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 1 -- 172.21.15.45:0/4244040801 wait complete. 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ed8c1fb80 20 client.26281 trim_cache size 0 max 16384 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:39:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:39:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:39:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:39:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:39:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:39:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:39:33.562 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:39:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6eb00a18e0 msgr2=0x7f6eb00a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6eb00a18e0 0x7f6eb00a3da0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005d90 tx=0x7f6ebc005d20 comp rx=0 tx=0).stop 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80ff600 msgr2=0x7f6ec8107690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80ff600 0x7f6ec8107690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000ba50 tx=0x7f6ec000be10 comp rx=0 tx=0).stop 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 shutdown_connections 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec81054c0 0x7f6ec8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec80ff600 0x7f6ec8107690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f6eb00a18e0 0x7f6eb00a3da0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 --2- 172.21.15.45:0/1212880604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec80fffc0 0x7f6ec8107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 >> 172.21.15.45:0/1212880604 conn(0x7f6ec8074f20 msgr2=0x7f6ec80fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 shutdown_connections 2023-09-23T09:39:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:39:33.555+0000 7f6ecca35700 1 -- 172.21.15.45:0/1212880604 wait complete. 2023-09-23T09:39:33.571 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:39:33.744 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:33 smithi045 ceph-mon[98627]: pgmap v315: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 755 op/s 2023-09-23T09:39:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 -- 172.21.15.45:0/380172903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498108740 msgr2=0x7f0498113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 --2- 172.21.15.45:0/380172903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498108740 0x7f0498113a30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0488009990 tx=0x7f0488009ca0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 -- 172.21.15.45:0/380172903 shutdown_connections 2023-09-23T09:39:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 --2- 172.21.15.45:0/380172903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0498113f70 0x7f0498116360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 --2- 172.21.15.45:0/380172903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498107e20 0x7f0498108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 --2- 172.21.15.45:0/380172903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498108740 0x7f0498113a30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 -- 172.21.15.45:0/380172903 >> 172.21.15.45:0/380172903 conn(0x7f049806cee0 msgr2=0x7f049806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 -- 172.21.15.45:0/380172903 shutdown_connections 2023-09-23T09:39:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.015+0000 7f049efd0700 1 -- 172.21.15.45:0/380172903 wait complete. 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 Processor -- start 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 -- start start 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498107e20 0x7f04981a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0498108740 0x7f04981a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498113f70 0x7f049819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049819fb50 con 0x7f0498113f70 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049819fcc0 con 0x7f0498108740 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.016+0000 7f049efd0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049819fe80 con 0x7f0498107e20 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f0497fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0498108740 0x7f04981a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f0497fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0498108740 0x7f04981a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57130/0 (socket says 172.21.15.45:57130) 2023-09-23T09:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f0497fff700 1 -- 172.21.15.45:0/74119034 learned_addr learned my addr 172.21.15.45:0/74119034 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049cd6c700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498107e20 0x7f04981a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049cd6c700 1 -- 172.21.15.45:0/74119034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0498108740 msgr2=0x7f04981a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049cd6c700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0498108740 0x7f04981a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049cd6c700 1 -- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498113f70 msgr2=0x7f049819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049d56d700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498113f70 0x7f049819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049cd6c700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498113f70 0x7f049819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049cd6c700 1 -- 172.21.15.45:0/74119034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880095f0 con 0x7f0498107e20 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049d56d700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498113f70 0x7f049819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049cd6c700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498107e20 0x7f04981a5680 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f048c00e9f0 tx=0x7f048c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f0495ffb700 1 -- 172.21.15.45:0/74119034 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00eee0 con 0x7f0498107e20 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f0495ffb700 1 -- 172.21.15.45:0/74119034 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f048c00cbd0 con 0x7f0498107e20 2023-09-23T09:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04981a0160 con 0x7f0498107e20 2023-09-23T09:39:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.017+0000 7f0495ffb700 1 -- 172.21.15.45:0/74119034 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c018680 con 0x7f0498107e20 2023-09-23T09:39:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.018+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04981ac670 con 0x7f0498107e20 2023-09-23T09:39:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.022+0000 7f04ab1bab80 1 Processor -- start 2023-09-23T09:39:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f04ab1bab80 1 -- start start 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb26320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb311d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 0x55b68bb33b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68ba8f400 con 0x55b68bb25f40 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68ba80bc0 con 0x55b68bb31710 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68ba7e4e0 con 0x55b68bb26860 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f0495ffb700 1 -- 172.21.15.45:0/74119034 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f048c010430 con 0x7f0498107e20 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f049d56d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb26320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f049d56d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb26320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35786/0 (socket says 172.21.15.45:35786) 2023-09-23T09:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f049d56d700 1 -- 172.21.15.45:0/4200520700 learned_addr learned my addr 172.21.15.45:0/4200520700 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f049cd6c700 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.023+0000 7f049d56d700 1 -- 172.21.15.45:0/4200520700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 msgr2=0x55b68bb311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f049d56d700 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb311d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f049d56d700 1 -- 172.21.15.45:0/4200520700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 msgr2=0x55b68bb33b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f049d56d700 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 0x55b68bb33b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f049d56d700 1 -- 172.21.15.45:0/4200520700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b68badc870 con 0x55b68bb25f40 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f049cd6c700 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f0495ffb700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f04800a1710 0x7f04800a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f049d56d700 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb26320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f04900093d0 tx=0x7f0490009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6674a6494f0bd6f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f049cd6c700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f04800a1710 0x7f04800a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f0495ffb700 1 -- 172.21.15.45:0/74119034 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f048c014070 con 0x7f0498107e20 2023-09-23T09:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f047effd700 1 -- 172.21.15.45:0/4200520700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490014610 con 0x55b68bb25f40 2023-09-23T09:39:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f047effd700 1 -- 172.21.15.45:0/4200520700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f049000f050 con 0x55b68bb25f40 2023-09-23T09:39:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.024+0000 7f047effd700 1 -- 172.21.15.45:0/4200520700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04900139a0 con 0x55b68bb25f40 2023-09-23T09:39:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.025+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4200520700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 msgr2=0x55b68bb26320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.025+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb26320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f04900093d0 tx=0x7f0490009920 comp rx=0 tx=0).stop 2023-09-23T09:39:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f049cd6c700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f04800a1710 0x7f04800a3bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f048c0cae10 tx=0x7f048c0d3060 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4200520700 shutdown_connections 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 0x55b68bb33b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb26320 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4200520700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4200520700 >> 172.21.15.45:0/4200520700 conn(0x55b68ba9b200 msgr2=0x55b68bb223c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4200520700 shutdown_connections 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.026+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4200520700 wait complete. 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.027+0000 7f04ab1bab80 1 -- start start 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.027+0000 7f04ab1bab80 1 Processor -- start 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb25f40 0x55b68bb31090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb26860 0x55b68bb2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb31710 0x55b68bb2e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68ba79dd0 con 0x55b68bb26860 2023-09-23T09:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68ba166b0 con 0x55b68bb25f40 2023-09-23T09:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68ba16350 con 0x55b68bb31710 2023-09-23T09:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f04ab1bab80 10 client.0 _ll_register_callbacks cb 0x55b68b8737a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb25f40 0x55b68bb31090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb25f40 0x55b68bb31090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57142/0 (socket says 172.21.15.45:57142) 2023-09-23T09:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 -- 172.21.15.45:0/4054417396 learned_addr learned my addr 172.21.15.45:0/4054417396 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb31710 0x55b68bb2e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f0497fff700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb26860 0x55b68bb2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 -- 172.21.15.45:0/4054417396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb25f40 msgr2=0x55b68bb31090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb25f40 0x55b68bb31090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 -- 172.21.15.45:0/4054417396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb26860 msgr2=0x55b68bb2e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb26860 0x55b68bb2e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f049d56d700 1 -- 172.21.15.45:0/4054417396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04981ac670 con 0x55b68bb31710 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.028+0000 7f0497fff700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb26860 0x55b68bb2e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f049d56d700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb31710 0x55b68bb2e940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0490013520 tx=0x7f0490000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9b178c9be1fb82e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f0477fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f0477fff700 1 -- 172.21.15.45:0/4054417396 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490007ea0 con 0x55b68bb31710 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b68bb1a090 con 0x55b68bb31710 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f0477fff700 1 -- 172.21.15.45:0/4054417396 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0490004ad0 con 0x55b68bb31710 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f0477fff700 1 -- 172.21.15.45:0/4054417396 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0490004c90 con 0x55b68bb31710 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f04ab1bab80 10 client.26293 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f04900090b0 con 0x55b68bb31710 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f0477fff700 1 -- 172.21.15.45:0/4054417396 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b68bb1a090 con 0x55b68bb31710 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f0477fff700 1 -- 172.21.15.45:0/4054417396 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f04900051f0 con 0x55b68bb31710 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f04ab1bab80 10 client.26293 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f04ab1bab80 10 client.26293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.029+0000 7f04ab1bab80 20 client.26293 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.030+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb45520 0x55b68bb47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.030+0000 7f04ab1bab80 4 client.26293 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.030+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b68ba72db0 con 0x55b68bb45520 2023-09-23T09:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.030+0000 7f0497fff700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb45520 0x55b68bb47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.032+0000 7f0477fff700 10 client.26293 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.032+0000 7f0497fff700 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb45520 0x55b68bb47940 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b0d3ac7f70cd1e server_cookie=590a8987a16dbb3f in_seq=0 out_seq=0 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.032+0000 7f0477fff700 1 -- 172.21.15.45:0/4054417396 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b68ba72db0 con 0x55b68bb45520 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.032+0000 7f0477fff700 10 client.26293 handle_command_reply: tid=0 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.032+0000 7f04ab1bab80 1 client.26293 shutdown 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb31710 msgr2=0x55b68bb2e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb31710 0x55b68bb2e940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0490013520 tx=0x7f0490000f40 comp rx=0 tx=0).stop 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 shutdown_connections 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb45520 0x55b68bb47940 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb31710 0x55b68bb2e940 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb25f40 0x55b68bb31090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f0477fff700 0 client.26293 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/4054417396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb26860 0x55b68bb2e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 >> 172.21.15.45:0/4054417396 conn(0x55b68ba9b200 msgr2=0x55b68bb21f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 shutdown_connections 2023-09-23T09:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 1 -- 172.21.15.45:0/4054417396 wait complete. 2023-09-23T09:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.033+0000 7f04ab1bab80 20 client.26293 trim_cache size 0 max 16384 2023-09-23T09:39:34.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:33 smithi098 ceph-mon[102386]: pgmap v315: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 755 op/s 2023-09-23T09:39:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.054+0000 7f04ab1bab80 1 Processor -- start 2023-09-23T09:39:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.054+0000 7f04ab1bab80 1 -- start start 2023-09-23T09:39:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb25f40 0x55b68bb2dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb26860 0x55b68bb295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb31710 0x55b68bb29b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68b8fa8c0 con 0x55b68bb31710 2023-09-23T09:39:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68bb1eba0 con 0x55b68bb26860 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68bb19f60 con 0x55b68bb25f40 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb25f40 0x55b68bb2dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb31710 0x55b68bb29b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb25f40 0x55b68bb2dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:37906/0 (socket says 172.21.15.45:37906) 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 -- 172.21.15.45:0/1611608652 learned_addr learned my addr 172.21.15.45:0/1611608652 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f0497fff700 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb26860 0x55b68bb295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 -- 172.21.15.45:0/1611608652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb26860 msgr2=0x55b68bb295f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb26860 0x55b68bb295f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 -- 172.21.15.45:0/1611608652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb31710 msgr2=0x55b68bb29b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.055+0000 7f049d56d700 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb31710 0x55b68bb29b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.056+0000 7f049d56d700 1 -- 172.21.15.45:0/1611608652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b68b8a4d80 con 0x55b68bb25f40 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.056+0000 7f049d56d700 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb25f40 0x55b68bb2dad0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0490002230 tx=0x7f0490000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f561012691a4d676 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.056+0000 7f047f7fe700 1 -- 172.21.15.45:0/1611608652 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490005770 con 0x55b68bb25f40 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.056+0000 7f047f7fe700 1 -- 172.21.15.45:0/1611608652 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f049003b070 con 0x55b68bb25f40 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.056+0000 7f047f7fe700 1 -- 172.21.15.45:0/1611608652 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490004a70 con 0x55b68bb25f40 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.057+0000 7f04ab1bab80 1 -- 172.21.15.45:0/1611608652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb25f40 msgr2=0x55b68bb2dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.057+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb25f40 0x55b68bb2dad0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0490002230 tx=0x7f0490000f40 comp rx=0 tx=0).stop 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.057+0000 7f04ab1bab80 1 -- 172.21.15.45:0/1611608652 shutdown_connections 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.057+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb31710 0x55b68bb29b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.057+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb25f40 0x55b68bb2dad0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.057+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/1611608652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb26860 0x55b68bb295f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.057+0000 7f04ab1bab80 1 -- 172.21.15.45:0/1611608652 >> 172.21.15.45:0/1611608652 conn(0x55b68ba9b200 msgr2=0x55b68bb41810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.068+0000 7f04ab1bab80 1 -- 172.21.15.45:0/1611608652 shutdown_connections 2023-09-23T09:39:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.090+0000 7f04ab1bab80 1 -- 172.21.15.45:0/1611608652 wait complete. 2023-09-23T09:39:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.090+0000 7f04ab1bab80 1 -- start start 2023-09-23T09:39:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.090+0000 7f04ab1bab80 1 Processor -- start 2023-09-23T09:39:34.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:33 smithi123 ceph-mon[103577]: pgmap v315: 97 pgs: 97 active+clean; 6.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 755 op/s 2023-09-23T09:39:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.103+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb38230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.103+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb40a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.103+0000 7f04ab1bab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 0x55b68bb40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.103+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68bb415e0 con 0x55b68bb25f40 2023-09-23T09:39:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.103+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68bb364b0 con 0x55b68bb31710 2023-09-23T09:39:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.103+0000 7f04ab1bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68bb36670 con 0x55b68bb26860 2023-09-23T09:39:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.103+0000 7f04ab1bab80 10 client.0 _ll_register_callbacks cb 0x7f04900139f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.111+0000 7f049d56d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 0x55b68bb40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.111+0000 7f049d56d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb38230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.111+0000 7f049d56d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb38230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35806/0 (socket says 172.21.15.45:35806) 2023-09-23T09:39:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.111+0000 7f049d56d700 1 -- 172.21.15.45:0/3081863752 learned_addr learned my addr 172.21.15.45:0/3081863752 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.111+0000 7f0497fff700 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb40a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.119+0000 7f0497fff700 1 -- 172.21.15.45:0/3081863752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 msgr2=0x55b68bb40f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.119+0000 7f0497fff700 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 0x55b68bb40f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.119+0000 7f0497fff700 1 -- 172.21.15.45:0/3081863752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 msgr2=0x55b68bb38230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.119+0000 7f0497fff700 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb38230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.119+0000 7f0497fff700 1 -- 172.21.15.45:0/3081863752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049003f040 con 0x55b68bb26860 2023-09-23T09:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.128+0000 7f0497fff700 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb40a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55b68bb2f8c0 tx=0x7f0488026550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b27741b61d1be4dc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.128+0000 7f0477fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.130+0000 7f0477fff700 1 -- 172.21.15.45:0/3081863752 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048801d040 con 0x55b68bb26860 2023-09-23T09:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.130+0000 7f0477fff700 1 -- 172.21.15.45:0/3081863752 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0488026c00 con 0x55b68bb26860 2023-09-23T09:39:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.132+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f049000f050 con 0x55b68bb26860 2023-09-23T09:39:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.132+0000 7f0477fff700 1 -- 172.21.15.45:0/3081863752 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0488026dc0 con 0x55b68bb26860 2023-09-23T09:39:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.132+0000 7f04ab1bab80 10 client.26299 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.132+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55b68b88dc00 con 0x55b68bb26860 2023-09-23T09:39:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f0477fff700 1 -- 172.21.15.45:0/3081863752 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f049000f050 con 0x55b68bb26860 2023-09-23T09:39:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f0477fff700 1 -- 172.21.15.45:0/3081863752 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f04880049c0 con 0x55b68bb26860 2023-09-23T09:39:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f04ab1bab80 10 client.26299 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f04ab1bab80 10 client.26299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f04ab1bab80 20 client.26299 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb95c20 0x55b68bb98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f04ab1bab80 4 client.26299 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:39:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b68ba72ad0 con 0x55b68bb95c20 2023-09-23T09:39:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.133+0000 7f049d56d700 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb95c20 0x55b68bb98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.134+0000 7f0477fff700 10 client.26299 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.134+0000 7f049d56d700 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb95c20 0x55b68bb98040 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be06b34a5bce3a2 server_cookie=a344fa6a7e1796f3 in_seq=0 out_seq=0 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f0477fff700 1 -- 172.21.15.45:0/3081863752 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b68ba72ad0 con 0x55b68bb95c20 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f0477fff700 10 client.26299 handle_command_reply: tid=0 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 client.26299 shutdown 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 msgr2=0x55b68bb40a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb40a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55b68bb2f8c0 tx=0x7f0488026550 comp rx=0 tx=0).stop 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 shutdown_connections 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b68bb95c20 0x55b68bb98040 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68bb31710 0x55b68bb40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f0477fff700 0 client.26299 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68bb25f40 0x55b68bb38230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 --2- 172.21.15.45:0/3081863752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b68bb26860 0x55b68bb40a20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 >> 172.21.15.45:0/3081863752 conn(0x55b68ba9b200 msgr2=0x55b68bb295f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 shutdown_connections 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 1 -- 172.21.15.45:0/3081863752 wait complete. 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.173+0000 7f04ab1bab80 20 client.26299 trim_cache size 0 max 16384 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:39:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:39:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:39:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f04800a1710 msgr2=0x7f04800a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f04800a1710 0x7f04800a3bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f048c0cae10 tx=0x7f048c0d3060 comp rx=0 tx=0).stop 2023-09-23T09:39:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498107e20 msgr2=0x7f04981a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498107e20 0x7f04981a5680 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f048c00e9f0 tx=0x7f048c00ed00 comp rx=0 tx=0).stop 2023-09-23T09:39:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 shutdown_connections 2023-09-23T09:39:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f04800a1710 0x7f04800a3bd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0498113f70 0x7f049819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498107e20 0x7f04981a5680 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 --2- 172.21.15.45:0/74119034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0498108740 0x7f04981a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 >> 172.21.15.45:0/74119034 conn(0x7f049806cee0 msgr2=0x7f049806f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 shutdown_connections 2023-09-23T09:39:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:34.174+0000 7f049efd0700 1 -- 172.21.15.45:0/74119034 wait complete. 2023-09-23T09:39:36.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:35 smithi098 ceph-mon[102386]: pgmap v316: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2023-09-23T09:39:36.089 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:35 smithi045 ceph-mon[98627]: pgmap v316: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2023-09-23T09:39:36.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:35 smithi123 ceph-mon[103577]: pgmap v316: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2023-09-23T09:39:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 590.0 s, 1604.0 tps, lat 62.586 ms stddev 65.935 2023-09-23T09:39:38.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:37 smithi098 ceph-mon[102386]: pgmap v317: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 998 op/s 2023-09-23T09:39:38.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:37 smithi045 ceph-mon[98627]: pgmap v317: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 998 op/s 2023-09-23T09:39:38.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:37 smithi123 ceph-mon[103577]: pgmap v317: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 998 op/s 2023-09-23T09:39:40.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:39 smithi098 ceph-mon[102386]: pgmap v318: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-23T09:39:40.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:39 smithi045 ceph-mon[98627]: pgmap v318: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-23T09:39:40.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:39 smithi123 ceph-mon[103577]: pgmap v318: 97 pgs: 97 active+clean; 6.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-23T09:39:41.886 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 595.0 s, 1023.8 tps, lat 96.875 ms stddev 236.172 2023-09-23T09:39:42.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:41 smithi045 ceph-mon[98627]: pgmap v319: 97 pgs: 97 active+clean; 6.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.26k op/s 2023-09-23T09:39:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:41 smithi123 ceph-mon[103577]: pgmap v319: 97 pgs: 97 active+clean; 6.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.26k op/s 2023-09-23T09:39:42.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:41 smithi098 ceph-mon[102386]: pgmap v319: 97 pgs: 97 active+clean; 6.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.26k op/s 2023-09-23T09:39:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:43 smithi045 ceph-mon[98627]: pgmap v320: 97 pgs: 97 active+clean; 6.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 914 op/s 2023-09-23T09:39:44.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:43 smithi123 ceph-mon[103577]: pgmap v320: 97 pgs: 97 active+clean; 6.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 914 op/s 2023-09-23T09:39:44.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:43 smithi098 ceph-mon[102386]: pgmap v320: 97 pgs: 97 active+clean; 6.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 914 op/s 2023-09-23T09:39:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:45 smithi045 ceph-mon[98627]: pgmap v321: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2023-09-23T09:39:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:45 smithi123 ceph-mon[103577]: pgmap v321: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2023-09-23T09:39:46.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:45 smithi098 ceph-mon[102386]: pgmap v321: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2023-09-23T09:39:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 600.0 s, 1762.5 tps, lat 57.122 ms stddev 70.022 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 -- 172.21.15.45:0/301277665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8107000 msgr2=0x7fc6f81073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/301277665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8107000 0x7fc6f81073e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40099c0 tx=0x7fc6f4009cd0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 -- 172.21.15.45:0/301277665 shutdown_connections 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/301277665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6f8069b20 0x7fc6f810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/301277665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8107000 0x7fc6f81073e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/301277665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f8069160 0x7fc6f80695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 -- 172.21.15.45:0/301277665 >> 172.21.15.45:0/301277665 conn(0x7fc6f8076c00 msgr2=0x7fc6f8077010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 -- 172.21.15.45:0/301277665 shutdown_connections 2023-09-23T09:39:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.312+0000 7fc6ff76a700 1 -- 172.21.15.45:0/301277665 wait complete. 2023-09-23T09:39:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 Processor -- start 2023-09-23T09:39:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 -- start start 2023-09-23T09:39:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8069160 0x7fc6f81a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6f8069b20 0x7fc6f81a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f8107000 0x7fc6f819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f819dbd0 con 0x7fc6f8069160 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f819dd40 con 0x7fc6f8069b20 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.313+0000 7fc6ff76a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f819df00 con 0x7fc6f8107000 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fd506700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8069160 0x7fc6f81a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fd506700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8069160 0x7fc6f81a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35820/0 (socket says 172.21.15.45:35820) 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fd506700 1 -- 172.21.15.45:0/809358603 learned_addr learned my addr 172.21.15.45:0/809358603 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fcd05700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6f8069b20 0x7fc6f81a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f8107000 0x7fc6f819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fcd05700 1 -- 172.21.15.45:0/809358603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f8107000 msgr2=0x7fc6f819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fcd05700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f8107000 0x7fc6f819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fcd05700 1 -- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8069160 msgr2=0x7fc6f81a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fcd05700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8069160 0x7fc6f81a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fcd05700 1 -- 172.21.15.45:0/809358603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40095f0 con 0x7fc6f8069b20 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fd506700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8069160 0x7fc6f81a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6fcd05700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6f8069b20 0x7fc6f81a3c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800da20 tx=0x7fc6e800dd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.314+0000 7fc6ee7fc700 1 -- 172.21.15.45:0/809358603 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800c970 con 0x7fc6f8069b20 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.315+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f819e1e0 con 0x7fc6f8069b20 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.315+0000 7fc6ee7fc700 1 -- 172.21.15.45:0/809358603 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e8015410 con 0x7fc6f8069b20 2023-09-23T09:39:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.315+0000 7fc6ee7fc700 1 -- 172.21.15.45:0/809358603 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8014610 con 0x7fc6f8069b20 2023-09-23T09:39:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.315+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f81aa6f0 con 0x7fc6f8069b20 2023-09-23T09:39:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.319+0000 7fc70b954b80 1 Processor -- start 2023-09-23T09:39:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.319+0000 7fc70b954b80 1 -- start start 2023-09-23T09:39:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72ba860 0x55d1d72c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72c5710 0x55d1d72c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc70b954b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d7223400 con 0x55d1d72b9f40 2023-09-23T09:39:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc70b954b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d7214bc0 con 0x55d1d72ba860 2023-09-23T09:39:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc70b954b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d72124e0 con 0x55d1d72c5710 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fd506700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72ba860 0x55d1d72c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35826/0 (socket says 172.21.15.45:35826) 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fd506700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72ba860 0x55d1d72c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57176/0 (socket says 172.21.15.45:57176) 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 -- 172.21.15.45:0/2377081289 learned_addr learned my addr 172.21.15.45:0/2377081289 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 -- 172.21.15.45:0/2377081289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72c5710 msgr2=0x55d1d72c7b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/2377081289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72c5710 0x55d1d72c7b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 -- 172.21.15.45:0/2377081289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72ba860 msgr2=0x55d1d72c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/2377081289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72ba860 0x55d1d72c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.320+0000 7fc6fdd07700 1 -- 172.21.15.45:0/2377081289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1d7270870 con 0x55d1d72b9f40 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.321+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/2377081289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0003f40 tx=0x7fc6f0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5580b32e12e5b399 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.321+0000 7fc6ee7fc700 1 -- 172.21.15.45:0/809358603 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fc6e8015780 con 0x7fc6f8069b20 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.321+0000 7fc6e37fe700 1 -- 172.21.15.45:0/2377081289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0019070 con 0x55d1d72b9f40 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.321+0000 7fc6e37fe700 1 -- 172.21.15.45:0/2377081289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6f000f040 con 0x55d1d72b9f40 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.321+0000 7fc6e37fe700 1 -- 172.21.15.45:0/2377081289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0007960 con 0x55d1d72b9f40 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc6ee7fc700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6e40a1710 0x7fc6e40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 -- 172.21.15.45:0/2377081289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 msgr2=0x55d1d72ba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 --2- 172.21.15.45:0/2377081289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0003f40 tx=0x7fc6f0009ba0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc6ee7fc700 1 -- 172.21.15.45:0/809358603 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc6e80cc890 con 0x7fc6f8069b20 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc6fd506700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6e40a1710 0x7fc6e40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 -- 172.21.15.45:0/2377081289 shutdown_connections 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 --2- 172.21.15.45:0/2377081289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72c5710 0x55d1d72c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 --2- 172.21.15.45:0/2377081289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 --2- 172.21.15.45:0/2377081289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72ba860 0x55d1d72c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 -- 172.21.15.45:0/2377081289 >> 172.21.15.45:0/2377081289 conn(0x55d1d722f200 msgr2=0x55d1d72b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 -- 172.21.15.45:0/2377081289 shutdown_connections 2023-09-23T09:39:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 -- 172.21.15.45:0/2377081289 wait complete. 2023-09-23T09:39:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc6fd506700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6e40a1710 0x7fc6e40a3bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003760 tx=0x7fc6f40058e0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.322+0000 7fc70b954b80 1 -- start start 2023-09-23T09:39:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 1 Processor -- start 2023-09-23T09:39:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72c2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d720ddd0 con 0x55d1d72b9f40 2023-09-23T09:39:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d71aa6b0 con 0x55d1d72c5710 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d71aa350 con 0x55d1d72ba860 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc70b954b80 10 client.0 _ll_register_callbacks cb 0x55d1d70077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc6fdd07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc6fdd07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57186/0 (socket says 172.21.15.45:57186) 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.323+0000 7fc6fdd07700 1 -- 172.21.15.45:0/3704404913 learned_addr learned my addr 172.21.15.45:0/3704404913 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.324+0000 7fc6fdd07700 1 -- 172.21.15.45:0/3704404913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 msgr2=0x55d1d72c2450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.324+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72c2450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.324+0000 7fc6fdd07700 1 -- 172.21.15.45:0/3704404913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 msgr2=0x55d1d72c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.324+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.324+0000 7fc6fdd07700 1 -- 172.21.15.45:0/3704404913 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f0003b70 con 0x55d1d72c5710 2023-09-23T09:39:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.324+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72c2b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0003f10 tx=0x7fc6f0007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff9f0ea648b7bfd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.324+0000 7fc6e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:39:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.325+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/3704404913 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0019040 con 0x55d1d72c5710 2023-09-23T09:39:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.325+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/3704404913 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc6f0004ad0 con 0x55d1d72c5710 2023-09-23T09:39:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.325+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1d72ae090 con 0x55d1d72c5710 2023-09-23T09:39:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.325+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/3704404913 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f00048f0 con 0x55d1d72c5710 2023-09-23T09:39:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.325+0000 7fc70b954b80 10 client.26312 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.325+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55d1d7038d80 con 0x55d1d72c5710 2023-09-23T09:39:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/3704404913 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1d72ae090 con 0x55d1d72c5710 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/3704404913 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc6f001f450 con 0x55d1d72c5710 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc70b954b80 10 client.26312 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc70b954b80 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc70b954b80 20 client.26312 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d72d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc70b954b80 4 client.26312 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1d7206db0 con 0x55d1d72d5230 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.326+0000 7fc6fd506700 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d72d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.327+0000 7fc6e0ff9700 10 client.26312 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.327+0000 7fc6fd506700 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d72d7650 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7250092e01c61e server_cookie=ee160a8e99c9bf0b in_seq=0 out_seq=0 2023-09-23T09:39:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/3704404913 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1d7206db0 con 0x55d1d72d5230 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc6e0ff9700 10 client.26312 handle_command_reply: tid=0 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 client.26312 shutdown 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 msgr2=0x55d1d72c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72c2b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0003f10 tx=0x7fc6f0007860 comp rx=0 tx=0).stop 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 shutdown_connections 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d72d7650 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72c2b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc6e0ff9700 0 client.26312 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3704404913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 >> 172.21.15.45:0/3704404913 conn(0x55d1d722f200 msgr2=0x55d1d72b5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 shutdown_connections 2023-09-23T09:39:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 1 -- 172.21.15.45:0/3704404913 wait complete. 2023-09-23T09:39:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.328+0000 7fc70b954b80 20 client.26312 trim_cache size 0 max 16384 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.345+0000 7fc70b954b80 1 Processor -- start 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc70b954b80 1 -- start start 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc70b954b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d708e8c0 con 0x55d1d72b9f40 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc70b954b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d72adf90 con 0x55d1d72c5710 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc70b954b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d72c3450 con 0x55d1d72ba860 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35840/0 (socket says 172.21.15.45:35840) 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 -- 172.21.15.45:0/3353109033 learned_addr learned my addr 172.21.15.45:0/3353109033 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fdd07700 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fcd05700 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 -- 172.21.15.45:0/3353109033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 msgr2=0x55d1d72d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 -- 172.21.15.45:0/3353109033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 msgr2=0x55d1d72bd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72bd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.346+0000 7fc6fd506700 1 -- 172.21.15.45:0/3353109033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1d7021c00 con 0x55d1d72b9f40 2023-09-23T09:39:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.347+0000 7fc6fd506700 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006010 tx=0x7fc6f401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c85f378aa8574b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.347+0000 7fc6e3fff700 1 -- 172.21.15.45:0/3353109033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4005420 con 0x55d1d72b9f40 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.347+0000 7fc6e3fff700 1 -- 172.21.15.45:0/3353109033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6f401e040 con 0x55d1d72b9f40 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.347+0000 7fc6e3fff700 1 -- 172.21.15.45:0/3353109033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4003fc0 con 0x55d1d72b9f40 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.348+0000 7fc70b954b80 1 -- 172.21.15.45:0/3353109033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 msgr2=0x55d1d72ba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.348+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006010 tx=0x7fc6f401a050 comp rx=0 tx=0).stop 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.348+0000 7fc70b954b80 1 -- 172.21.15.45:0/3353109033 shutdown_connections 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.348+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d72bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.348+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72ba320 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.348+0000 7fc70b954b80 1 --2- 172.21.15.45:0/3353109033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.348+0000 7fc70b954b80 1 -- 172.21.15.45:0/3353109033 >> 172.21.15.45:0/3353109033 conn(0x55d1d722f200 msgr2=0x55d1d72d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.349+0000 7fc70b954b80 1 -- 172.21.15.45:0/3353109033 shutdown_connections 2023-09-23T09:39:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.350+0000 7fc70b954b80 1 -- 172.21.15.45:0/3353109033 wait complete. 2023-09-23T09:39:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.350+0000 7fc70b954b80 1 -- start start 2023-09-23T09:39:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.350+0000 7fc70b954b80 1 Processor -- start 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.351+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.351+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.351+0000 7fc70b954b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d7328060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.351+0000 7fc70b954b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d72cc1f0 con 0x55d1d72b9f40 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.351+0000 7fc70b954b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d73285a0 con 0x55d1d72c5710 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.351+0000 7fc70b954b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d73396f0 con 0x55d1d72ba860 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.351+0000 7fc70b954b80 10 client.0 _ll_register_callbacks cb 0x7fc6f4009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fdd07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d7328060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fdd07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d7328060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57196/0 (socket says 172.21.15.45:57196) 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fdd07700 1 -- 172.21.15.45:0/4130433030 learned_addr learned my addr 172.21.15.45:0/4130433030 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:39:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fd506700 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fd506700 1 -- 172.21.15.45:0/4130433030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 msgr2=0x55d1d72cb9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fd506700 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72cb9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fd506700 1 -- 172.21.15.45:0/4130433030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 msgr2=0x55d1d7328060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fd506700 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d7328060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fd506700 1 -- 172.21.15.45:0/4130433030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f810e710 con 0x55d1d72b9f40 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.352+0000 7fc6fd506700 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72cb480 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005a60 tx=0x7fc6f40038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6879e1e095a81 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.355+0000 7fc6e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.355+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/4130433030 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4005710 con 0x55d1d72b9f40 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.355+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/4130433030 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc6f4033050 con 0x55d1d72b9f40 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.356+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6f000f040 con 0x55d1d72b9f40 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.356+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/4130433030 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f4033210 con 0x55d1d72b9f40 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.356+0000 7fc70b954b80 10 client.16632 fetch_fsmap learned FSMap version 45 2023-09-23T09:39:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.356+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55d1d705f1f0 con 0x55d1d72b9f40 2023-09-23T09:39:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/4130433030 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6f000f040 con 0x55d1d72b9f40 2023-09-23T09:39:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/4130433030 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc6f406e070 con 0x55d1d72b9f40 2023-09-23T09:39:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc70b954b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 45 2023-09-23T09:39:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc70b954b80 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:39:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc70b954b80 20 client.16632 populate_metadata read hostname 'smithi045' 2023-09-23T09:39:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc70b954b80 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d732a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:39:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc70b954b80 4 client.16632 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:39:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d1d7206ad0 con 0x55d1d72d5230 2023-09-23T09:39:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.357+0000 7fc6fcd05700 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d732a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:39:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.360+0000 7fc6fcd05700 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d732a870 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6fe34a05a74038 server_cookie=637c946581f48422 in_seq=0 out_seq=0 2023-09-23T09:39:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.360+0000 7fc6e0ff9700 10 client.16632 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.397+0000 7fc6e0ff9700 1 -- 172.21.15.45:0/4130433030 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d1d7206ad0 con 0x55d1d72d5230 2023-09-23T09:39:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.397+0000 7fc6e0ff9700 10 client.16632 handle_command_reply: tid=0 2023-09-23T09:39:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.397+0000 7fc70b954b80 1 client.16632 shutdown 2023-09-23T09:39:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.397+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 msgr2=0x55d1d72cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.397+0000 7fc70b954b80 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72cb480 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005a60 tx=0x7fc6f40038c0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.398+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 shutdown_connections 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.398+0000 7fc70b954b80 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55d1d72d5230 0x55d1d732a870 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.398+0000 7fc70b954b80 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d1d72c5710 0x55d1d7328060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.398+0000 7fc6e0ff9700 0 client.16632 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.398+0000 7fc70b954b80 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1d72b9f40 0x55d1d72cb480 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.398+0000 7fc70b954b80 1 --2- 172.21.15.45:0/4130433030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1d72ba860 0x55d1d72cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.398+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 >> 172.21.15.45:0/4130433030 conn(0x55d1d722f200 msgr2=0x55d1d72d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.399+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 shutdown_connections 2023-09-23T09:39:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.400+0000 7fc70b954b80 1 -- 172.21.15.45:0/4130433030 wait complete. 2023-09-23T09:39:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.400+0000 7fc70b954b80 20 client.16632 trim_cache size 0 max 16384 2023-09-23T09:39:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:39:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:39:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:39:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:39:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6e40a1710 msgr2=0x7fc6e40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6e40a1710 0x7fc6e40a3bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003760 tx=0x7fc6f40058e0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6f8069b20 msgr2=0x7fc6f81a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6f8069b20 0x7fc6f81a3c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800da20 tx=0x7fc6e800dd30 comp rx=0 tx=0).stop 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 shutdown_connections 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f8107000 0x7fc6f819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc6e40a1710 0x7fc6e40a3bd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6f8069160 0x7fc6f81a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 --2- 172.21.15.45:0/809358603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6f8069b20 0x7fc6f81a3c40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 >> 172.21.15.45:0/809358603 conn(0x7fc6f8076c00 msgr2=0x7fc6f80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 shutdown_connections 2023-09-23T09:39:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:39:47.401+0000 7fc6ff76a700 1 -- 172.21.15.45:0/809358603 wait complete. 2023-09-23T09:39:48.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:47 smithi045 ceph-mon[98627]: pgmap v322: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:39:48.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:47 smithi123 ceph-mon[103577]: pgmap v322: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:39:48.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:47 smithi098 ceph-mon[102386]: pgmap v322: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:39:49.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:48 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:39:49.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:48 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:39:49.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:48 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:39:49.820 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:49 smithi045 ceph-mon[98627]: pgmap v323: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-09-23T09:39:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:49 smithi123 ceph-mon[103577]: pgmap v323: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-09-23T09:39:50.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:49 smithi098 ceph-mon[102386]: pgmap v323: 97 pgs: 97 active+clean; 6.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-09-23T09:39:51.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 605.0 s, 1681.0 tps, lat 58.773 ms stddev 70.572 2023-09-23T09:39:52.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:51 smithi045 ceph-mon[98627]: pgmap v324: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 21 MiB/s wr, 1.47k op/s 2023-09-23T09:39:52.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:51 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:52.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:51 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:52.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:51 smithi123 ceph-mon[103577]: pgmap v324: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 21 MiB/s wr, 1.47k op/s 2023-09-23T09:39:52.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:51 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:52.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:51 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:52.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:51 smithi098 ceph-mon[102386]: pgmap v324: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 21 MiB/s wr, 1.47k op/s 2023-09-23T09:39:52.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:51 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:52.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:51 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:52 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:52 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:53.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:53.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:52 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:54.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: pgmap v325: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi045 to 197.2M 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: Unable to set osd_memory_target on smithi045 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:39:54.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:39:54.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: pgmap v325: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:39:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi045 to 197.2M 2023-09-23T09:39:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: Unable to set osd_memory_target on smithi045 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-23T09:39:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:39:54.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:39:54.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: pgmap v325: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-09-23T09:39:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi045 to 197.2M 2023-09-23T09:39:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: Unable to set osd_memory_target on smithi045 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-23T09:39:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:39:54.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:39:55.299 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:54 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-23T09:39:55.299 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:54 smithi098 ceph-mon[102386]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:39:55.299 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:54 smithi098 ceph-mon[102386]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-23T09:39:55.299 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:54 smithi098 ceph-mon[102386]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:39:55.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:54 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-23T09:39:55.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:54 smithi045 ceph-mon[98627]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:39:55.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:54 smithi045 ceph-mon[98627]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-23T09:39:55.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:54 smithi045 ceph-mon[98627]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:39:55.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:54 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-23T09:39:55.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:54 smithi123 ceph-mon[103577]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:39:55.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:54 smithi123 ceph-mon[103577]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-23T09:39:55.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:54 smithi123 ceph-mon[103577]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-23T09:39:56.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:55 smithi098 ceph-mon[102386]: pgmap v326: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 20 MiB/s wr, 1.46k op/s 2023-09-23T09:39:56.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:55 smithi045 ceph-mon[98627]: pgmap v326: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 20 MiB/s wr, 1.46k op/s 2023-09-23T09:39:56.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:55 smithi123 ceph-mon[103577]: pgmap v326: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 20 MiB/s wr, 1.46k op/s 2023-09-23T09:39:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 610.0 s, 1807.4 tps, lat 55.559 ms stddev 62.825 2023-09-23T09:39:57.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:56 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:39:57.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:56 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:39:57.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:56 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:39:58.300 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:57 smithi098 ceph-mon[102386]: pgmap v327: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 10 MiB/s wr, 573 op/s 2023-09-23T09:39:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:57 smithi045 ceph-mon[98627]: pgmap v327: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 10 MiB/s wr, 573 op/s 2023-09-23T09:39:58.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:57 smithi123 ceph-mon[103577]: pgmap v327: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 10 MiB/s wr, 573 op/s 2023-09-23T09:39:59.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:39:58 smithi098 ceph-mon[102386]: pgmap v328: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 585 op/s 2023-09-23T09:39:59.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:39:58 smithi045 ceph-mon[98627]: pgmap v328: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 585 op/s 2023-09-23T09:39:59.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:39:58 smithi123 ceph-mon[103577]: pgmap v328: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 585 op/s 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 -- 172.21.15.45:0/3373531435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0105050 msgr2=0x7f92b010cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 --2- 172.21.15.45:0/3373531435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0105050 0x7f92b010cbd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b0d0 tx=0x7f92a800b3e0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 -- 172.21.15.45:0/3373531435 shutdown_connections 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 --2- 172.21.15.45:0/3373531435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0105050 0x7f92b010cbd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 --2- 172.21.15.45:0/3373531435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b0100160 0x7f92b0100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 --2- 172.21.15.45:0/3373531435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b0100b10 0x7f92b0104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 -- 172.21.15.45:0/3373531435 >> 172.21.15.45:0/3373531435 conn(0x7f92b0076840 msgr2=0x7f92b0076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 -- 172.21.15.45:0/3373531435 shutdown_connections 2023-09-23T09:40:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.538+0000 7f92b84dd700 1 -- 172.21.15.45:0/3373531435 wait complete. 2023-09-23T09:40:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 Processor -- start 2023-09-23T09:40:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 -- start start 2023-09-23T09:40:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0100160 0x7f92b00724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b0105050 0x7f92b0072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b006b350 0x7f92b006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b006bfb0 con 0x7f92b0100160 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b006c120 con 0x7f92b006b350 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b84dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b0070070 con 0x7f92b0105050 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b6279700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0100160 0x7f92b00724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.539+0000 7f92b6a7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b006b350 0x7f92b006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b6279700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0100160 0x7f92b00724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35858/0 (socket says 172.21.15.45:35858) 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b6279700 1 -- 172.21.15.45:0/1168591805 learned_addr learned my addr 172.21.15.45:0/1168591805 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b5a78700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b0105050 0x7f92b0072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b5a78700 1 -- 172.21.15.45:0/1168591805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b006b350 msgr2=0x7f92b006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b5a78700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b006b350 0x7f92b006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b5a78700 1 -- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0100160 msgr2=0x7f92b00724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b5a78700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0100160 0x7f92b00724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b5a78700 1 -- 172.21.15.45:0/1168591805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a8009d00 con 0x7f92b0105050 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b5a78700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b0105050 0x7f92b0072a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e9c0 tx=0x7f92a000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92a77fe700 1 -- 172.21.15.45:0/1168591805 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a000c960 con 0x7f92b0105050 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b0070350 con 0x7f92b0105050 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92a77fe700 1 -- 172.21.15.45:0/1168591805 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a0010430 con 0x7f92b0105050 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92a77fe700 1 -- 172.21.15.45:0/1168591805 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0018680 con 0x7f92b0105050 2023-09-23T09:40:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.540+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b0070860 con 0x7f92b0105050 2023-09-23T09:40:00.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:00 smithi098 ceph-mon[102386]: overall HEALTH_OK 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.543+0000 7f92a77fe700 1 -- 172.21.15.45:0/1168591805 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f92a00105a0 con 0x7f92b0105050 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.543+0000 7f92a77fe700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f929c0a1710 0x7f929c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.543+0000 7f92a77fe700 1 -- 172.21.15.45:0/1168591805 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f92a0014070 con 0x7f92b0105050 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.543+0000 7f92b6279700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f929c0a1710 0x7f929c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.544+0000 7f92b6279700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f929c0a1710 0x7f929c0a3bd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009d20 tx=0x7f92ac010040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.544+0000 7f92c46c7b80 1 Processor -- start 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.544+0000 7f92c46c7b80 1 -- start start 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.544+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc51fdf40 0x556bc51fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.544+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fe860 0x556bc52091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc5209710 0x556bc520bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc5167400 con 0x556bc51fe860 2023-09-23T09:40:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc5158bc0 con 0x556bc5209710 2023-09-23T09:40:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc51564e0 con 0x556bc51fdf40 2023-09-23T09:40:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fe860 0x556bc52091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fe860 0x556bc52091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35868/0 (socket says 172.21.15.45:35868) 2023-09-23T09:40:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 -- 172.21.15.45:0/2901806923 learned_addr learned my addr 172.21.15.45:0/2901806923 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b6279700 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc51fdf40 0x556bc51fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b6a7a700 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc5209710 0x556bc520bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 -- 172.21.15.45:0/2901806923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc51fdf40 msgr2=0x556bc51fe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc51fdf40 0x556bc51fe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 -- 172.21.15.45:0/2901806923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc5209710 msgr2=0x556bc520bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc5209710 0x556bc520bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 -- 172.21.15.45:0/2901806923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bc51b4870 con 0x556bc51fe860 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92b5a78700 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fe860 0x556bc52091d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f92a0096e40 tx=0x7f92a009a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ff3c18438b0078 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.545+0000 7f92a4ff9700 1 -- 172.21.15.45:0/2901806923 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a009a3d0 con 0x556bc51fe860 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.546+0000 7f92a4ff9700 1 -- 172.21.15.45:0/2901806923 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a009e520 con 0x556bc51fe860 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.546+0000 7f92a4ff9700 1 -- 172.21.15.45:0/2901806923 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a009a890 con 0x556bc51fe860 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.546+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2901806923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fe860 msgr2=0x556bc52091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.546+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fe860 0x556bc52091d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f92a0096e40 tx=0x7f92a009a2d0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.546+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2901806923 shutdown_connections 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.546+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc5209710 0x556bc520bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.546+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc51fdf40 0x556bc51fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2901806923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fe860 0x556bc52091d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2901806923 >> 172.21.15.45:0/2901806923 conn(0x556bc5173200 msgr2=0x556bc51fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2901806923 shutdown_connections 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2901806923 wait complete. 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 -- start start 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 Processor -- start 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc5209170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc52063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc5206b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc5151dd0 con 0x556bc51fdf40 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc50ee6b0 con 0x556bc51fe860 2023-09-23T09:40:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc50ee350 con 0x556bc5209710 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.547+0000 7f92c46c7b80 10 client.0 _ll_register_callbacks cb 0x556bc4f4b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6279700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc5209170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b5a78700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc52063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6a7a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc5206b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6279700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc5209170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35872/0 (socket says 172.21.15.45:35872) 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6279700 1 -- 172.21.15.45:0/2449949403 learned_addr learned my addr 172.21.15.45:0/2449949403 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6a7a700 1 -- 172.21.15.45:0/2449949403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 msgr2=0x556bc52063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6a7a700 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc52063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6a7a700 1 -- 172.21.15.45:0/2449949403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 msgr2=0x556bc5209170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6a7a700 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc5209170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6a7a700 1 -- 172.21.15.45:0/2449949403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b0070860 con 0x556bc5209710 2023-09-23T09:40:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6279700 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc5209170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92b6a7a700 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc5206b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b770 tx=0x7f92a8008f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718336f7751046b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f9299ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f9299ffb700 1 -- 172.21.15.45:0/2449949403 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8024070 con 0x556bc5209710 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bc51f2090 con 0x556bc5209710 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f9299ffb700 1 -- 172.21.15.45:0/2449949403 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f92a800bcb0 con 0x556bc5209710 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f9299ffb700 1 -- 172.21.15.45:0/2449949403 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92a800e040 con 0x556bc5209710 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92c46c7b80 10 client.26320 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.548+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556bc4f7cd80 con 0x556bc5209710 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f9299ffb700 1 -- 172.21.15.45:0/2449949403 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a8035080 con 0x556bc5209710 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f9299ffb700 1 -- 172.21.15.45:0/2449949403 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f92a800b100 con 0x556bc5209710 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f92c46c7b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f92c46c7b80 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f92c46c7b80 20 client.26320 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc5219240 0x556bc521b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f92c46c7b80 4 client.26320 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bc514adb0 con 0x556bc5219240 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.549+0000 7f92b6279700 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc5219240 0x556bc521b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.550+0000 7f9299ffb700 10 client.26320 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.550+0000 7f92b6279700 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc5219240 0x556bc521b660 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b6394159fce1a9 server_cookie=504287f73ed00b52 in_seq=0 out_seq=0 2023-09-23T09:40:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.550+0000 7f9299ffb700 1 -- 172.21.15.45:0/2449949403 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556bc514adb0 con 0x556bc5219240 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.550+0000 7f9299ffb700 10 client.26320 handle_command_reply: tid=0 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.550+0000 7f92c46c7b80 1 client.26320 shutdown 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 msgr2=0x556bc5206b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc5206b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b770 tx=0x7f92a8008f80 comp rx=0 tx=0).stop 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 shutdown_connections 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc5219240 0x556bc521b660 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc5206b30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f9299ffb700 0 client.26320 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc5209170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/2449949403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc52063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 >> 172.21.15.45:0/2449949403 conn(0x556bc5173200 msgr2=0x556bc51f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 shutdown_connections 2023-09-23T09:40:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 1 -- 172.21.15.45:0/2449949403 wait complete. 2023-09-23T09:40:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.551+0000 7f92c46c7b80 20 client.26320 trim_cache size 0 max 16384 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.569+0000 7f92c46c7b80 1 Processor -- start 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.569+0000 7f92c46c7b80 1 -- start start 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5208d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc52036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc4fd28c0 con 0x556bc51fdf40 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc52022f0 con 0x556bc51fe860 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc51feed0 con 0x556bc5209710 2023-09-23T09:40:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35880/0 (socket says 172.21.15.45:35880) 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 -- 172.21.15.45:0/938170740 learned_addr learned my addr 172.21.15.45:0/938170740 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b5a78700 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5208d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6a7a700 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc52036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 -- 172.21.15.45:0/938170740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 msgr2=0x556bc52036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc52036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 -- 172.21.15.45:0/938170740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 msgr2=0x556bc5208d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5208d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 -- 172.21.15.45:0/938170740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bc4f65c00 con 0x556bc51fdf40 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.570+0000 7f92b6279700 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520bb00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00eb50 tx=0x7f92ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850a55ebbef6ac95 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92a57fa700 1 -- 172.21.15.45:0/938170740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac02b070 con 0x556bc51fdf40 2023-09-23T09:40:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92a57fa700 1 -- 172.21.15.45:0/938170740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ac014040 con 0x556bc51fdf40 2023-09-23T09:40:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92a57fa700 1 -- 172.21.15.45:0/938170740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0186b0 con 0x556bc51fdf40 2023-09-23T09:40:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 -- 172.21.15.45:0/938170740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 msgr2=0x556bc520bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520bb00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00eb50 tx=0x7f92ac016000 comp rx=0 tx=0).stop 2023-09-23T09:40:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 -- 172.21.15.45:0/938170740 shutdown_connections 2023-09-23T09:40:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc52036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520bb00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/938170740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5208d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 -- 172.21.15.45:0/938170740 >> 172.21.15.45:0/938170740 conn(0x556bc5173200 msgr2=0x556bc5201710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.571+0000 7f92c46c7b80 1 -- 172.21.15.45:0/938170740 shutdown_connections 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 -- 172.21.15.45:0/938170740 wait complete. 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 -- start start 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 Processor -- start 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5210160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc520e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc520e810 con 0x556bc51fdf40 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc520e980 con 0x556bc51fe860 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc520eb40 con 0x556bc5209710 2023-09-23T09:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92c46c7b80 10 client.0 _ll_register_callbacks cb 0x7f92a8005a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92b5a78700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5210160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92b6279700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92b5a78700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5210160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57238/0 (socket says 172.21.15.45:57238) 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92b5a78700 1 -- 172.21.15.45:0/1633036986 learned_addr learned my addr 172.21.15.45:0/1633036986 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.572+0000 7f92b6a7a700 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc520e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92b5a78700 1 -- 172.21.15.45:0/1633036986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 msgr2=0x556bc520e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92b5a78700 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc520e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92b5a78700 1 -- 172.21.15.45:0/1633036986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 msgr2=0x556bc520fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92b5a78700 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92b5a78700 1 -- 172.21.15.45:0/1633036986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac01a780 con 0x556bc51fe860 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92b5a78700 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5210160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f92a00cd470 tx=0x7f92a00cb110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c28a80e11ed4f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f9299ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f9299ffb700 1 -- 172.21.15.45:0/1633036986 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a00a2760 con 0x556bc51fe860 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92a009e520 con 0x556bc51fe860 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f9299ffb700 1 -- 172.21.15.45:0/1633036986 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f92a00998c0 con 0x556bc51fe860 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f9299ffb700 1 -- 172.21.15.45:0/1633036986 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92a0099a80 con 0x556bc51fe860 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92c46c7b80 10 client.26336 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556bc4fa31f0 con 0x556bc51fe860 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.573+0000 7f9299ffb700 1 -- 172.21.15.45:0/1633036986 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a009e520 con 0x556bc51fe860 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f9299ffb700 1 -- 172.21.15.45:0/1633036986 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f92a00a2360 con 0x556bc51fe860 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92c46c7b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92c46c7b80 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92c46c7b80 20 client.26336 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc526e030 0x556bc5270450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92c46c7b80 4 client.26336 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556bc514aad0 con 0x556bc526e030 2023-09-23T09:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92b6279700 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc526e030 0x556bc5270450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f9299ffb700 10 client.26336 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.574+0000 7f92b6279700 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc526e030 0x556bc5270450 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fc0b5f8cb07f93 server_cookie=8b649f87c8bace31 in_seq=0 out_seq=0 2023-09-23T09:40:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:00 smithi045 ceph-mon[98627]: overall HEALTH_OK 2023-09-23T09:40:00.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:00 smithi123 ceph-mon[103577]: overall HEALTH_OK 2023-09-23T09:40:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.599+0000 7f9299ffb700 1 -- 172.21.15.45:0/1633036986 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556bc514aad0 con 0x556bc526e030 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.599+0000 7f9299ffb700 10 client.26336 handle_command_reply: tid=0 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.599+0000 7f92c46c7b80 1 client.26336 shutdown 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.599+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 msgr2=0x556bc5210160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.599+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5210160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f92a00cd470 tx=0x7f92a00cb110 comp rx=0 tx=0).stop 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.599+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 shutdown_connections 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556bc526e030 0x556bc5270450 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc5209710 0x556bc520e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f9299ffb700 0 client.26336 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556bc51fdf40 0x556bc520fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 1 --2- 172.21.15.45:0/1633036986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bc51fe860 0x556bc5210160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 >> 172.21.15.45:0/1633036986 conn(0x556bc5173200 msgr2=0x556bc51fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 shutdown_connections 2023-09-23T09:40:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 1 -- 172.21.15.45:0/1633036986 wait complete. 2023-09-23T09:40:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92c46c7b80 20 client.26336 trim_cache size 0 max 16384 2023-09-23T09:40:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:40:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:40:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:40:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:40:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f929c0a1710 msgr2=0x7f929c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f929c0a1710 0x7f929c0a3bd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009d20 tx=0x7f92ac010040 comp rx=0 tx=0).stop 2023-09-23T09:40:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b0105050 msgr2=0x7f92b0072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b0105050 0x7f92b0072a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e9c0 tx=0x7f92a000ecd0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 shutdown_connections 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b006b350 0x7f92b006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f929c0a1710 0x7f929c0a3bd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0100160 0x7f92b00724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 --2- 172.21.15.45:0/1168591805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b0105050 0x7f92b0072a30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.600+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 >> 172.21.15.45:0/1168591805 conn(0x7f92b0076840 msgr2=0x7f92b00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.601+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 shutdown_connections 2023-09-23T09:40:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:00.601+0000 7f92b84dd700 1 -- 172.21.15.45:0/1168591805 wait complete. 2023-09-23T09:40:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 615.0 s, 1355.1 tps, lat 73.972 ms stddev 137.549 2023-09-23T09:40:02.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:01 smithi098 ceph-mon[102386]: pgmap v329: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 15 MiB/s wr, 827 op/s 2023-09-23T09:40:02.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:01 smithi045 ceph-mon[98627]: pgmap v329: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 15 MiB/s wr, 827 op/s 2023-09-23T09:40:02.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:01 smithi123 ceph-mon[103577]: pgmap v329: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 15 MiB/s wr, 827 op/s 2023-09-23T09:40:03.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:40:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 -- 172.21.15.45:0/1132672587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 msgr2=0x7f3d101002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 --2- 172.21.15.45:0/1132672587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d101002b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009990 tx=0x7f3d00009ca0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 -- 172.21.15.45:0/1132672587 shutdown_connections 2023-09-23T09:40:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 --2- 172.21.15.45:0/1132672587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d10104ff0 0x7f3d1010cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 --2- 172.21.15.45:0/1132672587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d101002b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 --2- 172.21.15.45:0/1132672587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d10100880 0x7f3d101048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 -- 172.21.15.45:0/1132672587 >> 172.21.15.45:0/1132672587 conn(0x7f3d100fb770 msgr2=0x7f3d100fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 -- 172.21.15.45:0/1132672587 shutdown_connections 2023-09-23T09:40:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.796+0000 7f3d14976700 1 -- 172.21.15.45:0/1132672587 wait complete. 2023-09-23T09:40:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 Processor -- start 2023-09-23T09:40:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 -- start start 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d10199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d10104ff0 0x7f3d1019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1010b7d0 0x7f3d1010bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1010c390 con 0x7f3d100ffed0 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1010c500 con 0x7f3d1010b7d0 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d14976700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1019d860 con 0x7f3d10104ff0 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d0e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d10199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d0dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d10104ff0 0x7f3d1019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d0e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d10199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35900/0 (socket says 172.21.15.45:35900) 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d0e59c700 1 -- 172.21.15.45:0/3238372857 learned_addr learned my addr 172.21.15.45:0/3238372857 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.798+0000 7f3d0ed9d700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1010b7d0 0x7f3d1010bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0e59c700 1 -- 172.21.15.45:0/3238372857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d10104ff0 msgr2=0x7f3d1019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d10104ff0 0x7f3d1019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0e59c700 1 -- 172.21.15.45:0/3238372857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1010b7d0 msgr2=0x7f3d1010bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1010b7d0 0x7f3d1010bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0e59c700 1 -- 172.21.15.45:0/3238372857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d08009c40 con 0x7f3d100ffed0 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d10104ff0 0x7f3d1019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0ed9d700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1010b7d0 0x7f3d1010bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d10199b60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3d00005770 tx=0x7f3d000057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3cff7fe700 1 -- 172.21.15.45:0/3238372857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0001d070 con 0x7f3d100ffed0 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3cff7fe700 1 -- 172.21.15.45:0/3238372857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d00022470 con 0x7f3d100ffed0 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d000095f0 con 0x7f3d100ffed0 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3cff7fe700 1 -- 172.21.15.45:0/3238372857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00017650 con 0x7f3d100ffed0 2023-09-23T09:40:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.799+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1019dcc0 con 0x7f3d100ffed0 2023-09-23T09:40:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.803+0000 7f3cff7fe700 1 -- 172.21.15.45:0/3238372857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f3d0000fb40 con 0x7f3d100ffed0 2023-09-23T09:40:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.804+0000 7f3cff7fe700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3cf80a1710 0x7f3cf80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.804+0000 7f3cff7fe700 1 -- 172.21.15.45:0/3238372857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d000ce780 con 0x7f3d100ffed0 2023-09-23T09:40:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.805+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3cf80a1710 0x7f3cf80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.805+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3cf80a1710 0x7f3cf80a3bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d10196430 tx=0x7f3d04009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:03.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.806+0000 7f3d20b60b80 1 Processor -- start 2023-09-23T09:40:03.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.806+0000 7f3d20b60b80 1 -- start start 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9a3400 0x55562b9a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b90c460 con 0x55562b9aebd0 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b8fdc20 con 0x55562b9a3400 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b8fb540 con 0x55562b9a3d20 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9a3400 0x55562b9a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38016/0 (socket says 172.21.15.45:38016) 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/3463391490 learned_addr learned my addr 172.21.15.45:0/3463391490 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0ed9d700 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/3463391490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9a3400 msgr2=0x55562b9a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9a3400 0x55562b9a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/3463391490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 msgr2=0x55562b9b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/3463391490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55562b9598d0 con 0x55562b9a3d20 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0ed9d700 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.807+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9ae690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004390 tx=0x7f3d04014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6578b8261b597fc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3cfcff9700 1 -- 172.21.15.45:0/3463391490 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04003d10 con 0x55562b9a3d20 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3cfcff9700 1 -- 172.21.15.45:0/3463391490 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d04003e70 con 0x55562b9a3d20 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3cfcff9700 1 -- 172.21.15.45:0/3463391490 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0401e850 con 0x55562b9a3d20 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 -- 172.21.15.45:0/3463391490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 msgr2=0x55562b9ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9ae690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004390 tx=0x7f3d04014000 comp rx=0 tx=0).stop 2023-09-23T09:40:03.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 -- 172.21.15.45:0/3463391490 shutdown_connections 2023-09-23T09:40:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9a3400 0x55562b9a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/3463391490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9ae690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 -- 172.21.15.45:0/3463391490 >> 172.21.15.45:0/3463391490 conn(0x55562b918260 msgr2=0x55562b99f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 -- 172.21.15.45:0/3463391490 shutdown_connections 2023-09-23T09:40:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.808+0000 7f3d20b60b80 1 -- 172.21.15.45:0/3463391490 wait complete. 2023-09-23T09:40:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 -- start start 2023-09-23T09:40:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 Processor -- start 2023-09-23T09:40:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3400 0x55562b9ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9a3d20 0x55562b9abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9aebd0 0x55562b9ac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b8f6e30 con 0x55562b9a3d20 2023-09-23T09:40:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b893710 con 0x55562b9aebd0 2023-09-23T09:40:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b8933b0 con 0x55562b9a3400 2023-09-23T09:40:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d20b60b80 10 client.0 _ll_register_callbacks cb 0x55562b6f0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d0e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3400 0x55562b9ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d0e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3400 0x55562b9ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38020/0 (socket says 172.21.15.45:38020) 2023-09-23T09:40:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.809+0000 7f3d0e59c700 1 -- 172.21.15.45:0/285471400 learned_addr learned my addr 172.21.15.45:0/285471400 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0ed9d700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9aebd0 0x55562b9ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9a3d20 0x55562b9abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0e59c700 1 -- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9aebd0 msgr2=0x55562b9ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9aebd0 0x55562b9ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0e59c700 1 -- 172.21.15.45:0/285471400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9a3d20 msgr2=0x55562b9abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9a3d20 0x55562b9abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0e59c700 1 -- 172.21.15.45:0/285471400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d04017070 con 0x55562b9a3400 2023-09-23T09:40:03.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9a3d20 0x55562b9abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3400 0x55562b9ab9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3d00097c40 tx=0x7f3d0009b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f66dafc15ca8a158 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3cf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.810+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/285471400 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0009bec0 con 0x55562b9a3400 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.811+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/285471400 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d0009b740 con 0x55562b9a3400 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.811+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55562b997250 con 0x55562b9a3400 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.811+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/285471400 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d0009f890 con 0x55562b9a3400 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 10 client.26338 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55562b721de0 con 0x55562b9a3400 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/285471400 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55562b997250 con 0x55562b9a3400 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/285471400 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3d0009b9a0 con 0x55562b9a3400 2023-09-23T09:40:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 10 client.26338 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 10 client.26338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 20 client.26338 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562b9be490 0x55562b9c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 4 client.26338 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:40:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.812+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55562b8efe10 con 0x55562b9be490 2023-09-23T09:40:03.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.821+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562b9be490 0x55562b9c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.855+0000 7f3cf5ffb700 10 client.26338 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.855+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562b9be490 0x55562b9c08b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cddd00b199406b76 server_cookie=d92d1e8cddb06b04 in_seq=0 out_seq=0 2023-09-23T09:40:03.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.869+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/285471400 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55562b8efe10 con 0x55562b9be490 2023-09-23T09:40:03.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.869+0000 7f3cf5ffb700 10 client.26338 handle_command_reply: tid=0 2023-09-23T09:40:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.872+0000 7f3d20b60b80 1 client.26338 shutdown 2023-09-23T09:40:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.873+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3400 msgr2=0x55562b9ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.873+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3400 0x55562b9ab9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3d00097c40 tx=0x7f3d0009b330 comp rx=0 tx=0).stop 2023-09-23T09:40:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.874+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 shutdown_connections 2023-09-23T09:40:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.874+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562b9be490 0x55562b9c08b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.874+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9aebd0 0x55562b9ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.874+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3400 0x55562b9ab9f0 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3d00097c40 tx=0x7f3d0009b330 comp rx=0 tx=0).stop 2023-09-23T09:40:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.874+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/285471400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9a3d20 0x55562b9abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.874+0000 7f3cf5ffb700 0 client.26338 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.874+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 >> 172.21.15.45:0/285471400 conn(0x55562b918260 msgr2=0x55562b99f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:03.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.887+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 shutdown_connections 2023-09-23T09:40:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.896+0000 7f3d20b60b80 1 -- 172.21.15.45:0/285471400 wait complete. 2023-09-23T09:40:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.903+0000 7f3d20b60b80 20 client.26338 trim_cache size 0 max 16384 2023-09-23T09:40:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.932+0000 7f3d20b60b80 1 Processor -- start 2023-09-23T09:40:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.932+0000 7f3d20b60b80 1 -- start start 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b777920 con 0x55562b9aebd0 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b9a6890 con 0x55562b9ae310 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562b8efe40 con 0x55562b9a3d20 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35924/0 (socket says 172.21.15.45:35924) 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/2998833361 learned_addr learned my addr 172.21.15.45:0/2998833361 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0ed9d700 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/2998833361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 msgr2=0x55562b9a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/2998833361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 msgr2=0x55562b9aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.933+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/2998833361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55562b99b480 con 0x55562b9aebd0 2023-09-23T09:40:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.934+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9ac2d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3d04005950 tx=0x7f3d04026f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844e797fe6ddb09c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.934+0000 7f3cfd7fa700 1 -- 172.21.15.45:0/2998833361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04031910 con 0x55562b9aebd0 2023-09-23T09:40:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.934+0000 7f3cfd7fa700 1 -- 172.21.15.45:0/2998833361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d0402c040 con 0x55562b9aebd0 2023-09-23T09:40:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.934+0000 7f3cfd7fa700 1 -- 172.21.15.45:0/2998833361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d040046a0 con 0x55562b9aebd0 2023-09-23T09:40:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.935+0000 7f3d20b60b80 1 -- 172.21.15.45:0/2998833361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 msgr2=0x55562b9ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.935+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9ac2d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3d04005950 tx=0x7f3d04026f10 comp rx=0 tx=0).stop 2023-09-23T09:40:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.937+0000 7f3d20b60b80 1 -- 172.21.15.45:0/2998833361 shutdown_connections 2023-09-23T09:40:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.937+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.937+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.937+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/2998833361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9ac2d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.937+0000 7f3d20b60b80 1 -- 172.21.15.45:0/2998833361 >> 172.21.15.45:0/2998833361 conn(0x55562b918260 msgr2=0x55562b9aa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.937+0000 7f3d20b60b80 1 -- 172.21.15.45:0/2998833361 shutdown_connections 2023-09-23T09:40:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.937+0000 7f3d20b60b80 1 -- 172.21.15.45:0/2998833361 wait complete. 2023-09-23T09:40:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 -- start start 2023-09-23T09:40:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 Processor -- start 2023-09-23T09:40:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562ba122d0 con 0x55562b9aebd0 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562ba12410 con 0x55562b9ae310 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562ba12550 con 0x55562b9a3d20 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d20b60b80 10 client.0 _ll_register_callbacks cb 0x55562b6f0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d0ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d0dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d0dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57280/0 (socket says 172.21.15.45:57280) 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d0ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35932/0 (socket says 172.21.15.45:35932) 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.938+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/792732594 learned_addr learned my addr 172.21.15.45:0/792732594 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/792732594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 msgr2=0x55562b9b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/792732594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 msgr2=0x55562b9b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d0dd9b700 1 -- 172.21.15.45:0/792732594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55562b9598d0 con 0x55562b9ae310 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d0dd9b700 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9b4320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3d04003960 tx=0x7f3d04011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad2bcb9578046f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3cf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/792732594 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04005540 con 0x55562b9ae310 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/792732594 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d040056a0 con 0x55562b9ae310 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d04003e70 con 0x55562b9ae310 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/792732594 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d040315b0 con 0x55562b9ae310 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d20b60b80 10 client.26354 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.939+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55562b70ac60 con 0x55562b9ae310 2023-09-23T09:40:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/792732594 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d04003e70 con 0x55562b9ae310 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/792732594 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3d04024020 con 0x55562b9ae310 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d20b60b80 10 client.26354 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d20b60b80 10 client.26354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d20b60b80 20 client.26354 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562ba13790 0x55562ba15bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d20b60b80 4 client.26354 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55562b8efb30 con 0x55562ba13790 2023-09-23T09:40:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562ba13790 0x55562ba15bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3cf5ffb700 10 client.26354 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.940+0000 7f3d0e59c700 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562ba13790 0x55562ba15bb0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5688951967c8853b server_cookie=5f6a37006eda901d in_seq=0 out_seq=0 2023-09-23T09:40:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3cf5ffb700 1 -- 172.21.15.45:0/792732594 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55562b8efb30 con 0x55562ba13790 2023-09-23T09:40:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3cf5ffb700 10 client.26354 handle_command_reply: tid=0 2023-09-23T09:40:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 client.26354 shutdown 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 msgr2=0x55562b9b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9b4320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3d04003960 tx=0x7f3d04011cc0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 shutdown_connections 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55562ba13790 0x55562ba15bb0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55562b9aebd0 0x55562b9b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55562b9a3d20 0x55562b9b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 --2- 172.21.15.45:0/792732594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55562b9ae310 0x55562b9b4320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.941+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 >> 172.21.15.45:0/792732594 conn(0x55562b918260 msgr2=0x55562b9adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3cf5ffb700 0 client.26354 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 shutdown_connections 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d20b60b80 1 -- 172.21.15.45:0/792732594 wait complete. 2023-09-23T09:40:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d20b60b80 20 client.26354 trim_cache size 0 max 16384 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:40:03.947 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:40:03.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3cf80a1710 msgr2=0x7f3cf80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3cf80a1710 0x7f3cf80a3bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d10196430 tx=0x7f3d04009260 comp rx=0 tx=0).stop 2023-09-23T09:40:03.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 msgr2=0x7f3d10199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:03.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d10199b60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3d00005770 tx=0x7f3d000057a0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 shutdown_connections 2023-09-23T09:40:03.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1010b7d0 0x7f3d1010bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3cf80a1710 0x7f3cf80a3bd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d100ffed0 0x7f3d10199b60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 --2- 172.21.15.45:0/3238372857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d10104ff0 0x7f3d1019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:03.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 >> 172.21.15.45:0/3238372857 conn(0x7f3d100fb770 msgr2=0x7f3d100fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:03.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 shutdown_connections 2023-09-23T09:40:03.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:03.942+0000 7f3d14976700 1 -- 172.21.15.45:0/3238372857 wait complete. 2023-09-23T09:40:03.960 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:03 smithi045 ceph-mon[98627]: pgmap v330: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.8 MiB/s wr, 522 op/s 2023-09-23T09:40:03.962 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:40:04.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:03 smithi098 ceph-mon[102386]: pgmap v330: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.8 MiB/s wr, 522 op/s 2023-09-23T09:40:04.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:03 smithi123 ceph-mon[103577]: pgmap v330: 97 pgs: 97 active+clean; 6.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.8 MiB/s wr, 522 op/s 2023-09-23T09:40:06.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:05 smithi098 ceph-mon[102386]: pgmap v331: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 23 MiB/s wr, 1.78k op/s 2023-09-23T09:40:06.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:05 smithi098 ceph-mon[102386]: osdmap e96: 12 total, 12 up, 12 in 2023-09-23T09:40:06.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:05 smithi045 ceph-mon[98627]: pgmap v331: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 23 MiB/s wr, 1.78k op/s 2023-09-23T09:40:06.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:05 smithi045 ceph-mon[98627]: osdmap e96: 12 total, 12 up, 12 in 2023-09-23T09:40:06.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:05 smithi123 ceph-mon[103577]: pgmap v331: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 23 MiB/s wr, 1.78k op/s 2023-09-23T09:40:06.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:05 smithi123 ceph-mon[103577]: osdmap e96: 12 total, 12 up, 12 in 2023-09-23T09:40:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 620.0 s, 1319.2 tps, lat 76.310 ms stddev 106.255 2023-09-23T09:40:08.046 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:07 smithi098 ceph-mon[102386]: pgmap v333: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 23 MiB/s wr, 1.83k op/s 2023-09-23T09:40:08.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:07 smithi045 ceph-mon[98627]: pgmap v333: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 23 MiB/s wr, 1.83k op/s 2023-09-23T09:40:08.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:07 smithi123 ceph-mon[103577]: pgmap v333: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 23 MiB/s wr, 1.83k op/s 2023-09-23T09:40:10.000 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:09 smithi123 ceph-mon[103577]: pgmap v334: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-09-23T09:40:10.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:09 smithi098 ceph-mon[102386]: pgmap v334: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-09-23T09:40:10.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:09 smithi045 ceph-mon[98627]: pgmap v334: 97 pgs: 97 active+clean; 6.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-09-23T09:40:11.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 625.0 s, 1893.4 tps, lat 52.890 ms stddev 63.984 2023-09-23T09:40:12.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:11 smithi098 ceph-mon[102386]: pgmap v335: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2023-09-23T09:40:12.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:11 smithi045 ceph-mon[98627]: pgmap v335: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2023-09-23T09:40:12.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:11 smithi123 ceph-mon[103577]: pgmap v335: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2023-09-23T09:40:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 -- 172.21.15.45:0/2418014964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d0108920 msgr2=0x7f22d0108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 --2- 172.21.15.45:0/2418014964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d0108920 0x7f22d0108d00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009990 tx=0x7f22c0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 -- 172.21.15.45:0/2418014964 shutdown_connections 2023-09-23T09:40:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 --2- 172.21.15.45:0/2418014964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d010d010 0x7f22d0114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 --2- 172.21.15.45:0/2418014964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d0108920 0x7f22d0108d00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 --2- 172.21.15.45:0/2418014964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d01092d0 0x7f22d010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 -- 172.21.15.45:0/2418014964 >> 172.21.15.45:0/2418014964 conn(0x7f22d006cee0 msgr2=0x7f22d006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 -- 172.21.15.45:0/2418014964 shutdown_connections 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.757+0000 7f22d67bc700 1 -- 172.21.15.45:0/2418014964 wait complete. 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.758+0000 7f22d67bc700 1 Processor -- start 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.758+0000 7f22d67bc700 1 -- start start 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.758+0000 7f22d67bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0108920 0x7f22d01a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.758+0000 7f22d67bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d010d010 0x7f22d01a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.758+0000 7f22d67bc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d019f4f0 0x7f22d019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22d67bc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d019ff70 con 0x7f22d010d010 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22d67bc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d01a00b0 con 0x7f22d019f4f0 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22d67bc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d01a0270 con 0x7f22d0108920 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d010d010 0x7f22d01a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0108920 0x7f22d01a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22d4d59700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d019f4f0 0x7f22d019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d010d010 0x7f22d01a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35942/0 (socket says 172.21.15.45:35942) 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0108920 0x7f22d01a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38048/0 (socket says 172.21.15.45:38048) 2023-09-23T09:40:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 -- 172.21.15.45:0/3080395707 learned_addr learned my addr 172.21.15.45:0/3080395707 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 -- 172.21.15.45:0/3080395707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0108920 msgr2=0x7f22d01a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0108920 0x7f22d01a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 -- 172.21.15.45:0/3080395707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d019f4f0 msgr2=0x7f22d019f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d019f4f0 0x7f22d019f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 -- 172.21.15.45:0/3080395707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00095f0 con 0x7f22d010d010 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cffff700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0108920 0x7f22d01a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22d4d59700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d019f4f0 0x7f22d019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d010d010 0x7f22d01a5bd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e940 tx=0x7f22c400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.759+0000 7f22cd7fa700 1 -- 172.21.15.45:0/3080395707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4009730 con 0x7f22d010d010 2023-09-23T09:40:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.760+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d01ac500 con 0x7f22d010d010 2023-09-23T09:40:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.760+0000 7f22cd7fa700 1 -- 172.21.15.45:0/3080395707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c4010430 con 0x7f22d010d010 2023-09-23T09:40:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.760+0000 7f22cd7fa700 1 -- 172.21.15.45:0/3080395707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c400cea0 con 0x7f22d010d010 2023-09-23T09:40:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.760+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d01aca60 con 0x7f22d010d010 2023-09-23T09:40:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.762+0000 7f22cd7fa700 1 -- 172.21.15.45:0/3080395707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f22c4010810 con 0x7f22d010d010 2023-09-23T09:40:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.763+0000 7f22cd7fa700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22b80a1790 0x7f22b80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.763+0000 7f22cd7fa700 1 -- 172.21.15.45:0/3080395707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f22c4014070 con 0x7f22d010d010 2023-09-23T09:40:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.763+0000 7f22cffff700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22b80a1790 0x7f22b80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.764+0000 7f22cffff700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22b80a1790 0x7f22b80a3c50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b2e0 tx=0x7f22c0005960 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.764+0000 7f22e29a6b80 1 Processor -- start 2023-09-23T09:40:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.764+0000 7f22e29a6b80 1 -- start start 2023-09-23T09:40:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e9ad0 0x56170c6e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6e8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c651400 con 0x56170c6e9ad0 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c642bc0 con 0x56170c6e8480 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c6404e0 con 0x56170c6fbdb0 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22cffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e9ad0 0x56170c6e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22d4d59700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22cffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e9ad0 0x56170c6e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35950/0 (socket says 172.21.15.45:35950) 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22cffff700 1 -- 172.21.15.45:0/1105424711 learned_addr learned my addr 172.21.15.45:0/1105424711 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22d4d59700 1 -- 172.21.15.45:0/1105424711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 msgr2=0x56170c6e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22d4d59700 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6e8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22d4d59700 1 -- 172.21.15.45:0/1105424711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e9ad0 msgr2=0x56170c6e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22d4d59700 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e9ad0 0x56170c6e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22d4d59700 1 -- 172.21.15.45:0/1105424711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170c69e870 con 0x56170c6fbdb0 2023-09-23T09:40:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22cffff700 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e9ad0 0x56170c6e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.765+0000 7f22d4d59700 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fe1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f22c8003db0 tx=0x7f22c8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450a6dddcb3d67f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.767+0000 7f22b67fc700 1 -- 172.21.15.45:0/1105424711 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8004bb0 con 0x56170c6fbdb0 2023-09-23T09:40:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.767+0000 7f22b67fc700 1 -- 172.21.15.45:0/1105424711 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c800f040 con 0x56170c6fbdb0 2023-09-23T09:40:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.767+0000 7f22b67fc700 1 -- 172.21.15.45:0/1105424711 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8007cb0 con 0x56170c6fbdb0 2023-09-23T09:40:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.768+0000 7f22e29a6b80 1 -- 172.21.15.45:0/1105424711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 msgr2=0x56170c6fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.768+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fe1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f22c8003db0 tx=0x7f22c8009920 comp rx=0 tx=0).stop 2023-09-23T09:40:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.770+0000 7f22e29a6b80 1 -- 172.21.15.45:0/1105424711 shutdown_connections 2023-09-23T09:40:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.770+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fe1a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.770+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e9ad0 0x56170c6e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.770+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/1105424711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.770+0000 7f22e29a6b80 1 -- 172.21.15.45:0/1105424711 >> 172.21.15.45:0/1105424711 conn(0x56170c65d200 msgr2=0x56170c6e43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.770+0000 7f22e29a6b80 1 -- 172.21.15.45:0/1105424711 shutdown_connections 2023-09-23T09:40:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.771+0000 7f22e29a6b80 1 -- 172.21.15.45:0/1105424711 wait complete. 2023-09-23T09:40:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.771+0000 7f22e29a6b80 1 -- start start 2023-09-23T09:40:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.771+0000 7f22e29a6b80 1 Processor -- start 2023-09-23T09:40:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e8480 0x56170c6f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e9ad0 0x56170c6f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c63bdd0 con 0x56170c6e8480 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c5d86b0 con 0x56170c6e9ad0 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c5d8350 con 0x56170c6fbdb0 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22e29a6b80 10 client.0 _ll_register_callbacks cb 0x56170c4357a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22cf7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e9ad0 0x56170c6f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22cffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e8480 0x56170c6f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22cffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e8480 0x56170c6f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35956/0 (socket says 172.21.15.45:35956) 2023-09-23T09:40:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.772+0000 7f22cffff700 1 -- 172.21.15.45:0/3124818464 learned_addr learned my addr 172.21.15.45:0/3124818464 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22cffff700 1 -- 172.21.15.45:0/3124818464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 msgr2=0x56170c6fb0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22cffff700 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fb0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22cffff700 1 -- 172.21.15.45:0/3124818464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e9ad0 msgr2=0x56170c6f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22cffff700 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e9ad0 0x56170c6f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22cffff700 1 -- 172.21.15.45:0/3124818464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d0108170 con 0x56170c6e8480 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22cffff700 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e8480 0x56170c6f53d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f22c00098e0 tx=0x7f22c0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549300902bc936f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22af7fe700 1 -- 172.21.15.45:0/3124818464 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c000f670 con 0x56170c6e8480 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22af7fe700 1 -- 172.21.15.45:0/3124818464 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f22c000f7d0 con 0x56170c6e8480 2023-09-23T09:40:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.773+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56170c6dc090 con 0x56170c6e8480 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22af7fe700 1 -- 172.21.15.45:0/3124818464 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22c000f9d0 con 0x56170c6e8480 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 10 client.16680 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f22c8003a60 con 0x56170c6e8480 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22af7fe700 1 -- 172.21.15.45:0/3124818464 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56170c6dc090 con 0x56170c6e8480 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22af7fe700 1 -- 172.21.15.45:0/3124818464 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f22c002f070 con 0x56170c6e8480 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 10 client.16680 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 20 client.16680 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7142a0 0x56170c7166c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 4 client.16680 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:40:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.774+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56170c634db0 con 0x56170c7142a0 2023-09-23T09:40:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.775+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7142a0 0x56170c7166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.775+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7142a0 0x56170c7166c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf06e5973f7db56e server_cookie=48e06311638fe8e9 in_seq=0 out_seq=0 2023-09-23T09:40:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.775+0000 7f22af7fe700 10 client.16680 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22af7fe700 1 -- 172.21.15.45:0/3124818464 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56170c634db0 con 0x56170c7142a0 2023-09-23T09:40:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22af7fe700 10 client.16680 handle_command_reply: tid=0 2023-09-23T09:40:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 client.16680 shutdown 2023-09-23T09:40:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e8480 msgr2=0x56170c6f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e8480 0x56170c6f53d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f22c00098e0 tx=0x7f22c0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 shutdown_connections 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7142a0 0x56170c7166c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6fbdb0 0x56170c6fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22af7fe700 0 client.16680 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6e8480 0x56170c6f53d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3124818464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e9ad0 0x56170c6f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 >> 172.21.15.45:0/3124818464 conn(0x56170c65d200 msgr2=0x56170c6e3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 shutdown_connections 2023-09-23T09:40:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3124818464 wait complete. 2023-09-23T09:40:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.776+0000 7f22e29a6b80 20 client.16680 trim_cache size 0 max 16384 2023-09-23T09:40:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.799+0000 7f22e29a6b80 1 Processor -- start 2023-09-23T09:40:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.799+0000 7f22e29a6b80 1 -- start start 2023-09-23T09:40:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6fb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e9ad0 0x56170c6e3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c6efe40 con 0x56170c6fbdb0 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c4bc8c0 con 0x56170c6e8480 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c6e9160 con 0x56170c6e9ad0 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e9ad0 0x56170c6e3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e9ad0 0x56170c6e3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38070/0 (socket says 172.21.15.45:38070) 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 -- 172.21.15.45:0/2725774261 learned_addr learned my addr 172.21.15.45:0/2725774261 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22d4d59700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 -- 172.21.15.45:0/2725774261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 msgr2=0x56170c6fb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6fb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 -- 172.21.15.45:0/2725774261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 msgr2=0x56170c6e43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6e43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 -- 172.21.15.45:0/2725774261 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170c466d80 con 0x56170c6e9ad0 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cffff700 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e9ad0 0x56170c6e3e90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f22c40cabd0 tx=0x7f22c401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6289882cde0eae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.800+0000 7f22b6ffd700 1 -- 172.21.15.45:0/2725774261 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c409e4e0 con 0x56170c6e9ad0 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22b6ffd700 1 -- 172.21.15.45:0/2725774261 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c40a2720 con 0x56170c6e9ad0 2023-09-23T09:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22b6ffd700 1 -- 172.21.15.45:0/2725774261 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c409a850 con 0x56170c6e9ad0 2023-09-23T09:40:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 -- 172.21.15.45:0/2725774261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e9ad0 msgr2=0x56170c6e3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e9ad0 0x56170c6e3e90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f22c40cabd0 tx=0x7f22c401a040 comp rx=0 tx=0).stop 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 -- 172.21.15.45:0/2725774261 shutdown_connections 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6e8480 0x56170c6fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/2725774261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e9ad0 0x56170c6e3e90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 -- 172.21.15.45:0/2725774261 >> 172.21.15.45:0/2725774261 conn(0x56170c65d200 msgr2=0x56170c6e2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 -- 172.21.15.45:0/2725774261 shutdown_connections 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.801+0000 7f22e29a6b80 1 -- 172.21.15.45:0/2725774261 wait complete. 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 -- start start 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 Processor -- start 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e8480 0x56170c6fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6f9390 0x56170c6f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c6fb2f0 con 0x56170c6fbdb0 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c6fb430 con 0x56170c6f9390 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170c7680c0 con 0x56170c6e8480 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22e29a6b80 10 client.0 _ll_register_callbacks cb 0x7f22c0005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22cffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e8480 0x56170c6fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22d4d59700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6f9390 0x56170c6f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22cffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e8480 0x56170c6fabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38074/0 (socket says 172.21.15.45:38074) 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.802+0000 7f22cffff700 1 -- 172.21.15.45:0/3579333889 learned_addr learned my addr 172.21.15.45:0/3579333889 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22cffff700 1 -- 172.21.15.45:0/3579333889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6f9390 msgr2=0x56170c6f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22cffff700 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6f9390 0x56170c6f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22cffff700 1 -- 172.21.15.45:0/3579333889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 msgr2=0x56170c6f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22cffff700 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22cffff700 1 -- 172.21.15.45:0/3579333889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c8003a60 con 0x56170c6e8480 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22cffff700 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e8480 0x56170c6fabc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005f50 tx=0x7f22c00246c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9d66e04f275fb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22af7fe700 1 -- 172.21.15.45:0/3579333889 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00188b0 con 0x56170c6e8480 2023-09-23T09:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.803+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22c800f040 con 0x56170c6e8480 2023-09-23T09:40:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.805+0000 7f22af7fe700 1 -- 172.21.15.45:0/3579333889 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f22c0018400 con 0x56170c6e8480 2023-09-23T09:40:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.805+0000 7f22af7fe700 1 -- 172.21.15.45:0/3579333889 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22c0018a10 con 0x56170c6e8480 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.805+0000 7f22e29a6b80 10 client.26353 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.805+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f22d01aca60 con 0x56170c6e8480 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22af7fe700 1 -- 172.21.15.45:0/3579333889 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c800f040 con 0x56170c6e8480 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22af7fe700 1 -- 172.21.15.45:0/3579333889 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f22c0004780 con 0x56170c6e8480 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22e29a6b80 10 client.26353 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22e29a6b80 10 client.26353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22e29a6b80 20 client.26353 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7693f0 0x56170c76b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22e29a6b80 4 client.26353 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:40:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56170c634ad0 con 0x56170c7693f0 2023-09-23T09:40:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.806+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7693f0 0x56170c76b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.807+0000 7f22af7fe700 10 client.26353 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.807+0000 7f22cf7fe700 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7693f0 0x56170c76b810 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f453b410f55a8cab server_cookie=22c6730c1a3d0c70 in_seq=0 out_seq=0 2023-09-23T09:40:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22af7fe700 1 -- 172.21.15.45:0/3579333889 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56170c634ad0 con 0x56170c7693f0 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22af7fe700 10 client.26353 handle_command_reply: tid=0 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 client.26353 shutdown 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e8480 msgr2=0x56170c6fabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e8480 0x56170c6fabc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005f50 tx=0x7f22c00246c0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 shutdown_connections 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x56170c7693f0 0x56170c76b810 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56170c6f9390 0x56170c6f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56170c6e8480 0x56170c6fabc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 --2- 172.21.15.45:0/3579333889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56170c6fbdb0 0x56170c6f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22af7fe700 0 client.26353 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 >> 172.21.15.45:0/3579333889 conn(0x56170c65d200 msgr2=0x56170c6e3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 shutdown_connections 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 1 -- 172.21.15.45:0/3579333889 wait complete. 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.822+0000 7f22e29a6b80 20 client.26353 trim_cache size 0 max 16384 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:40:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:40:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22b80a1790 msgr2=0x7f22b80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22b80a1790 0x7f22b80a3c50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b2e0 tx=0x7f22c0005960 comp rx=0 tx=0).stop 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d010d010 msgr2=0x7f22d01a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d010d010 0x7f22d01a5bd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e940 tx=0x7f22c400ec50 comp rx=0 tx=0).stop 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 shutdown_connections 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22d019f4f0 0x7f22d019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d0108920 0x7f22d01a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f22b80a1790 0x7f22b80a3c50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 --2- 172.21.15.45:0/3080395707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d010d010 0x7f22d01a5bd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 >> 172.21.15.45:0/3080395707 conn(0x7f22d006cee0 msgr2=0x7f22d010b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 shutdown_connections 2023-09-23T09:40:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:13.823+0000 7f22d67bc700 1 -- 172.21.15.45:0/3080395707 wait complete. 2023-09-23T09:40:14.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:13 smithi098 ceph-mon[102386]: pgmap v336: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2023-09-23T09:40:14.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:13 smithi045 ceph-mon[98627]: pgmap v336: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2023-09-23T09:40:14.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:13 smithi123 ceph-mon[103577]: pgmap v336: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2023-09-23T09:40:16.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:15 smithi098 ceph-mon[102386]: pgmap v337: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2023-09-23T09:40:16.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:15 smithi098 ceph-mon[102386]: osdmap e97: 12 total, 12 up, 12 in 2023-09-23T09:40:16.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:15 smithi045 ceph-mon[98627]: pgmap v337: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2023-09-23T09:40:16.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:15 smithi045 ceph-mon[98627]: osdmap e97: 12 total, 12 up, 12 in 2023-09-23T09:40:16.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:15 smithi123 ceph-mon[103577]: pgmap v337: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2023-09-23T09:40:16.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:15 smithi123 ceph-mon[103577]: osdmap e97: 12 total, 12 up, 12 in 2023-09-23T09:40:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 630.0 s, 1630.7 tps, lat 61.234 ms stddev 68.979 2023-09-23T09:40:18.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:17 smithi098 ceph-mon[102386]: pgmap v339: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2023-09-23T09:40:18.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:17 smithi045 ceph-mon[98627]: pgmap v339: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2023-09-23T09:40:18.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:17 smithi123 ceph-mon[103577]: pgmap v339: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2023-09-23T09:40:20.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:19 smithi098 ceph-mon[102386]: pgmap v340: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 640 op/s 2023-09-23T09:40:20.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:19 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:40:20.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:19 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:40:20.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:19 smithi045 ceph-mon[98627]: pgmap v340: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 640 op/s 2023-09-23T09:40:20.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:19 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:40:20.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:19 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:40:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:19 smithi123 ceph-mon[103577]: pgmap v340: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 640 op/s 2023-09-23T09:40:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:19 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:40:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:19 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:40:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 635.0 s, 1931.0 tps, lat 51.181 ms stddev 62.391 2023-09-23T09:40:22.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:21 smithi098 ceph-mon[102386]: pgmap v341: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 714 op/s 2023-09-23T09:40:22.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:21 smithi045 ceph-mon[98627]: pgmap v341: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 714 op/s 2023-09-23T09:40:22.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:21 smithi123 ceph-mon[103577]: pgmap v341: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 714 op/s 2023-09-23T09:40:24.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:23 smithi098 ceph-mon[102386]: pgmap v342: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 714 op/s 2023-09-23T09:40:24.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:23 smithi045 ceph-mon[98627]: pgmap v342: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 714 op/s 2023-09-23T09:40:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:23 smithi123 ceph-mon[103577]: pgmap v342: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 714 op/s 2023-09-23T09:40:26.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:25 smithi098 ceph-mon[102386]: pgmap v343: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.1 MiB/s wr, 717 op/s 2023-09-23T09:40:26.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:25 smithi045 ceph-mon[98627]: pgmap v343: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.1 MiB/s wr, 717 op/s 2023-09-23T09:40:26.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:25 smithi123 ceph-mon[103577]: pgmap v343: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.1 MiB/s wr, 717 op/s 2023-09-23T09:40:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 640.0 s, 1837.3 tps, lat 54.422 ms stddev 66.487 2023-09-23T09:40:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 -- 172.21.15.45:0/2799465041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa468100160 msgr2=0x7fa468100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/2799465041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa468100160 0x7fa468100540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa458009990 tx=0x7fa458009ca0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 -- 172.21.15.45:0/2799465041 shutdown_connections 2023-09-23T09:40:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/2799465041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468105050 0x7fa46810cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/2799465041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa468100160 0x7fa468100540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/2799465041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa468100b10 0x7fa468104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 -- 172.21.15.45:0/2799465041 >> 172.21.15.45:0/2799465041 conn(0x7fa468076840 msgr2=0x7fa468076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.004+0000 7fa46f5cd700 1 -- 172.21.15.45:0/2799465041 shutdown_connections 2023-09-23T09:40:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.005+0000 7fa46f5cd700 1 -- 172.21.15.45:0/2799465041 wait complete. 2023-09-23T09:40:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.005+0000 7fa46f5cd700 1 Processor -- start 2023-09-23T09:40:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.005+0000 7fa46f5cd700 1 -- start start 2023-09-23T09:40:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46f5cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468100160 0x7fa468196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46f5cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa468105050 0x7fa468197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46f5cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa46819baa0 0x7fa46819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46f5cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46819c520 con 0x7fa468105050 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46f5cd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46819c690 con 0x7fa46819baa0 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46f5cd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46819c850 con 0x7fa468100160 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468100160 0x7fa468196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468100160 0x7fa468196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38086/0 (socket says 172.21.15.45:38086) 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 -- 172.21.15.45:0/405556043 learned_addr learned my addr 172.21.15.45:0/405556043 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46db6a700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa46819baa0 0x7fa46819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 -- 172.21.15.45:0/405556043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa46819baa0 msgr2=0x7fa46819bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa46819baa0 0x7fa46819bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 -- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa468105050 msgr2=0x7fa468197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa468105050 0x7fa468197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 -- 172.21.15.45:0/405556043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4580095f0 con 0x7fa468100160 2023-09-23T09:40:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.006+0000 7fa46d369700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468100160 0x7fa468196ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa458005910 tx=0x7fa458005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.007+0000 7fa45e7fc700 1 -- 172.21.15.45:0/405556043 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45801d070 con 0x7fa468100160 2023-09-23T09:40:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.007+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4681a3d50 con 0x7fa468100160 2023-09-23T09:40:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.007+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4681a41d0 con 0x7fa468100160 2023-09-23T09:40:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.007+0000 7fa45e7fc700 1 -- 172.21.15.45:0/405556043 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45802b3b0 con 0x7fa468100160 2023-09-23T09:40:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.007+0000 7fa45e7fc700 1 -- 172.21.15.45:0/405556043 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800e640 con 0x7fa468100160 2023-09-23T09:40:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.010+0000 7fa45e7fc700 1 -- 172.21.15.45:0/405556043 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fa45800fad0 con 0x7fa468100160 2023-09-23T09:40:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.010+0000 7fa45e7fc700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa4540a1710 0x7fa4540a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.011+0000 7fa45e7fc700 1 -- 172.21.15.45:0/405556043 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa4580ce6a0 con 0x7fa468100160 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.011+0000 7fa46cb68700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa4540a1710 0x7fa4540a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.012+0000 7fa46cb68700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa4540a1710 0x7fa4540a3bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa464005950 tx=0x7fa4640058e0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.012+0000 7fa47b7b7b80 1 Processor -- start 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.012+0000 7fa47b7b7b80 1 -- start start 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.013+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 0x55646c2bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.013+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.013+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.013+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c227400 con 0x55646c2d1db0 2023-09-23T09:40:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.013+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c218bc0 con 0x55646c2be480 2023-09-23T09:40:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.013+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c2164e0 con 0x55646c2bfad0 2023-09-23T09:40:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.013+0000 7fa46cb68700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.014+0000 7fa46cb68700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2be900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57342/0 (socket says 172.21.15.45:57342) 2023-09-23T09:40:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.014+0000 7fa46cb68700 1 -- 172.21.15.45:0/11505947 learned_addr learned my addr 172.21.15.45:0/11505947 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.014+0000 7fa46db6a700 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa46db6a700 1 -- 172.21.15.45:0/11505947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 msgr2=0x55646c2bdf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:40:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa46db6a700 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 0x55646c2bdf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa46db6a700 1 -- 172.21.15.45:0/11505947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 msgr2=0x55646c2be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa46db6a700 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa46db6a700 1 -- 172.21.15.45:0/11505947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55646c274870 con 0x55646c2d1db0 2023-09-23T09:40:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa46db6a700 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d41a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa460011580 tx=0x7fa460009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6693039ce190b30 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa4537fe700 1 -- 172.21.15.45:0/11505947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46000e040 con 0x55646c2d1db0 2023-09-23T09:40:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa4537fe700 1 -- 172.21.15.45:0/11505947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46000f040 con 0x55646c2d1db0 2023-09-23T09:40:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.015+0000 7fa4537fe700 1 -- 172.21.15.45:0/11505947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600044d0 con 0x55646c2d1db0 2023-09-23T09:40:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/11505947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 msgr2=0x55646c2d41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d41a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa460011580 tx=0x7fa460009920 comp rx=0 tx=0).stop 2023-09-23T09:40:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/11505947 shutdown_connections 2023-09-23T09:40:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d41a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 0x55646c2bdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/11505947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/11505947 >> 172.21.15.45:0/11505947 conn(0x55646c233200 msgr2=0x55646c2ba3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/11505947 shutdown_connections 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.016+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/11505947 wait complete. 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 -- start start 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 Processor -- start 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 0x55646c2cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c211dd0 con 0x55646c2d1db0 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c1ae6b0 con 0x55646c2be480 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.017+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c1ae350 con 0x55646c2bfad0 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.018+0000 7fa47b7b7b80 10 client.0 _ll_register_callbacks cb 0x55646c00b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.018+0000 7fa46db6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.018+0000 7fa46db6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35996/0 (socket says 172.21.15.45:35996) 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.018+0000 7fa46db6a700 1 -- 172.21.15.45:0/2285424256 learned_addr learned my addr 172.21.15.45:0/2285424256 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.018+0000 7fa46d369700 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.019+0000 7fa46d369700 1 -- 172.21.15.45:0/2285424256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 msgr2=0x55646c2cb890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:40:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.019+0000 7fa46d369700 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 0x55646c2cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.019+0000 7fa46d369700 1 -- 172.21.15.45:0/2285424256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 msgr2=0x55646c2d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.019+0000 7fa46d369700 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.019+0000 7fa46d369700 1 -- 172.21.15.45:0/2285424256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa460011230 con 0x55646c2be480 2023-09-23T09:40:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.019+0000 7fa46d369700 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2cb350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4580cbdb0 tx=0x7fa45809b220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98810cf81f27a1a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.019+0000 7fa450ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:40:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.020+0000 7fa450ff9700 1 -- 172.21.15.45:0/2285424256 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45809b660 con 0x55646c2be480 2023-09-23T09:40:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.020+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55646c2b2090 con 0x55646c2be480 2023-09-23T09:40:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.020+0000 7fa450ff9700 1 -- 172.21.15.45:0/2285424256 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa45809f7b0 con 0x55646c2be480 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.020+0000 7fa450ff9700 1 -- 172.21.15.45:0/2285424256 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4580a3960 con 0x55646c2be480 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.020+0000 7fa47b7b7b80 10 client.26384 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.020+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55646c03cd80 con 0x55646c2be480 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa450ff9700 1 -- 172.21.15.45:0/2285424256 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55646c2b2090 con 0x55646c2be480 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa450ff9700 1 -- 172.21.15.45:0/2285424256 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa4580a34d0 con 0x55646c2be480 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa47b7b7b80 10 client.26384 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa47b7b7b80 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa47b7b7b80 20 client.26384 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c2e9f40 0x55646c2ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa47b7b7b80 4 client.26384 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55646c20adb0 con 0x55646c2e9f40 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.021+0000 7fa46cb68700 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c2e9f40 0x55646c2ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.022+0000 7fa46cb68700 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c2e9f40 0x55646c2ec360 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e2fa7027c45968 server_cookie=9108a68759b9d9fb in_seq=0 out_seq=0 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.022+0000 7fa450ff9700 10 client.26384 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.022+0000 7fa450ff9700 1 -- 172.21.15.45:0/2285424256 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55646c20adb0 con 0x55646c2e9f40 2023-09-23T09:40:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.022+0000 7fa450ff9700 10 client.26384 handle_command_reply: tid=0 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.022+0000 7fa47b7b7b80 1 client.26384 shutdown 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.022+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 msgr2=0x55646c2cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.022+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2cb350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4580cbdb0 tx=0x7fa45809b220 comp rx=0 tx=0).stop 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 shutdown_connections 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c2e9f40 0x55646c2ec360 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2cb350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/2285424256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2bfad0 0x55646c2cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 >> 172.21.15.45:0/2285424256 conn(0x55646c233200 msgr2=0x55646c2b9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa450ff9700 0 client.26384 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 shutdown_connections 2023-09-23T09:40:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/2285424256 wait complete. 2023-09-23T09:40:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.024+0000 7fa47b7b7b80 20 client.26384 trim_cache size 0 max 16384 2023-09-23T09:40:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.050+0000 7fa47b7b7b80 1 Processor -- start 2023-09-23T09:40:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.050+0000 7fa47b7b7b80 1 -- start start 2023-09-23T09:40:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.051+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.051+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2bfad0 0x55646c2caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.051+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2d1db0 0x55646c2cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.051+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c0928c0 con 0x55646c2bfad0 2023-09-23T09:40:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.051+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c2c7c00 con 0x55646c2be480 2023-09-23T09:40:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.051+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c2be860 con 0x55646c2d1db0 2023-09-23T09:40:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.057+0000 7fa46d369700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.057+0000 7fa46d369700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2c1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57350/0 (socket says 172.21.15.45:57350) 2023-09-23T09:40:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.057+0000 7fa46d369700 1 -- 172.21.15.45:0/1661276542 learned_addr learned my addr 172.21.15.45:0/1661276542 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.065+0000 7fa46db6a700 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2d1db0 0x55646c2cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.065+0000 7fa46cb68700 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2bfad0 0x55646c2caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.066+0000 7fa46d369700 1 -- 172.21.15.45:0/1661276542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2d1db0 msgr2=0x55646c2cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.066+0000 7fa46d369700 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2d1db0 0x55646c2cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.066+0000 7fa46d369700 1 -- 172.21.15.45:0/1661276542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2bfad0 msgr2=0x55646c2caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.066+0000 7fa46d369700 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2bfad0 0x55646c2caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.066+0000 7fa46d369700 1 -- 172.21.15.45:0/1661276542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55646c025c00 con 0x55646c2be480 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.071+0000 7fa46d369700 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2c1470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4580cbe60 tx=0x7fa458009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33e84de2e12b6cf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.071+0000 7fa453fff700 1 -- 172.21.15.45:0/1661276542 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458097800 con 0x55646c2be480 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.071+0000 7fa453fff700 1 -- 172.21.15.45:0/1661276542 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4580975a0 con 0x55646c2be480 2023-09-23T09:40:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.072+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/1661276542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 msgr2=0x55646c2c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.072+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2c1470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4580cbe60 tx=0x7fa458009ca0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.075+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/1661276542 shutdown_connections 2023-09-23T09:40:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.075+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2d1db0 0x55646c2cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.075+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2be480 0x55646c2c1470 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.075+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/1661276542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2bfad0 0x55646c2caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.075+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/1661276542 >> 172.21.15.45:0/1661276542 conn(0x55646c233200 msgr2=0x55646c2f0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.075+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/1661276542 shutdown_connections 2023-09-23T09:40:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.075+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/1661276542 wait complete. 2023-09-23T09:40:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.076+0000 7fa47b7b7b80 1 -- start start 2023-09-23T09:40:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.076+0000 7fa47b7b7b80 1 Processor -- start 2023-09-23T09:40:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.077+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2be480 0x55646c2d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.077+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2bfad0 0x55646c2ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.077+0000 7fa47b7b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.077+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c2cef90 con 0x55646c2d1db0 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.077+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c2cf100 con 0x55646c2bfad0 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.077+0000 7fa47b7b7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646c34ddb0 con 0x55646c2be480 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.077+0000 7fa47b7b7b80 10 client.0 _ll_register_callbacks cb 0x7fa4580979c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.079+0000 7fa46d369700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2be480 0x55646c2d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.079+0000 7fa46d369700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2be480 0x55646c2d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38114/0 (socket says 172.21.15.45:38114) 2023-09-23T09:40:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.079+0000 7fa46d369700 1 -- 172.21.15.45:0/417655812 learned_addr learned my addr 172.21.15.45:0/417655812 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.079+0000 7fa46db6a700 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa46db6a700 1 -- 172.21.15.45:0/417655812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2be480 msgr2=0x55646c2d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa46db6a700 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2be480 0x55646c2d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa46db6a700 1 -- 172.21.15.45:0/417655812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2bfad0 msgr2=0x55646c2ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa46db6a700 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2bfad0 0x55646c2ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa46db6a700 1 -- 172.21.15.45:0/417655812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55646c03cd80 con 0x55646c2d1db0 2023-09-23T09:40:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa46db6a700 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2ce880 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa460011520 tx=0x7fa4600077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e05f447b9df87a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa450ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:40:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa450ff9700 1 -- 172.21.15.45:0/417655812 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46000e040 con 0x55646c2d1db0 2023-09-23T09:40:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.080+0000 7fa450ff9700 1 -- 172.21.15.45:0/417655812 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa460007c10 con 0x55646c2d1db0 2023-09-23T09:40:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.081+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa46000f040 con 0x55646c2d1db0 2023-09-23T09:40:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.081+0000 7fa450ff9700 1 -- 172.21.15.45:0/417655812 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa460009ac0 con 0x55646c2d1db0 2023-09-23T09:40:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.084+0000 7fa47b7b7b80 10 client.16701 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.084+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55646c0631f0 con 0x55646c2d1db0 2023-09-23T09:40:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.087+0000 7fa450ff9700 1 -- 172.21.15.45:0/417655812 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46000f040 con 0x55646c2d1db0 2023-09-23T09:40:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.087+0000 7fa450ff9700 1 -- 172.21.15.45:0/417655812 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa46001b020 con 0x55646c2d1db0 2023-09-23T09:40:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.091+0000 7fa47b7b7b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.091+0000 7fa47b7b7b80 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.091+0000 7fa47b7b7b80 20 client.16701 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.091+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c34f440 0x55646c351860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.091+0000 7fa47b7b7b80 4 client.16701 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:40:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.091+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55646c20aad0 con 0x55646c34f440 2023-09-23T09:40:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.094+0000 7fa46d369700 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c34f440 0x55646c351860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.094+0000 7fa450ff9700 10 client.16701 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.094+0000 7fa46d369700 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c34f440 0x55646c351860 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2d9f0a6994c23a server_cookie=f537533fb19b9e2b in_seq=0 out_seq=0 2023-09-23T09:40:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa450ff9700 1 -- 172.21.15.45:0/417655812 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55646c20aad0 con 0x55646c34f440 2023-09-23T09:40:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa450ff9700 10 client.16701 handle_command_reply: tid=0 2023-09-23T09:40:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 client.16701 shutdown 2023-09-23T09:40:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 msgr2=0x55646c2ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2ce880 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa460011520 tx=0x7fa4600077e0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 shutdown_connections 2023-09-23T09:40:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55646c34f440 0x55646c351860 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55646c2d1db0 0x55646c2ce880 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa450ff9700 0 client.16701 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646c2be480 0x55646c2d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 --2- 172.21.15.45:0/417655812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55646c2bfad0 0x55646c2ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.127+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 >> 172.21.15.45:0/417655812 conn(0x55646c233200 msgr2=0x55646c2c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.128+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 shutdown_connections 2023-09-23T09:40:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.128+0000 7fa47b7b7b80 1 -- 172.21.15.45:0/417655812 wait complete. 2023-09-23T09:40:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.128+0000 7fa47b7b7b80 20 client.16701 trim_cache size 0 max 16384 2023-09-23T09:40:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:40:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:40:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:40:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:40:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.128+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa4540a1710 msgr2=0x7fa4540a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.128+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa4540a1710 0x7fa4540a3bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa464005950 tx=0x7fa4640058e0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.128+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468100160 msgr2=0x7fa468196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.128+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468100160 0x7fa468196ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa458005910 tx=0x7fa458005740 comp rx=0 tx=0).stop 2023-09-23T09:40:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 shutdown_connections 2023-09-23T09:40:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa46819baa0 0x7fa46819bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468100160 0x7fa468196ef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa4540a1710 0x7fa4540a3bd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 --2- 172.21.15.45:0/405556043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa468105050 0x7fa468197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 >> 172.21.15.45:0/405556043 conn(0x7fa468076840 msgr2=0x7fa4680fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 shutdown_connections 2023-09-23T09:40:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:27.129+0000 7fa46f5cd700 1 -- 172.21.15.45:0/405556043 wait complete. 2023-09-23T09:40:28.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:27 smithi098 ceph-mon[102386]: pgmap v344: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.7 MiB/s wr, 687 op/s 2023-09-23T09:40:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:27 smithi045 ceph-mon[98627]: pgmap v344: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.7 MiB/s wr, 687 op/s 2023-09-23T09:40:28.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:27 smithi123 ceph-mon[103577]: pgmap v344: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.7 MiB/s wr, 687 op/s 2023-09-23T09:40:30.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:29 smithi098 ceph-mon[102386]: pgmap v345: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.1 MiB/s wr, 631 op/s 2023-09-23T09:40:30.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:29 smithi045 ceph-mon[98627]: pgmap v345: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.1 MiB/s wr, 631 op/s 2023-09-23T09:40:30.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:29 smithi123 ceph-mon[103577]: pgmap v345: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.1 MiB/s wr, 631 op/s 2023-09-23T09:40:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 645.0 s, 525.2 tps, lat 186.337 ms stddev 350.588 2023-09-23T09:40:32.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:31 smithi098 ceph-mon[102386]: pgmap v346: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 12 MiB/s wr, 898 op/s 2023-09-23T09:40:32.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:31 smithi045 ceph-mon[98627]: pgmap v346: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 12 MiB/s wr, 898 op/s 2023-09-23T09:40:32.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:31 smithi123 ceph-mon[103577]: pgmap v346: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 12 MiB/s wr, 898 op/s 2023-09-23T09:40:33.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:33 smithi098 ceph-mon[102386]: pgmap v347: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.3 MiB/s wr, 589 op/s 2023-09-23T09:40:33.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:33 smithi045 ceph-mon[98627]: pgmap v347: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.3 MiB/s wr, 589 op/s 2023-09-23T09:40:33.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:33 smithi123 ceph-mon[103577]: pgmap v347: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.3 MiB/s wr, 589 op/s 2023-09-23T09:40:33.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:40:34.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.180+0000 7fc41991a700 1 -- 172.21.15.45:0/1056434077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 msgr2=0x7fc41410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.180+0000 7fc41991a700 1 --2- 172.21.15.45:0/1056434077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410ad90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc4080098f0 tx=0x7fc408009c00 comp rx=0 tx=0).stop 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.181+0000 7fc41991a700 1 -- 172.21.15.45:0/1056434077 shutdown_connections 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.181+0000 7fc41991a700 1 --2- 172.21.15.45:0/1056434077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41410b360 0x7fc41410d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.181+0000 7fc41991a700 1 --2- 172.21.15.45:0/1056434077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc414101f90 0x7fc414102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.181+0000 7fc41991a700 1 --2- 172.21.15.45:0/1056434077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410ad90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.181+0000 7fc41991a700 1 -- 172.21.15.45:0/1056434077 >> 172.21.15.45:0/1056434077 conn(0x7fc4140fb770 msgr2=0x7fc4140fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.181+0000 7fc41991a700 1 -- 172.21.15.45:0/1056434077 shutdown_connections 2023-09-23T09:40:34.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.181+0000 7fc41991a700 1 -- 172.21.15.45:0/1056434077 wait complete. 2023-09-23T09:40:34.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.182+0000 7fc41991a700 1 Processor -- start 2023-09-23T09:40:34.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc41991a700 1 -- start start 2023-09-23T09:40:34.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc41991a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc414101f90 0x7fc414109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc41991a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc41991a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41410b360 0x7fc4141ba320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc41991a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141ba860 con 0x7fc414101f90 2023-09-23T09:40:34.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc41991a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141ba9a0 con 0x7fc41410b360 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc41991a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141baae0 con 0x7fc4141028b0 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc4127fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc4127fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38126/0 (socket says 172.21.15.45:38126) 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc4127fc700 1 -- 172.21.15.45:0/510937313 learned_addr learned my addr 172.21.15.45:0/510937313 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc4137fe700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41410b360 0x7fc4141ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.183+0000 7fc412ffd700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc414101f90 0x7fc414109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc4127fc700 1 -- 172.21.15.45:0/510937313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41410b360 msgr2=0x7fc4141ba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc4127fc700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41410b360 0x7fc4141ba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc4127fc700 1 -- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc414101f90 msgr2=0x7fc414109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc4127fc700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc414101f90 0x7fc414109c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc4127fc700 1 -- 172.21.15.45:0/510937313 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040095f0 con 0x7fc4141028b0 2023-09-23T09:40:34.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc4127fc700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410a1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc408009230 tx=0x7fc40800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:34.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc418918700 1 -- 172.21.15.45:0/510937313 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40801c070 con 0x7fc4141028b0 2023-09-23T09:40:34.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc408009520 con 0x7fc4141028b0 2023-09-23T09:40:34.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc418918700 1 -- 172.21.15.45:0/510937313 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc408005040 con 0x7fc4141028b0 2023-09-23T09:40:34.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc418918700 1 -- 172.21.15.45:0/510937313 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408003aa0 con 0x7fc4141028b0 2023-09-23T09:40:34.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.184+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4141bb080 con 0x7fc4141028b0 2023-09-23T09:40:34.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.188+0000 7fc418918700 1 -- 172.21.15.45:0/510937313 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fc408005470 con 0x7fc4141028b0 2023-09-23T09:40:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc425b04b80 1 Processor -- start 2023-09-23T09:40:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc425b04b80 1 -- start start 2023-09-23T09:40:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc418918700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc3fc0a1710 0x7fc3fc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc418918700 1 -- 172.21.15.45:0/510937313 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc4080cdf40 con 0x7fc4141028b0 2023-09-23T09:40:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc4137fe700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc3fc0a1710 0x7fc3fc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5400 0x558c5c9a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5d20 0x558c5c9b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.189+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc425b04b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c90e460 con 0x558c5c9a5d20 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc425b04b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c8ffc20 con 0x558c5c9a5400 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc425b04b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c8fd540 con 0x558c5c9b0bd0 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5400 0x558c5c9a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5400 0x558c5c9a57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57378/0 (socket says 172.21.15.45:57378) 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 -- 172.21.15.45:0/1766860633 learned_addr learned my addr 172.21.15.45:0/1766860633 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc4137fe700 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc4137fe700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc3fc0a1710 0x7fc3fc0a3bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00f590 tx=0x7fc40c009920 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc4127fc700 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5d20 0x558c5c9b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 -- 172.21.15.45:0/1766860633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 msgr2=0x558c5c9b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 -- 172.21.15.45:0/1766860633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5d20 msgr2=0x558c5c9b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5d20 0x558c5c9b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 -- 172.21.15.45:0/1766860633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c5c95b8d0 con 0x558c5c9a5400 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc4137fe700 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc4127fc700 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5d20 0x558c5c9b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc412ffd700 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5400 0x558c5c9a57e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc404009f90 tx=0x7fc404011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3564df120d4f6b45 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:34.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.190+0000 7fc3f9ffb700 1 -- 172.21.15.45:0/1766860633 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404004310 con 0x558c5c9a5400 2023-09-23T09:40:34.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.191+0000 7fc3f9ffb700 1 -- 172.21.15.45:0/1766860633 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc404004470 con 0x558c5c9a5400 2023-09-23T09:40:34.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.191+0000 7fc3f9ffb700 1 -- 172.21.15.45:0/1766860633 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4040199d0 con 0x558c5c9a5400 2023-09-23T09:40:34.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.191+0000 7fc425b04b80 1 -- 172.21.15.45:0/1766860633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5400 msgr2=0x558c5c9a57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.191+0000 7fc425b04b80 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5400 0x558c5c9a57e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc404009f90 tx=0x7fc404011720 comp rx=0 tx=0).stop 2023-09-23T09:40:34.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 -- 172.21.15.45:0/1766860633 shutdown_connections 2023-09-23T09:40:34.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5400 0x558c5c9a57e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 --2- 172.21.15.45:0/1766860633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5d20 0x558c5c9b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 -- 172.21.15.45:0/1766860633 >> 172.21.15.45:0/1766860633 conn(0x558c5c91a260 msgr2=0x558c5c9a1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 -- 172.21.15.45:0/1766860633 shutdown_connections 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 -- 172.21.15.45:0/1766860633 wait complete. 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.192+0000 7fc425b04b80 1 -- start start 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 1 Processor -- start 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5400 0x558c5c9b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c8f8e30 con 0x558c5c9a5400 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c895710 con 0x558c5c9a5d20 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c8953b0 con 0x558c5c9b0bd0 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc425b04b80 10 client.0 _ll_register_callbacks cb 0x558c5c6f2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc4127fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc412ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5400 0x558c5c9b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc4127fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57386/0 (socket says 172.21.15.45:57386) 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.193+0000 7fc4127fc700 1 -- 172.21.15.45:0/2777149156 learned_addr learned my addr 172.21.15.45:0/2777149156 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc4137fe700 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc4127fc700 1 -- 172.21.15.45:0/2777149156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 msgr2=0x558c5c9adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc4127fc700 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc4127fc700 1 -- 172.21.15.45:0/2777149156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5400 msgr2=0x558c5c9b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc4127fc700 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5400 0x558c5c9b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc4127fc700 1 -- 172.21.15.45:0/2777149156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c019460 con 0x558c5c9a5d20 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc412ffd700 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5400 0x558c5c9b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc4127fc700 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad8a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4080057c0 tx=0x7fc40809b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c43faa8b4f6349 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc3f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/2777149156 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40809f050 con 0x558c5c9a5d20 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/2777149156 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc4080a3200 con 0x558c5c9a5d20 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.194+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c5c999250 con 0x558c5c9a5d20 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.195+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/2777149156 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4080b9170 con 0x558c5c9a5d20 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.195+0000 7fc425b04b80 10 client.26399 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.195+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fc404009bc0 con 0x558c5c9a5d20 2023-09-23T09:40:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.195+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/2777149156 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc40807d080 con 0x558c5c9a5d20 2023-09-23T09:40:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.195+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/2777149156 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc40809a3f0 con 0x558c5c9a5d20 2023-09-23T09:40:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.197+0000 7fc425b04b80 10 client.26399 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.197+0000 7fc425b04b80 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.197+0000 7fc425b04b80 20 client.26399 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.197+0000 7fc425b04b80 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5c9c0310 0x558c5c9c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.197+0000 7fc425b04b80 4 client.26399 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.197+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c5c8f1e10 con 0x558c5c9c0310 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.197+0000 7fc412ffd700 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5c9c0310 0x558c5c9c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.198+0000 7fc3f2ffd700 10 client.26399 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.198+0000 7fc412ffd700 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5c9c0310 0x558c5c9c2730 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e7619e45d81587 server_cookie=bb911abf78a48771 in_seq=0 out_seq=0 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/2777149156 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558c5c8f1e10 con 0x558c5c9c0310 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc3f2ffd700 10 client.26399 handle_command_reply: tid=0 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 client.26399 shutdown 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 msgr2=0x558c5c9ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad8a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4080057c0 tx=0x7fc40809b680 comp rx=0 tx=0).stop 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 shutdown_connections 2023-09-23T09:40:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5c9c0310 0x558c5c9c2730 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9b0bd0 0x558c5c9adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc3f2ffd700 0 client.26399 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9a5400 0x558c5c9b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 --2- 172.21.15.45:0/2777149156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad8a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 >> 172.21.15.45:0/2777149156 conn(0x558c5c91a260 msgr2=0x558c5c9a1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 shutdown_connections 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 1 -- 172.21.15.45:0/2777149156 wait complete. 2023-09-23T09:40:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.199+0000 7fc425b04b80 20 client.26399 trim_cache size 0 max 16384 2023-09-23T09:40:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 Processor -- start 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 -- start start 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 0x558c5c9a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c779920 con 0x558c5c9b0bd0 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c9a6650 con 0x558c5c9a5d20 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc425b04b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c9a6e90 con 0x558c5c9a5400 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc4127fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36044/0 (socket says 172.21.15.45:36044) 2023-09-23T09:40:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.225+0000 7fc4137fe700 1 -- 172.21.15.45:0/3202727161 learned_addr learned my addr 172.21.15.45:0/3202727161 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.226+0000 7fc4137fe700 1 -- 172.21.15.45:0/3202727161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 msgr2=0x558c5c9a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.226+0000 7fc4137fe700 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 0x558c5c9a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.226+0000 7fc4137fe700 1 -- 172.21.15.45:0/3202727161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 msgr2=0x558c5c9ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.226+0000 7fc4137fe700 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.226+0000 7fc4137fe700 1 -- 172.21.15.45:0/3202727161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c5c723de0 con 0x558c5c9b0bd0 2023-09-23T09:40:34.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.226+0000 7fc4127fc700 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:34.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.226+0000 7fc4137fe700 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9ad600 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0199f0 tx=0x7fc40c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dedb14268a3e52 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:34.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.228+0000 7fc3fa7fc700 1 -- 172.21.15.45:0/3202727161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00e040 con 0x558c5c9b0bd0 2023-09-23T09:40:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.228+0000 7fc3fa7fc700 1 -- 172.21.15.45:0/3202727161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc40c00f040 con 0x558c5c9b0bd0 2023-09-23T09:40:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.228+0000 7fc3fa7fc700 1 -- 172.21.15.45:0/3202727161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c017d30 con 0x558c5c9b0bd0 2023-09-23T09:40:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 -- 172.21.15.45:0/3202727161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 msgr2=0x558c5c9ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9ad600 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0199f0 tx=0x7fc40c01c000 comp rx=0 tx=0).stop 2023-09-23T09:40:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 -- 172.21.15.45:0/3202727161 shutdown_connections 2023-09-23T09:40:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9ad600 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 0x558c5c9a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 --2- 172.21.15.45:0/3202727161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 -- 172.21.15.45:0/3202727161 >> 172.21.15.45:0/3202727161 conn(0x558c5c91a260 msgr2=0x558c5c9a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 -- 172.21.15.45:0/3202727161 shutdown_connections 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 -- 172.21.15.45:0/3202727161 wait complete. 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 -- start start 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.229+0000 7fc425b04b80 1 Processor -- start 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 0x558c5c9b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc425b04b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc425b04b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c9b6410 con 0x558c5c9b0bd0 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc425b04b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c9b6580 con 0x558c5c9a5d20 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc425b04b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5c9b6740 con 0x558c5c9a5400 2023-09-23T09:40:34.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc425b04b80 10 client.0 _ll_register_callbacks cb 0x7fc40809aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36052/0 (socket says 172.21.15.45:36052) 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4137fe700 1 -- 172.21.15.45:0/216792347 learned_addr learned my addr 172.21.15.45:0/216792347 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc412ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 0x558c5c9b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4127fc700 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4137fe700 1 -- 172.21.15.45:0/216792347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 msgr2=0x558c5c9b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4137fe700 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 0x558c5c9b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4137fe700 1 -- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 msgr2=0x558c5c9b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.230+0000 7fc4137fe700 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.231+0000 7fc4137fe700 1 -- 172.21.15.45:0/216792347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc404009bc0 con 0x558c5c9b0bd0 2023-09-23T09:40:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.231+0000 7fc4137fe700 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b5e80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009fd0 tx=0x7fc40c018b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4054248d27080057 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.231+0000 7fc3f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.232+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/216792347 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00e040 con 0x558c5c9b0bd0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.232+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/216792347 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc40c034660 con 0x558c5c9b0bd0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.232+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc404004470 con 0x558c5c9b0bd0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.232+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/216792347 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc40c034820 con 0x558c5c9b0bd0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.232+0000 7fc425b04b80 10 client.16719 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.232+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fc40c0196a0 con 0x558c5c9b0bd0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/216792347 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc404004470 con 0x558c5c9b0bd0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/216792347 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc40c005530 con 0x558c5c9b0bd0 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc425b04b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc425b04b80 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc425b04b80 20 client.16719 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc425b04b80 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5ca15570 0x558c5ca17990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc425b04b80 4 client.16719 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:40:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c5c8f1b30 con 0x558c5ca15570 2023-09-23T09:40:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc412ffd700 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5ca15570 0x558c5ca17990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc3f2ffd700 10 client.16719 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.233+0000 7fc412ffd700 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5ca15570 0x558c5ca17990 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bebf8ef92bf388 server_cookie=bb758482d2ca2458 in_seq=0 out_seq=0 2023-09-23T09:40:34.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.235+0000 7fc3f2ffd700 1 -- 172.21.15.45:0/216792347 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558c5c8f1b30 con 0x558c5ca15570 2023-09-23T09:40:34.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.235+0000 7fc3f2ffd700 10 client.16719 handle_command_reply: tid=0 2023-09-23T09:40:34.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.235+0000 7fc425b04b80 1 client.16719 shutdown 2023-09-23T09:40:34.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 msgr2=0x558c5c9b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b5e80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009fd0 tx=0x7fc40c018b30 comp rx=0 tx=0).stop 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 shutdown_connections 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558c5ca15570 0x558c5ca17990 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c5c9b0bd0 0x558c5c9b5e80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc3f2ffd700 0 client.16719 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c5c9a5400 0x558c5c9b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 --2- 172.21.15.45:0/216792347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c5c9a5d20 0x558c5c9b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 >> 172.21.15.45:0/216792347 conn(0x558c5c91a260 msgr2=0x558c5c9af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 shutdown_connections 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 1 -- 172.21.15.45:0/216792347 wait complete. 2023-09-23T09:40:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc425b04b80 20 client.16719 trim_cache size 0 max 16384 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:40:34.240 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:40:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc3fc0a1710 msgr2=0x7fc3fc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc3fc0a1710 0x7fc3fc0a3bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00f590 tx=0x7fc40c009920 comp rx=0 tx=0).stop 2023-09-23T09:40:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 msgr2=0x7fc41410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410a1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc408009230 tx=0x7fc40800fa00 comp rx=0 tx=0).stop 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 shutdown_connections 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41410b360 0x7fc4141ba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fc3fc0a1710 0x7fc3fc0a3bd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc414101f90 0x7fc414109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 --2- 172.21.15.45:0/510937313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4141028b0 0x7fc41410a1d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.236+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 >> 172.21.15.45:0/510937313 conn(0x7fc4140fb770 msgr2=0x7fc41410d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.237+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 shutdown_connections 2023-09-23T09:40:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:40:34.237+0000 7fc41991a700 1 -- 172.21.15.45:0/510937313 wait complete. 2023-09-23T09:40:34.253 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:40:35.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:35 smithi098 ceph-mon[102386]: pgmap v348: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 27 MiB/s wr, 1.98k op/s 2023-09-23T09:40:35.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:35 smithi045 ceph-mon[98627]: pgmap v348: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 27 MiB/s wr, 1.98k op/s 2023-09-23T09:40:35.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:35 smithi123 ceph-mon[103577]: pgmap v348: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 27 MiB/s wr, 1.98k op/s 2023-09-23T09:40:36.889 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 650.0 s, 1115.3 tps, lat 92.419 ms stddev 112.671 2023-09-23T09:40:38.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:37 smithi098 ceph-mon[102386]: pgmap v349: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-09-23T09:40:38.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:37 smithi045 ceph-mon[98627]: pgmap v349: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-09-23T09:40:38.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:37 smithi123 ceph-mon[103577]: pgmap v349: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-09-23T09:40:40.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:39 smithi098 ceph-mon[102386]: pgmap v350: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 1.72k op/s 2023-09-23T09:40:40.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:39 smithi045 ceph-mon[98627]: pgmap v350: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 1.72k op/s 2023-09-23T09:40:40.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:39 smithi123 ceph-mon[103577]: pgmap v350: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 1.72k op/s 2023-09-23T09:40:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.344+0000 7fb5118ab700 1 -- 172.21.15.45:0/2129859739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c10f9e0 msgr2=0x7fb50c108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.344+0000 7fb5118ab700 1 --2- 172.21.15.45:0/2129859739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c10f9e0 0x7fb50c108360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb500009990 tx=0x7fb500009ca0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.345+0000 7fb5118ab700 1 -- 172.21.15.45:0/2129859739 shutdown_connections 2023-09-23T09:40:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.345+0000 7fb5118ab700 1 --2- 172.21.15.45:0/2129859739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c11a600 0x7fb50c11c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.345+0000 7fb5118ab700 1 --2- 172.21.15.45:0/2129859739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb50c10f570 0x7fb50c107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.345+0000 7fb5118ab700 1 --2- 172.21.15.45:0/2129859739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c10f9e0 0x7fb50c108360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.345+0000 7fb5118ab700 1 -- 172.21.15.45:0/2129859739 >> 172.21.15.45:0/2129859739 conn(0x7fb50c06cee0 msgr2=0x7fb50c06d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.345+0000 7fb5118ab700 1 -- 172.21.15.45:0/2129859739 shutdown_connections 2023-09-23T09:40:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.346+0000 7fb5118ab700 1 -- 172.21.15.45:0/2129859739 wait complete. 2023-09-23T09:40:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.346+0000 7fb5118ab700 1 Processor -- start 2023-09-23T09:40:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.347+0000 7fb5118ab700 1 -- start start 2023-09-23T09:40:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.347+0000 7fb5118ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c10f570 0x7fb50c1188e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.347+0000 7fb5118ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb50c10f9e0 0x7fb50c118e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb5118ab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c11a600 0x7fb50c1116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb5118ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c111e80 con 0x7fb50c10f9e0 2023-09-23T09:40:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb5118ab700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c111ff0 con 0x7fb50c11a600 2023-09-23T09:40:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb5118ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c1121b0 con 0x7fb50c10f570 2023-09-23T09:40:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c10f570 0x7fb50c1188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c10f570 0x7fb50c1188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38166/0 (socket says 172.21.15.45:38166) 2023-09-23T09:40:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 -- 172.21.15.45:0/4258985241 learned_addr learned my addr 172.21.15.45:0/4258985241 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c11a600 0x7fb50c1116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb50c10f9e0 0x7fb50c118e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 -- 172.21.15.45:0/4258985241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c11a600 msgr2=0x7fb50c1116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c11a600 0x7fb50c1116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 -- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb50c10f9e0 msgr2=0x7fb50c118e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb50c10f9e0 0x7fb50c118e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.348+0000 7fb50affd700 1 -- 172.21.15.45:0/4258985241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5000095f0 con 0x7fb50c10f570 2023-09-23T09:40:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.349+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb50c10f9e0 0x7fb50c118e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:40:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.349+0000 7fb50affd700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c10f570 0x7fb50c1188e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb5040060b0 tx=0x7fb50400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.349+0000 7fb5108a9700 1 -- 172.21.15.45:0/4258985241 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50400ddd0 con 0x7fb50c10f570 2023-09-23T09:40:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.349+0000 7fb5108a9700 1 -- 172.21.15.45:0/4258985241 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb50400f040 con 0x7fb50c10f570 2023-09-23T09:40:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.349+0000 7fb5108a9700 1 -- 172.21.15.45:0/4258985241 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504014690 con 0x7fb50c10f570 2023-09-23T09:40:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.349+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c112490 con 0x7fb50c10f570 2023-09-23T09:40:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.349+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c116710 con 0x7fb50c10f570 2023-09-23T09:40:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 Processor -- start 2023-09-23T09:40:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 -- start start 2023-09-23T09:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635a5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635b0710 0x5639635b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396350e400 con 0x5639635a4f40 2023-09-23T09:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639634ffbc0 con 0x5639635b0710 2023-09-23T09:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.354+0000 7fb51da95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639634fd4e0 con 0x5639635a5860 2023-09-23T09:40:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.356+0000 7fb5108a9700 1 -- 172.21.15.45:0/4258985241 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fb504015400 con 0x7fb50c10f570 2023-09-23T09:40:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.357+0000 7fb5108a9700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb4f40a17d0 0x7fb4f40a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.357+0000 7fb5108a9700 1 -- 172.21.15.45:0/4258985241 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb5040ccdb0 con 0x7fb50c10f570 2023-09-23T09:40:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.357+0000 7fb50a7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635b0710 0x5639635b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.357+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.357+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36068/0 (socket says 172.21.15.45:36068) 2023-09-23T09:40:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.357+0000 7fb50b7fe700 1 -- 172.21.15.45:0/2189242251 learned_addr learned my addr 172.21.15.45:0/2189242251 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb4f40a17d0 0x7fb4f40a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 -- 172.21.15.45:0/2189242251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 msgr2=0x5639635b01d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:40:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/2189242251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b01d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 -- 172.21.15.45:0/2189242251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635b0710 msgr2=0x5639635b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/2189242251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635b0710 0x5639635b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 -- 172.21.15.45:0/2189242251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396355b870 con 0x5639635a4f40 2023-09-23T09:40:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/2189242251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635a5320 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005950 tx=0x7fb4fc013db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de03d4c4230f272 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.358+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb4f40a17d0 0x7fb4f40a3c90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00f6a0 tx=0x7fb4fc01f000 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.359+0000 7fb4f1ffb700 1 -- 172.21.15.45:0/2189242251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00fe90 con 0x5639635a4f40 2023-09-23T09:40:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.359+0000 7fb4f1ffb700 1 -- 172.21.15.45:0/2189242251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4fc009e20 con 0x5639635a4f40 2023-09-23T09:40:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.359+0000 7fb4f1ffb700 1 -- 172.21.15.45:0/2189242251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004ab0 con 0x5639635a4f40 2023-09-23T09:40:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.360+0000 7fb51da95b80 1 -- 172.21.15.45:0/2189242251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 msgr2=0x5639635a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.360+0000 7fb51da95b80 1 --2- 172.21.15.45:0/2189242251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635a5320 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005950 tx=0x7fb4fc013db0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 -- 172.21.15.45:0/2189242251 shutdown_connections 2023-09-23T09:40:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 --2- 172.21.15.45:0/2189242251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635b0710 0x5639635b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 --2- 172.21.15.45:0/2189242251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635a5320 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 --2- 172.21.15.45:0/2189242251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 -- 172.21.15.45:0/2189242251 >> 172.21.15.45:0/2189242251 conn(0x56396351a200 msgr2=0x5639635a13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 -- 172.21.15.45:0/2189242251 shutdown_connections 2023-09-23T09:40:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 -- 172.21.15.45:0/2189242251 wait complete. 2023-09-23T09:40:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.361+0000 7fb51da95b80 1 -- start start 2023-09-23T09:40:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 1 Processor -- start 2023-09-23T09:40:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635b0710 0x5639635ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639634f8dd0 con 0x5639635a4f40 2023-09-23T09:40:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639634956b0 con 0x5639635a5860 2023-09-23T09:40:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963495350 con 0x5639635b0710 2023-09-23T09:40:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb51da95b80 10 client.0 _ll_register_callbacks cb 0x5639632f27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635b0710 0x5639635ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635b0710 0x5639635ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38180/0 (socket says 172.21.15.45:38180) 2023-09-23T09:40:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.362+0000 7fb50b7fe700 1 -- 172.21.15.45:0/1924581014 learned_addr learned my addr 172.21.15.45:0/1924581014 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50b7fe700 1 -- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 msgr2=0x5639635ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50b7fe700 1 -- 172.21.15.45:0/1924581014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 msgr2=0x5639635b0110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:40:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635b0110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50b7fe700 1 -- 172.21.15.45:0/1924581014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc01b040 con 0x5639635b0710 2023-09-23T09:40:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:40:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635b0710 0x5639635ada80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009d80 tx=0x7fb4fc00ff40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64893d1b4a1a6d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb4eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:40:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb4eaffd700 1 -- 172.21.15.45:0/1924581014 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004020 con 0x5639635b0710 2023-09-23T09:40:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb4eaffd700 1 -- 172.21.15.45:0/1924581014 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4fc004180 con 0x5639635b0710 2023-09-23T09:40:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.363+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563963599090 con 0x5639635b0710 2023-09-23T09:40:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb4eaffd700 1 -- 172.21.15.45:0/1924581014 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4fc004340 con 0x5639635b0710 2023-09-23T09:40:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 10 client.26389 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x563963323d80 con 0x5639635b0710 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb4eaffd700 1 -- 172.21.15.45:0/1924581014 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563963599090 con 0x5639635b0710 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb4eaffd700 1 -- 172.21.15.45:0/1924581014 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb4fc025070 con 0x5639635b0710 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 10 client.26389 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 10 client.26389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 20 client.26389 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5639635c0220 0x5639635c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 4 client.26389 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:40:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.364+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639634f1db0 con 0x5639635c0220 2023-09-23T09:40:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.367+0000 7fb50affd700 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5639635c0220 0x5639635c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.367+0000 7fb4eaffd700 10 client.26389 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.367+0000 7fb50affd700 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5639635c0220 0x5639635c2660 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d6a4ff7a7e40b6 server_cookie=c5ce2cf62e9e3c7f in_seq=0 out_seq=0 2023-09-23T09:40:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb4eaffd700 1 -- 172.21.15.45:0/1924581014 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5639634f1db0 con 0x5639635c0220 2023-09-23T09:40:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb4eaffd700 10 client.26389 handle_command_reply: tid=0 2023-09-23T09:40:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 client.26389 shutdown 2023-09-23T09:40:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635b0710 msgr2=0x5639635ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635b0710 0x5639635ada80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009d80 tx=0x7fb4fc00ff40 comp rx=0 tx=0).stop 2023-09-23T09:40:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 shutdown_connections 2023-09-23T09:40:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5639635c0220 0x5639635c2660 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635b0710 0x5639635ada80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635a4f40 0x5639635b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb4eaffd700 0 client.26389 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1924581014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 >> 172.21.15.45:0/1924581014 conn(0x56396351a200 msgr2=0x5639635a0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.368+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 shutdown_connections 2023-09-23T09:40:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.369+0000 7fb51da95b80 1 -- 172.21.15.45:0/1924581014 wait complete. 2023-09-23T09:40:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.369+0000 7fb51da95b80 20 client.26389 trim_cache size 0 max 16384 2023-09-23T09:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.387+0000 7fb51da95b80 1 Processor -- start 2023-09-23T09:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.387+0000 7fb51da95b80 1 -- start start 2023-09-23T09:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.387+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a4f40 0x5639635afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.387+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639635ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb51da95b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639633798c0 con 0x5639635b0710 2023-09-23T09:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb51da95b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639635bcb70 con 0x5639635a5860 2023-09-23T09:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb51da95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963598f80 con 0x5639635a4f40 2023-09-23T09:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a4f40 0x5639635afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639635ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a4f40 0x5639635afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38186/0 (socket says 172.21.15.45:38186) 2023-09-23T09:40:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639635ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36084/0 (socket says 172.21.15.45:36084) 2023-09-23T09:40:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 -- 172.21.15.45:0/1374994747 learned_addr learned my addr 172.21.15.45:0/1374994747 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 -- 172.21.15.45:0/1374994747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 msgr2=0x5639635acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 -- 172.21.15.45:0/1374994747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 msgr2=0x5639635ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639635ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 -- 172.21.15.45:0/1374994747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396330cc00 con 0x5639635a4f40 2023-09-23T09:40:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb50affd700 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a4f40 0x5639635afcb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb504096180 tx=0x7fb50400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667d982fe52ffd9a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.388+0000 7fb4f27fc700 1 -- 172.21.15.45:0/1374994747 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50409a380 con 0x5639635a4f40 2023-09-23T09:40:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.389+0000 7fb4f27fc700 1 -- 172.21.15.45:0/1374994747 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb504099c80 con 0x5639635a4f40 2023-09-23T09:40:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.389+0000 7fb4f27fc700 1 -- 172.21.15.45:0/1374994747 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504015a90 con 0x5639635a4f40 2023-09-23T09:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 -- 172.21.15.45:0/1374994747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a4f40 msgr2=0x5639635afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a4f40 0x5639635afcb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb504096180 tx=0x7fb50400b040 comp rx=0 tx=0).stop 2023-09-23T09:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 -- 172.21.15.45:0/1374994747 shutdown_connections 2023-09-23T09:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639635ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a4f40 0x5639635afcb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 --2- 172.21.15.45:0/1374994747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a5860 0x5639635acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 -- 172.21.15.45:0/1374994747 >> 172.21.15.45:0/1374994747 conn(0x56396351a200 msgr2=0x5639635bc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 -- 172.21.15.45:0/1374994747 shutdown_connections 2023-09-23T09:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 -- 172.21.15.45:0/1374994747 wait complete. 2023-09-23T09:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 -- start start 2023-09-23T09:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.390+0000 7fb51da95b80 1 Processor -- start 2023-09-23T09:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a4f40 0x5639635b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb51da95b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639636130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb51da95b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639636135f0 con 0x5639635b0710 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb51da95b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963613730 con 0x5639635a4f40 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb51da95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963613870 con 0x5639635a5860 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb51da95b80 10 client.0 _ll_register_callbacks cb 0x7fb4fc0138d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38194/0 (socket says 172.21.15.45:38194) 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 -- 172.21.15.45:0/3294617753 learned_addr learned my addr 172.21.15.45:0/3294617753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639636130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 -- 172.21.15.45:0/3294617753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a4f40 msgr2=0x5639635b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a4f40 0x5639635b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 -- 172.21.15.45:0/3294617753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 msgr2=0x5639636130b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639636130b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50a7fc700 1 -- 172.21.15.45:0/3294617753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c116710 con 0x5639635a5860 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.391+0000 7fb50b7fe700 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639636130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb50a7fc700 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b5940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb50c06e660 tx=0x7fb500009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc26dfc947c4a187 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb4eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb4eaffd700 1 -- 172.21.15.45:0/3294617753 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500004500 con 0x5639635a5860 2023-09-23T09:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb4eaffd700 1 -- 172.21.15.45:0/3294617753 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb500004660 con 0x5639635a5860 2023-09-23T09:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4fc009e20 con 0x5639635a5860 2023-09-23T09:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb4eaffd700 1 -- 172.21.15.45:0/3294617753 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb500004820 con 0x5639635a5860 2023-09-23T09:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 10 client.26395 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56396334a1f0 con 0x5639635a5860 2023-09-23T09:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb4eaffd700 1 -- 172.21.15.45:0/3294617753 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4fc009e20 con 0x5639635a5860 2023-09-23T09:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb4eaffd700 1 -- 172.21.15.45:0/3294617753 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb500022040 con 0x5639635a5860 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 10 client.26395 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 10 client.26395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 20 client.26395 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563963614ff0 0x563963617410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 4 client.26395 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.392+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639634f1ad0 con 0x563963614ff0 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.393+0000 7fb50affd700 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563963614ff0 0x563963617410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.393+0000 7fb4eaffd700 10 client.26395 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.393+0000 7fb50affd700 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563963614ff0 0x563963617410 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be6de7b3496e711 server_cookie=4ee03e5e8cbdba52 in_seq=0 out_seq=0 2023-09-23T09:40:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.410+0000 7fb4eaffd700 1 -- 172.21.15.45:0/3294617753 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639634f1ad0 con 0x563963614ff0 2023-09-23T09:40:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.410+0000 7fb4eaffd700 10 client.26395 handle_command_reply: tid=0 2023-09-23T09:40:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.410+0000 7fb51da95b80 1 client.26395 shutdown 2023-09-23T09:40:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 msgr2=0x5639635b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b5940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb50c06e660 tx=0x7fb500009530 comp rx=0 tx=0).stop 2023-09-23T09:40:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 shutdown_connections 2023-09-23T09:40:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x563963614ff0 0x563963617410 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639635b0710 0x5639636130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5639635a4f40 0x5639635b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 --2- 172.21.15.45:0/3294617753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639635a5860 0x5639635b5940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 >> 172.21.15.45:0/3294617753 conn(0x56396351a200 msgr2=0x5639635a0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb4eaffd700 0 client.26395 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 shutdown_connections 2023-09-23T09:40:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 1 -- 172.21.15.45:0/3294617753 wait complete. 2023-09-23T09:40:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.411+0000 7fb51da95b80 20 client.26395 trim_cache size 0 max 16384 2023-09-23T09:40:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:40:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:40:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:40:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb4f40a17d0 msgr2=0x7fb4f40a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb4f40a17d0 0x7fb4f40a3c90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00f6a0 tx=0x7fb4fc01f000 comp rx=0 tx=0).stop 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c10f570 msgr2=0x7fb50c1188e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c10f570 0x7fb50c1188e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb5040060b0 tx=0x7fb50400d3b0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 shutdown_connections 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb50c11a600 0x7fb50c1116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb50c10f570 0x7fb50c1188e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fb4f40a17d0 0x7fb4f40a3c90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 --2- 172.21.15.45:0/4258985241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb50c10f9e0 0x7fb50c118e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 >> 172.21.15.45:0/4258985241 conn(0x7fb50c06cee0 msgr2=0x7fb50c06f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 shutdown_connections 2023-09-23T09:40:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:40.412+0000 7fb5118ab700 1 -- 172.21.15.45:0/4258985241 wait complete. 2023-09-23T09:40:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 655.0 s, 1378.3 tps, lat 72.099 ms stddev 84.762 2023-09-23T09:40:42.051 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:41 smithi098 ceph-mon[102386]: pgmap v351: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 2.15k op/s 2023-09-23T09:40:42.087 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:41 smithi045 ceph-mon[98627]: pgmap v351: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 2.15k op/s 2023-09-23T09:40:42.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:41 smithi123 ceph-mon[103577]: pgmap v351: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 2.15k op/s 2023-09-23T09:40:44.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:43 smithi098 ceph-mon[102386]: pgmap v352: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-23T09:40:44.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:43 smithi045 ceph-mon[98627]: pgmap v352: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-23T09:40:44.091 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:43 smithi123 ceph-mon[103577]: pgmap v352: 97 pgs: 97 active+clean; 6.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-23T09:40:46.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:45 smithi098 ceph-mon[102386]: pgmap v353: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 33 MiB/s wr, 2.30k op/s 2023-09-23T09:40:46.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:45 smithi045 ceph-mon[98627]: pgmap v353: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 33 MiB/s wr, 2.30k op/s 2023-09-23T09:40:46.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:45 smithi123 ceph-mon[103577]: pgmap v353: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 33 MiB/s wr, 2.30k op/s 2023-09-23T09:40:46.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 660.0 s, 1570.3 tps, lat 63.576 ms stddev 76.396 2023-09-23T09:40:48.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:47 smithi045 ceph-mon[98627]: pgmap v354: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 908 op/s 2023-09-23T09:40:48.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:47 smithi098 ceph-mon[102386]: pgmap v354: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 908 op/s 2023-09-23T09:40:48.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:47 smithi123 ceph-mon[103577]: pgmap v354: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 908 op/s 2023-09-23T09:40:50.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:49 smithi098 ceph-mon[102386]: pgmap v355: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 933 op/s 2023-09-23T09:40:50.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:49 smithi045 ceph-mon[98627]: pgmap v355: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 933 op/s 2023-09-23T09:40:50.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:49 smithi123 ceph-mon[103577]: pgmap v355: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 933 op/s 2023-09-23T09:40:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 665.0 s, 1154.1 tps, lat 87.438 ms stddev 133.925 2023-09-23T09:40:52.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:51 smithi098 ceph-mon[102386]: pgmap v356: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 1.55k op/s 2023-09-23T09:40:52.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:51 smithi045 ceph-mon[98627]: pgmap v356: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 1.55k op/s 2023-09-23T09:40:52.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:51 smithi123 ceph-mon[103577]: pgmap v356: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 1.55k op/s 2023-09-23T09:40:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.613+0000 7fbc02db1700 1 -- 172.21.15.45:0/1806143289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc112d70 msgr2=0x7fbbfc115160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.613+0000 7fbc02db1700 1 --2- 172.21.15.45:0/1806143289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc112d70 0x7fbbfc115160 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400b0a0 tx=0x7fbbf400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.614+0000 7fbc02db1700 1 -- 172.21.15.45:0/1806143289 shutdown_connections 2023-09-23T09:40:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.614+0000 7fbc02db1700 1 --2- 172.21.15.45:0/1806143289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc112d70 0x7fbbfc115160 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.614+0000 7fbc02db1700 1 --2- 172.21.15.45:0/1806143289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1097c0 0x7fbbfc109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.614+0000 7fbc02db1700 1 --2- 172.21.15.45:0/1806143289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbfc10a0e0 0x7fbbfc112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.614+0000 7fbc02db1700 1 -- 172.21.15.45:0/1806143289 >> 172.21.15.45:0/1806143289 conn(0x7fbbfc06cee0 msgr2=0x7fbbfc06d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.614+0000 7fbc02db1700 1 -- 172.21.15.45:0/1806143289 shutdown_connections 2023-09-23T09:40:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.614+0000 7fbc02db1700 1 -- 172.21.15.45:0/1806143289 wait complete. 2023-09-23T09:40:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 Processor -- start 2023-09-23T09:40:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 -- start start 2023-09-23T09:40:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbfc1097c0 0x7fbbfc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc10a0e0 0x7fbbfc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc112d70 0x7fbbfc1a3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1a3a30 con 0x7fbbfc10a0e0 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1a3ba0 con 0x7fbbfc1097c0 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc02db1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1a3d60 con 0x7fbbfc112d70 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc10a0e0 0x7fbbfc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc00b4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbfc1097c0 0x7fbbfc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc00b4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbfc1097c0 0x7fbbfc19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57450/0 (socket says 172.21.15.45:57450) 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc00b4d700 1 -- 172.21.15.45:0/556603690 learned_addr learned my addr 172.21.15.45:0/556603690 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc10a0e0 0x7fbbfc19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36102/0 (socket says 172.21.15.45:36102) 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.615+0000 7fbc0134e700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc112d70 0x7fbbfc1a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.616+0000 7fbc0134e700 1 -- 172.21.15.45:0/556603690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbfc1097c0 msgr2=0x7fbbfc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.616+0000 7fbc0134e700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbfc1097c0 0x7fbbfc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.616+0000 7fbc0134e700 1 -- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc10a0e0 msgr2=0x7fbbfc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.616+0000 7fbc0134e700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc10a0e0 0x7fbbfc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.616+0000 7fbc0134e700 1 -- 172.21.15.45:0/556603690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8009520 con 0x7fbbfc112d70 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.616+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc10a0e0 0x7fbbfc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:40:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.617+0000 7fbc0134e700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc112d70 0x7fbbfc1a3430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400b070 tx=0x7fbbf400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.617+0000 7fbbf9ffb700 1 -- 172.21.15.45:0/556603690 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400e040 con 0x7fbbfc112d70 2023-09-23T09:40:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.617+0000 7fbbf9ffb700 1 -- 172.21.15.45:0/556603690 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf400be90 con 0x7fbbfc112d70 2023-09-23T09:40:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.617+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf4009d00 con 0x7fbbfc112d70 2023-09-23T09:40:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.617+0000 7fbbf9ffb700 1 -- 172.21.15.45:0/556603690 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf401b460 con 0x7fbbfc112d70 2023-09-23T09:40:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.617+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc1a4270 con 0x7fbbfc112d70 2023-09-23T09:40:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.622+0000 7fbc0ef9bb80 1 Processor -- start 2023-09-23T09:40:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.634+0000 7fbc0ef9bb80 1 -- start start 2023-09-23T09:40:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.634+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dbf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.634+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbf860 0x556e92dca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.634+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92dccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.634+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92d28400 con 0x556e92dca710 2023-09-23T09:40:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.634+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92d19bc0 con 0x556e92dbf860 2023-09-23T09:40:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.634+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92d174e0 con 0x556e92dbef40 2023-09-23T09:40:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.637+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92dccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.637+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92dccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36110/0 (socket says 172.21.15.45:36110) 2023-09-23T09:40:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.637+0000 7fbbfbfff700 1 -- 172.21.15.45:0/611864905 learned_addr learned my addr 172.21.15.45:0/611864905 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbf860 0x556e92dca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dbf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbbfbfff700 1 -- 172.21.15.45:0/611864905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbf860 msgr2=0x556e92dca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbf860 0x556e92dca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbbfbfff700 1 -- 172.21.15.45:0/611864905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 msgr2=0x556e92dccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92dccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbbfbfff700 1 -- 172.21.15.45:0/611864905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e92d75870 con 0x556e92dbef40 2023-09-23T09:40:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.646+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dbf320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8013db0 tx=0x7fbbe8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=692ad2d250441ad2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.647+0000 7fbbe6ffd700 1 -- 172.21.15.45:0/611864905 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8013ea0 con 0x556e92dbef40 2023-09-23T09:40:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.647+0000 7fbbe6ffd700 1 -- 172.21.15.45:0/611864905 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe8013a70 con 0x556e92dbef40 2023-09-23T09:40:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.647+0000 7fbbe6ffd700 1 -- 172.21.15.45:0/611864905 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800ec00 con 0x556e92dbef40 2023-09-23T09:40:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.648+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/611864905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 msgr2=0x556e92dbf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.648+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dbf320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8013db0 tx=0x7fbbe8009260 comp rx=0 tx=0).stop 2023-09-23T09:40:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.651+0000 7fbbf9ffb700 1 -- 172.21.15.45:0/556603690 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fbbf401d470 con 0x7fbbfc112d70 2023-09-23T09:40:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/611864905 shutdown_connections 2023-09-23T09:40:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92dccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dbf320 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/611864905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbf860 0x556e92dca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/611864905 >> 172.21.15.45:0/611864905 conn(0x556e92d34200 msgr2=0x556e92dbb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbbf9ffb700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbbec0a18a0 0x7fbbec0a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/611864905 shutdown_connections 2023-09-23T09:40:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/611864905 wait complete. 2023-09-23T09:40:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbbf9ffb700 1 -- 172.21.15.45:0/556603690 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbbf40cdaf0 con 0x7fbbfc112d70 2023-09-23T09:40:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.652+0000 7fbc0ef9bb80 1 -- start start 2023-09-23T09:40:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.653+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbbec0a18a0 0x7fbbec0a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.653+0000 7fbc0ef9bb80 1 Processor -- start 2023-09-23T09:40:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.654+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbbec0a18a0 0x7fbbec0a3d60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009e20 tx=0x7fbbf0009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.661+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbef40 0x556e92dca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.661+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dc73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.661+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 0x556e92dc7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.661+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92d12dd0 con 0x556e92dbef40 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.661+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92caf6b0 con 0x556e92dca710 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.661+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92caf350 con 0x556e92dbf860 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.661+0000 7fbc0ef9bb80 10 client.0 _ll_register_callbacks cb 0x556e92b0c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dc73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dc73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38220/0 (socket says 172.21.15.45:38220) 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbbfbfff700 1 -- 172.21.15.45:0/1942037728 learned_addr learned my addr 172.21.15.45:0/1942037728 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbef40 0x556e92dca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbc00b4d700 1 -- 172.21.15.45:0/1942037728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 msgr2=0x556e92dc73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dc73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbc00b4d700 1 -- 172.21.15.45:0/1942037728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 msgr2=0x556e92dc7b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:40:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 0x556e92dc7b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbc00b4d700 1 -- 172.21.15.45:0/1942037728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe801b040 con 0x556e92dbef40 2023-09-23T09:40:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.662+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbef40 0x556e92dca140 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0012fb0 tx=0x7fbbf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9935911232bacf8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.664+0000 7fbbdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:40:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.665+0000 7fbbdbfff700 1 -- 172.21.15.45:0/1942037728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0017070 con 0x556e92dbef40 2023-09-23T09:40:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.665+0000 7fbbdbfff700 1 -- 172.21.15.45:0/1942037728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbbf004b430 con 0x556e92dbef40 2023-09-23T09:40:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.667+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e92db3090 con 0x556e92dbef40 2023-09-23T09:40:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.667+0000 7fbbdbfff700 1 -- 172.21.15.45:0/1942037728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf004b5f0 con 0x556e92dbef40 2023-09-23T09:40:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.667+0000 7fbc0ef9bb80 10 client.16743 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.667+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556e92b3dd80 con 0x556e92dbef40 2023-09-23T09:40:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbbdbfff700 1 -- 172.21.15.45:0/1942037728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e92db3090 con 0x556e92dbef40 2023-09-23T09:40:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbbdbfff700 1 -- 172.21.15.45:0/1942037728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbbf0053070 con 0x556e92dbef40 2023-09-23T09:40:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbc0ef9bb80 10 client.16743 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbc0ef9bb80 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbc0ef9bb80 20 client.16743 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92dda2c0 0x556e92ddc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbc0ef9bb80 4 client.16743 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:40:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.668+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e92d0bdb0 con 0x556e92dda2c0 2023-09-23T09:40:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.669+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92dda2c0 0x556e92ddc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.672+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92dda2c0 0x556e92ddc6e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2052037ccf7e5bf0 server_cookie=89293b0fbf28e9c in_seq=0 out_seq=0 2023-09-23T09:40:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.673+0000 7fbbdbfff700 10 client.16743 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.673+0000 7fbbdbfff700 1 -- 172.21.15.45:0/1942037728 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556e92d0bdb0 con 0x556e92dda2c0 2023-09-23T09:40:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.673+0000 7fbbdbfff700 10 client.16743 handle_command_reply: tid=0 2023-09-23T09:40:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.673+0000 7fbc0ef9bb80 1 client.16743 shutdown 2023-09-23T09:40:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.674+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbef40 msgr2=0x556e92dca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.674+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbef40 0x556e92dca140 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0012fb0 tx=0x7fbbf00047c0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.675+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 shutdown_connections 2023-09-23T09:40:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.675+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92dda2c0 0x556e92ddc6e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.675+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 0x556e92dc7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.675+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbef40 0x556e92dca140 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.675+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/1942037728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dc73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.675+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 >> 172.21.15.45:0/1942037728 conn(0x556e92d34200 msgr2=0x556e92dbafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.675+0000 7fbbdbfff700 0 client.16743 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.676+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 shutdown_connections 2023-09-23T09:40:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.677+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/1942037728 wait complete. 2023-09-23T09:40:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.678+0000 7fbc0ef9bb80 20 client.16743 trim_cache size 0 max 16384 2023-09-23T09:40:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.697+0000 7fbc0ef9bb80 1 Processor -- start 2023-09-23T09:40:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.697+0000 7fbc0ef9bb80 1 -- start start 2023-09-23T09:40:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dc7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbf860 0x556e92dc6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 0x556e92dc9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92b938c0 con 0x556e92dbf860 2023-09-23T09:40:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92dbc100 con 0x556e92dca710 2023-09-23T09:40:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92dbc950 con 0x556e92dbef40 2023-09-23T09:40:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc00b4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dc7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc00b4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dc7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38226/0 (socket says 172.21.15.45:38226) 2023-09-23T09:40:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.698+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3536300778 learned_addr learned my addr 172.21.15.45:0/3536300778 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3536300778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 msgr2=0x556e92dc9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/3536300778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 0x556e92dc9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3536300778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbf860 msgr2=0x556e92dc6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/3536300778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbf860 0x556e92dc6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3536300778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e92b26c00 con 0x556e92dbef40 2023-09-23T09:40:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/3536300778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dc7380 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0054560 tx=0x7fbbf0009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f76014cbac78ba7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbbe77fe700 1 -- 172.21.15.45:0/3536300778 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0054d00 con 0x556e92dbef40 2023-09-23T09:40:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbbe77fe700 1 -- 172.21.15.45:0/3536300778 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf0045070 con 0x556e92dbef40 2023-09-23T09:40:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.699+0000 7fbbe77fe700 1 -- 172.21.15.45:0/3536300778 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf005ed40 con 0x556e92dbef40 2023-09-23T09:40:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.700+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3536300778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 msgr2=0x556e92dc7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.700+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3536300778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dc7380 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0054560 tx=0x7fbbf0009b20 comp rx=0 tx=0).stop 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.700+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3536300778 shutdown_connections 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.700+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3536300778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dca710 0x556e92dc9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.700+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3536300778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbef40 0x556e92dc7380 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.700+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3536300778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dbf860 0x556e92dc6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.700+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3536300778 >> 172.21.15.45:0/3536300778 conn(0x556e92d34200 msgr2=0x556e92dd5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.701+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3536300778 shutdown_connections 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.701+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3536300778 wait complete. 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.701+0000 7fbc0ef9bb80 1 -- start start 2023-09-23T09:40:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.701+0000 7fbc0ef9bb80 1 Processor -- start 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbef40 0x556e92dcf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dcf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0ef9bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92e2d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92e2d670 con 0x556e92dca710 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92e2d7b0 con 0x556e92dbef40 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0ef9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e92e2d8f0 con 0x556e92dbf860 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0ef9bb80 10 client.0 _ll_register_callbacks cb 0x7fbbf001e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbef40 0x556e92dcf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbef40 0x556e92dcf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57476/0 (socket says 172.21.15.45:57476) 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3065700693 learned_addr learned my addr 172.21.15.45:0/3065700693 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc0134e700 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92e2d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3065700693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 msgr2=0x556e92dcf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dcf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3065700693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 msgr2=0x556e92e2d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92e2d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.702+0000 7fbc00b4d700 1 -- 172.21.15.45:0/3065700693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e92b3dd80 con 0x556e92dbef40 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbc00b4d700 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbef40 0x556e92dcf0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0019040 tx=0x7fbbf0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fbe052289b862d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbbdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbbdbfff700 1 -- 172.21.15.45:0/3065700693 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0017040 con 0x556e92dbef40 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbbdbfff700 1 -- 172.21.15.45:0/3065700693 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbbf00547d0 con 0x556e92dbef40 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbe8013a70 con 0x556e92dbef40 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbbdbfff700 1 -- 172.21.15.45:0/3065700693 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf00543a0 con 0x556e92dbef40 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbc0ef9bb80 10 client.26426 fetch_fsmap learned FSMap version 46 2023-09-23T09:40:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556e92b641f0 con 0x556e92dbef40 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.703+0000 7fbbdbfff700 1 -- 172.21.15.45:0/3065700693 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf0073080 con 0x556e92dbef40 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbbdbfff700 1 -- 172.21.15.45:0/3065700693 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbbf0065030 con 0x556e92dbef40 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbc0ef9bb80 10 client.26426 fetch_fsmap finished waiting for FSMap version 46 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbc0ef9bb80 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbc0ef9bb80 20 client.26426 populate_metadata read hostname 'smithi045' 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92e2ed40 0x556e92e31160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbc0ef9bb80 4 client.26426 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e92d0bad0 con 0x556e92e2ed40 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92e2ed40 0x556e92e31160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbbfbfff700 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92e2ed40 0x556e92e31160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ffaae49a5afeda server_cookie=69d72f5a357072d3 in_seq=0 out_seq=0 2023-09-23T09:40:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.704+0000 7fbbdbfff700 10 client.26426 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbbdbfff700 1 -- 172.21.15.45:0/3065700693 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e92d0bad0 con 0x556e92e2ed40 2023-09-23T09:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbbdbfff700 10 client.26426 handle_command_reply: tid=0 2023-09-23T09:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 client.26426 shutdown 2023-09-23T09:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbef40 msgr2=0x556e92dcf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbef40 0x556e92dcf0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0019040 tx=0x7fbbf0005270 comp rx=0 tx=0).stop 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 shutdown_connections 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x556e92e2ed40 0x556e92e31160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e92dca710 0x556e92e2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92dbef40 0x556e92dcf0a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 --2- 172.21.15.45:0/3065700693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e92dbf860 0x556e92dcf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbbdbfff700 0 client.26426 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 >> 172.21.15.45:0/3065700693 conn(0x556e92d34200 msgr2=0x556e92dc2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 shutdown_connections 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 1 -- 172.21.15.45:0/3065700693 wait complete. 2023-09-23T09:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.718+0000 7fbc0ef9bb80 20 client.26426 trim_cache size 0 max 16384 2023-09-23T09:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbbec0a18a0 msgr2=0x7fbbec0a3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbbec0a18a0 0x7fbbec0a3d60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009e20 tx=0x7fbbf0009260 comp rx=0 tx=0).stop 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc112d70 msgr2=0x7fbbfc1a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc112d70 0x7fbbfc1a3430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400b070 tx=0x7fbbf400bce0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 shutdown_connections 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc112d70 0x7fbbfc1a3430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbfc1097c0 0x7fbbfc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbbec0a18a0 0x7fbbec0a3d60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 --2- 172.21.15.45:0/556603690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbfc10a0e0 0x7fbbfc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 >> 172.21.15.45:0/556603690 conn(0x7fbbfc06cee0 msgr2=0x7fbbfc06f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:40:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 shutdown_connections 2023-09-23T09:40:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:40:53.719+0000 7fbc02db1700 1 -- 172.21.15.45:0/556603690 wait complete. 2023-09-23T09:40:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:53 smithi098 ceph-mon[102386]: pgmap v357: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-09-23T09:40:54.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:53 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:40:54.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:53 smithi045 ceph-mon[98627]: pgmap v357: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-09-23T09:40:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:53 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:40:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:53 smithi123 ceph-mon[103577]: pgmap v357: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-09-23T09:40:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:53 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:40:56.300 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:55 smithi098 ceph-mon[102386]: pgmap v358: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 1.66k op/s 2023-09-23T09:40:56.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:55 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:40:56.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:55 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:40:56.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:55 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:40:56.819 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:55 smithi123 ceph-mon[103577]: pgmap v358: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 1.66k op/s 2023-09-23T09:40:56.819 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:55 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:40:56.819 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:55 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:40:56.819 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:55 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:40:56.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:55 smithi045 ceph-mon[98627]: pgmap v358: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 1.66k op/s 2023-09-23T09:40:56.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:55 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:40:56.820 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:55 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:40:56.820 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:55 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:40:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 670.0 s, 1647.6 tps, lat 60.561 ms stddev 77.815 2023-09-23T09:40:58.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:57 smithi098 ceph-mon[102386]: pgmap v359: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-09-23T09:41:00.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:57 smithi123 ceph-mon[103577]: pgmap v359: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-09-23T09:41:00.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:57 smithi045 ceph-mon[98627]: pgmap v359: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-09-23T09:41:00.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:40:59 smithi098 ceph-mon[102386]: pgmap v360: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-09-23T09:41:00.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:40:59 smithi045 ceph-mon[98627]: pgmap v360: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-09-23T09:41:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:40:59 smithi123 ceph-mon[103577]: pgmap v360: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-09-23T09:41:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 675.0 s, 1244.0 tps, lat 78.461 ms stddev 114.837 2023-09-23T09:41:02.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:01 smithi098 ceph-mon[102386]: pgmap v361: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:41:02.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:01 smithi045 ceph-mon[98627]: pgmap v361: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:41:02.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:01 smithi123 ceph-mon[103577]: pgmap v361: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-23T09:41:04.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:41:04.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:03 smithi123 ceph-mon[103577]: pgmap v362: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 870 op/s 2023-09-23T09:41:04.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:03 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:04.958 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:03 smithi045 ceph-mon[98627]: pgmap v362: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 870 op/s 2023-09-23T09:41:04.958 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:03 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:04.958 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:03 smithi098 ceph-mon[102386]: pgmap v362: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 870 op/s 2023-09-23T09:41:04.959 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:03 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:05.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.244+0000 7f81613e5700 1 -- 172.21.15.45:0/2520487776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c10c750 msgr2=0x7f815c1142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:05.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.244+0000 7f81613e5700 1 --2- 172.21.15.45:0/2520487776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c10c750 0x7f815c1142d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f815400b0a0 tx=0x7f815400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.250+0000 7f81613e5700 1 -- 172.21.15.45:0/2520487776 shutdown_connections 2023-09-23T09:41:05.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.250+0000 7f81613e5700 1 --2- 172.21.15.45:0/2520487776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c10c750 0x7f815c1142d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.250+0000 7f81613e5700 1 --2- 172.21.15.45:0/2520487776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c107630 0x7f815c107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.250+0000 7f81613e5700 1 --2- 172.21.15.45:0/2520487776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f815c107fe0 0x7f815c10c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.250+0000 7f81613e5700 1 -- 172.21.15.45:0/2520487776 >> 172.21.15.45:0/2520487776 conn(0x7f815c076bf0 msgr2=0x7f815c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:05.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.254+0000 7f81613e5700 1 -- 172.21.15.45:0/2520487776 shutdown_connections 2023-09-23T09:41:05.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.254+0000 7f81613e5700 1 -- 172.21.15.45:0/2520487776 wait complete. 2023-09-23T09:41:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.255+0000 7f81613e5700 1 Processor -- start 2023-09-23T09:41:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.255+0000 7f81613e5700 1 -- start start 2023-09-23T09:41:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f81613e5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f815c107630 0x7f815c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f81613e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c10c750 0x7f815c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f81613e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c06b270 0x7f815c06b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f81613e5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c06bd30 con 0x7f815c10c750 2023-09-23T09:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f81613e5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c06be70 con 0x7f815c107630 2023-09-23T09:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f81613e5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c06c030 con 0x7f815c06b270 2023-09-23T09:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c10c750 0x7f815c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c10c750 0x7f815c070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36144/0 (socket says 172.21.15.45:36144) 2023-09-23T09:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 -- 172.21.15.45:0/3830864903 learned_addr learned my addr 172.21.15.45:0/3830864903 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 -- 172.21.15.45:0/3830864903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c06b270 msgr2=0x7f815c06b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c06b270 0x7f815c06b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 -- 172.21.15.45:0/3830864903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f815c107630 msgr2=0x7f815c0700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f815c107630 0x7f815c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.256+0000 7f815a7fc700 1 -- 172.21.15.45:0/3830864903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8154009d00 con 0x7f815c10c750 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.257+0000 7f815a7fc700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c10c750 0x7f815c070630 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f814c00b480 tx=0x7f814c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.257+0000 7f8143fff700 1 -- 172.21.15.45:0/3830864903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c011830 con 0x7f815c10c750 2023-09-23T09:41:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.257+0000 7f8143fff700 1 -- 172.21.15.45:0/3830864903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f814c011e50 con 0x7f815c10c750 2023-09-23T09:41:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.257+0000 7f8143fff700 1 -- 172.21.15.45:0/3830864903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c00f2e0 con 0x7f815c10c750 2023-09-23T09:41:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.262+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c06ef10 con 0x7f815c10c750 2023-09-23T09:41:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.262+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c06f3a0 con 0x7f815c10c750 2023-09-23T09:41:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.266+0000 7f816d5cfb80 1 Processor -- start 2023-09-23T09:41:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.266+0000 7f816d5cfb80 1 -- start start 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.267+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157d400 0x55e98157d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.267+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 0x55e9815996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.267+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e981599be0 0x55e98159bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.267+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9814e6460 con 0x55e981599be0 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.267+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9814d7c20 con 0x55e98157d400 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.267+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9814d5540 con 0x55e98157dd20 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.267+0000 7f815b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157d400 0x55e98157d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.268+0000 7f815b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157d400 0x55e98157d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57496/0 (socket says 172.21.15.45:57496) 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.268+0000 7f815b7fe700 1 -- 172.21.15.45:0/3541853776 learned_addr learned my addr 172.21.15.45:0/3541853776 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.269+0000 7f815b7fe700 1 -- 172.21.15.45:0/3541853776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 msgr2=0x55e9815996a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:41:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.269+0000 7f815b7fe700 1 --2- 172.21.15.45:0/3541853776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 0x55e9815996a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.269+0000 7f815b7fe700 1 -- 172.21.15.45:0/3541853776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e981599be0 msgr2=0x55e98159bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.269+0000 7f815b7fe700 1 --2- 172.21.15.45:0/3541853776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e981599be0 0x55e98159bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.269+0000 7f815b7fe700 1 -- 172.21.15.45:0/3541853776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9815338d0 con 0x55e98157d400 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.270+0000 7f8143fff700 1 -- 172.21.15.45:0/3830864903 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f814c010680 con 0x7f815c10c750 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.271+0000 7f8143fff700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f81440a1850 0x7f81440a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.271+0000 7f8143fff700 1 -- 172.21.15.45:0/3830864903 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f814c0ceee0 con 0x7f815c10c750 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.271+0000 7f815affd700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f81440a1850 0x7f81440a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.272+0000 7f815affd700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f81440a1850 0x7f81440a3d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f815000a9b0 tx=0x7f8150005d00 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:05.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.273+0000 7f815b7fe700 1 --2- 172.21.15.45:0/3541853776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157d400 0x55e98157d7e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81540062a0 tx=0x7f815400ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41bc98c902b44f21 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.275+0000 7f81417fa700 1 -- 172.21.15.45:0/3541853776 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815400e040 con 0x55e98157d400 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.275+0000 7f81417fa700 1 -- 172.21.15.45:0/3541853776 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8154009210 con 0x55e98157d400 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.275+0000 7f81417fa700 1 -- 172.21.15.45:0/3541853776 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81540052a0 con 0x55e98157d400 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 -- 172.21.15.45:0/3541853776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157d400 msgr2=0x55e98157d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/3541853776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157d400 0x55e98157d7e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81540062a0 tx=0x7f815400ce40 comp rx=0 tx=0).stop 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 -- 172.21.15.45:0/3541853776 shutdown_connections 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/3541853776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e981599be0 0x55e98159bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/3541853776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157d400 0x55e98157d7e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/3541853776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 0x55e9815996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 -- 172.21.15.45:0/3541853776 >> 172.21.15.45:0/3541853776 conn(0x55e9814f2260 msgr2=0x55e981579790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 -- 172.21.15.45:0/3541853776 shutdown_connections 2023-09-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 -- 172.21.15.45:0/3541853776 wait complete. 2023-09-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 -- start start 2023-09-23T09:41:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.276+0000 7f816d5cfb80 1 Processor -- start 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e98158ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 0x55e98158d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e981588090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9814d0e30 con 0x55e98157d400 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98146d710 con 0x55e981599be0 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98146d3b0 con 0x55e98157dd20 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f816d5cfb80 10 client.0 _ll_register_callbacks cb 0x55e9812ca800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e98158ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e98158ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36152/0 (socket says 172.21.15.45:36152) 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 -- 172.21.15.45:0/1778136576 learned_addr learned my addr 172.21.15.45:0/1778136576 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815b7fe700 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e981588090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 -- 172.21.15.45:0/1778136576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 msgr2=0x55e98158d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 0x55e98158d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 -- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 msgr2=0x55e981588090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e981588090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.277+0000 7f815affd700 1 -- 172.21.15.45:0/1778136576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c114b40 con 0x55e98157d400 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f815affd700 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e98158ce10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f815000d010 tx=0x7f8150004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e468783b7bbcf020 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f813eff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f813eff5700 1 -- 172.21.15.45:0/1778136576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815001c640 con 0x55e98157d400 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f813eff5700 1 -- 172.21.15.45:0/1778136576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f815001c7a0 con 0x55e98157d400 2023-09-23T09:41:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e981571250 con 0x55e98157d400 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f813eff5700 1 -- 172.21.15.45:0/1778136576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f815000fe70 con 0x55e98157d400 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f816d5cfb80 10 client.16752 fetch_fsmap learned FSMap version 47 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55e9812fbde0 con 0x55e98157d400 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.278+0000 7f813eff5700 1 -- 172.21.15.45:0/1778136576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e981571250 con 0x55e98157d400 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.279+0000 7f813eff5700 1 -- 172.21.15.45:0/1778136576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f815000eae0 con 0x55e98157d400 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.279+0000 7f816d5cfb80 10 client.16752 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.279+0000 7f816d5cfb80 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.279+0000 7f816d5cfb80 20 client.16752 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.279+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e981598410 0x55e981598820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.279+0000 7f816d5cfb80 4 client.16752 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:41:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.279+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9814c9e10 con 0x55e981598410 2023-09-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.280+0000 7f815a7fc700 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e981598410 0x55e981598820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.281+0000 7f813eff5700 10 client.16752 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.281+0000 7f815a7fc700 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e981598410 0x55e981598820 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6465cbfb1cd016c8 server_cookie=66520f7d91a763fa in_seq=0 out_seq=0 2023-09-23T09:41:05.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:05 smithi098 ceph-mon[102386]: pgmap v363: 97 pgs: 97 active+clean; 6.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-09-23T09:41:05.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:05 smithi098 ceph-mon[102386]: osdmap e98: 12 total, 12 up, 12 in 2023-09-23T09:41:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.294+0000 7f813eff5700 1 -- 172.21.15.45:0/1778136576 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9814c9e10 con 0x55e981598410 2023-09-23T09:41:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.294+0000 7f813eff5700 10 client.16752 handle_command_reply: tid=0 2023-09-23T09:41:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.299+0000 7f816d5cfb80 1 client.16752 shutdown 2023-09-23T09:41:08.032 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:05 smithi123 ceph-mon[103577]: pgmap v363: 97 pgs: 97 active+clean; 6.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-09-23T09:41:08.032 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:05 smithi123 ceph-mon[103577]: osdmap e98: 12 total, 12 up, 12 in 2023-09-23T09:41:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.305+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 msgr2=0x55e98158ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.305+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e98158ce10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f815000d010 tx=0x7f8150004750 comp rx=0 tx=0).stop 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 shutdown_connections 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e981598410 0x55e981598820 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e981588090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e98158ce10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1778136576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157dd20 0x55e98158d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 >> 172.21.15.45:0/1778136576 conn(0x55e9814f2260 msgr2=0x55e981579320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f813eff5700 0 client.16752 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 shutdown_connections 2023-09-23T09:41:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1778136576 wait complete. 2023-09-23T09:41:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.314+0000 7f816d5cfb80 20 client.16752 trim_cache size 0 max 16384 2023-09-23T09:41:08.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:05 smithi045 ceph-mon[98627]: pgmap v363: 97 pgs: 97 active+clean; 6.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-09-23T09:41:08.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:05 smithi045 ceph-mon[98627]: osdmap e98: 12 total, 12 up, 12 in 2023-09-23T09:41:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.346+0000 7f816d5cfb80 1 Processor -- start 2023-09-23T09:41:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.352+0000 7f816d5cfb80 1 -- start start 2023-09-23T09:41:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.352+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157d400 0x55e98157f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.352+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157dd20 0x55e98159fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.352+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e98157fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.352+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e981351920 con 0x55e98157dd20 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.352+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98157ebf0 con 0x55e981599be0 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.352+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9815a0530 con 0x55e98157d400 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.353+0000 7f815b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e98157fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.353+0000 7f815b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e98157fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57514/0 (socket says 172.21.15.45:57514) 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.353+0000 7f815b7fe700 1 -- 172.21.15.45:0/174318464 learned_addr learned my addr 172.21.15.45:0/174318464 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.355+0000 7f815affd700 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157d400 0x55e98157f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.364+0000 7f815a7fc700 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157dd20 0x55e98159fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.372+0000 7f815affd700 1 -- 172.21.15.45:0/174318464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 msgr2=0x55e98157fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.372+0000 7f815affd700 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e98157fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.372+0000 7f815affd700 1 -- 172.21.15.45:0/174318464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157dd20 msgr2=0x55e98159fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.372+0000 7f815affd700 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157dd20 0x55e98159fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.372+0000 7f815affd700 1 -- 172.21.15.45:0/174318464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9812e4c60 con 0x55e98157d400 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.386+0000 7f815affd700 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157d400 0x55e98157f9a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f815000ee10 tx=0x7f815000a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c78e625c8b81216c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.393+0000 7f8141ffb700 1 -- 172.21.15.45:0/174318464 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815001b5f0 con 0x55e98157d400 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.393+0000 7f8141ffb700 1 -- 172.21.15.45:0/174318464 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815000fe40 con 0x55e98157d400 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.393+0000 7f8141ffb700 1 -- 172.21.15.45:0/174318464 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150004ea0 con 0x55e98157d400 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.394+0000 7f816d5cfb80 1 -- 172.21.15.45:0/174318464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157d400 msgr2=0x55e98157f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.394+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157d400 0x55e98157f9a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f815000ee10 tx=0x7f815000a780 comp rx=0 tx=0).stop 2023-09-23T09:41:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.396+0000 7f816d5cfb80 1 -- 172.21.15.45:0/174318464 shutdown_connections 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.396+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e981599be0 0x55e98157fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.396+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e98157d400 0x55e98157f9a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.396+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/174318464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157dd20 0x55e98159fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.396+0000 7f816d5cfb80 1 -- 172.21.15.45:0/174318464 >> 172.21.15.45:0/174318464 conn(0x55e9814f2260 msgr2=0x55e98157f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.400+0000 7f816d5cfb80 1 -- 172.21.15.45:0/174318464 shutdown_connections 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 -- 172.21.15.45:0/174318464 wait complete. 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 -- start start 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 Processor -- start 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e981582130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157dd20 0x55e9815851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e981599be0 0x55e981585710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e981585c50 con 0x55e98157d400 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e981585dc0 con 0x55e98157dd20 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.403+0000 7f816d5cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e981585f80 con 0x55e981599be0 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.404+0000 7f816d5cfb80 10 client.0 _ll_register_callbacks cb 0x7f815000cc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.404+0000 7f815affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e981582130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.404+0000 7f815affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e981582130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36168/0 (socket says 172.21.15.45:36168) 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.404+0000 7f815affd700 1 -- 172.21.15.45:0/1238644504 learned_addr learned my addr 172.21.15.45:0/1238644504 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.405+0000 7f815a7fc700 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157dd20 0x55e9815851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.405+0000 7f815affd700 1 -- 172.21.15.45:0/1238644504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e981599be0 msgr2=0x55e981585710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.405+0000 7f815affd700 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e981599be0 0x55e981585710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.405+0000 7f815affd700 1 -- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157dd20 msgr2=0x55e9815851d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.405+0000 7f815affd700 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157dd20 0x55e9815851d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.405+0000 7f815affd700 1 -- 172.21.15.45:0/1238644504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9812fbde0 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.406+0000 7f815affd700 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e981582130 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f815000ffd0 tx=0x7f8150004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afeb8740604fa76 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.406+0000 7f813eff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.420+0000 7f813eff5700 1 -- 172.21.15.45:0/1238644504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150004430 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.428+0000 7f813eff5700 1 -- 172.21.15.45:0/1238644504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8150003680 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.428+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8154009210 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.435+0000 7f813eff5700 1 -- 172.21.15.45:0/1238644504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8150003840 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.436+0000 7f816d5cfb80 10 client.16758 fetch_fsmap learned FSMap version 47 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.436+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55e981322250 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f813eff5700 1 -- 172.21.15.45:0/1238644504 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8154009210 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f813eff5700 1 -- 172.21.15.45:0/1238644504 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f815000e8a0 con 0x55e98157d400 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f816d5cfb80 10 client.16758 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T09:41:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f816d5cfb80 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f816d5cfb80 20 client.16758 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e98160f0f0 0x55e981611510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f816d5cfb80 4 client.16758 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.437+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e9814c9b30 con 0x55e98160f0f0 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.449+0000 7f815a7fc700 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e98160f0f0 0x55e981611510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.463+0000 7f813eff5700 10 client.16758 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.463+0000 7f815a7fc700 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e98160f0f0 0x55e981611510 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15953c419b2d979 server_cookie=ffc982968e16dd29 in_seq=0 out_seq=0 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:41:08.049 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:41:08.050 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:41:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.468+0000 7f813eff5700 1 -- 172.21.15.45:0/1238644504 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e9814c9b30 con 0x55e98160f0f0 2023-09-23T09:41:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.468+0000 7f813eff5700 10 client.16758 handle_command_reply: tid=0 2023-09-23T09:41:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.468+0000 7f816d5cfb80 1 client.16758 shutdown 2023-09-23T09:41:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 msgr2=0x55e981582130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e981582130 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f815000ffd0 tx=0x7f8150004830 comp rx=0 tx=0).stop 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 shutdown_connections 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e98160f0f0 0x55e981611510 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e981599be0 0x55e981585710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e98157d400 0x55e981582130 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 --2- 172.21.15.45:0/1238644504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e98157dd20 0x55e9815851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 >> 172.21.15.45:0/1238644504 conn(0x55e9814f2260 msgr2=0x55e98157fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f813eff5700 0 client.16758 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 shutdown_connections 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 1 -- 172.21.15.45:0/1238644504 wait complete. 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.472+0000 7f816d5cfb80 20 client.16758 trim_cache size 0 max 16384 2023-09-23T09:41:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f81440a1850 msgr2=0x7f81440a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f81440a1850 0x7f81440a3d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f815000a9b0 tx=0x7f8150005d00 comp rx=0 tx=0).stop 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c10c750 msgr2=0x7f815c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c10c750 0x7f815c070630 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f814c00b480 tx=0x7f814c00b840 comp rx=0 tx=0).stop 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 shutdown_connections 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c06b270 0x7f815c06b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f815c107630 0x7f815c0700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f81440a1850 0x7f81440a3d10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 --2- 172.21.15.45:0/3830864903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c10c750 0x7f815c070630 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.473+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 >> 172.21.15.45:0/3830864903 conn(0x7f815c076bf0 msgr2=0x7f815c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.474+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 shutdown_connections 2023-09-23T09:41:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:05.474+0000 7f81613e5700 1 -- 172.21.15.45:0/3830864903 wait complete. 2023-09-23T09:41:08.054 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:41:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 680.0 s, 1396.2 tps, lat 73.105 ms stddev 110.239 2023-09-23T09:41:08.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:07 smithi098 ceph-mon[102386]: pgmap v365: 97 pgs: 97 active+clean; 6.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-09-23T09:41:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.882+0000 7f76eb1c6700 1 -- 172.21.15.45:0/862365013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4069aa0 msgr2=0x7f76e410dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.882+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/862365013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4069aa0 0x7f76e410dc60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00b0d0 tx=0x7f76dc00b3e0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.882+0000 7f76eb1c6700 1 -- 172.21.15.45:0/862365013 shutdown_connections 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.882+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/862365013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4069aa0 0x7f76e410dc60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.882+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/862365013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4106f30 0x7f76e4107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.882+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/862365013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76e40690e0 0x7f76e4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.882+0000 7f76eb1c6700 1 -- 172.21.15.45:0/862365013 >> 172.21.15.45:0/862365013 conn(0x7f76e4077bd0 msgr2=0x7f76e4077fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.884+0000 7f76eb1c6700 1 -- 172.21.15.45:0/862365013 shutdown_connections 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.884+0000 7f76eb1c6700 1 -- 172.21.15.45:0/862365013 wait complete. 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.884+0000 7f76eb1c6700 1 Processor -- start 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.884+0000 7f76eb1c6700 1 -- start start 2023-09-23T09:41:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.884+0000 7f76eb1c6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76e40690e0 0x7f76e4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76eb1c6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4069aa0 0x7f76e41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76eb1c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4106f30 0x7f76e419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76eb1c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e419e310 con 0x7f76e4106f30 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76eb1c6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e419e480 con 0x7f76e4069aa0 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76eb1c6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e419e640 con 0x7f76e40690e0 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4069aa0 0x7f76e41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4069aa0 0x7f76e41996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57532/0 (socket says 172.21.15.45:57532) 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e3fff700 1 -- 172.21.15.45:0/1497099157 learned_addr learned my addr 172.21.15.45:0/1497099157 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e9763700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4106f30 0x7f76e419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e8f62700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76e40690e0 0x7f76e4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e8f62700 1 -- 172.21.15.45:0/1497099157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4069aa0 msgr2=0x7f76e41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e8f62700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4069aa0 0x7f76e41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e8f62700 1 -- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4106f30 msgr2=0x7f76e419dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e8f62700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4106f30 0x7f76e419dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e8f62700 1 -- 172.21.15.45:0/1497099157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0009520 con 0x7f76e40690e0 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e9763700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4106f30 0x7f76e419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e3fff700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4069aa0 0x7f76e41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e8f62700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76e40690e0 0x7f76e4199170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f76d800e9c0 tx=0x7f76d800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.885+0000 7f76e1ffb700 1 -- 172.21.15.45:0/1497099157 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8004220 con 0x7f76e40690e0 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.886+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc009d00 con 0x7f76e40690e0 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.886+0000 7f76e1ffb700 1 -- 172.21.15.45:0/1497099157 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d8004380 con 0x7f76e40690e0 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.886+0000 7f76e1ffb700 1 -- 172.21.15.45:0/1497099157 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8003be0 con 0x7f76e40690e0 2023-09-23T09:41:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.886+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e4107d60 con 0x7f76e40690e0 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.890+0000 7f76f73b0b80 1 Processor -- start 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76f73b0b80 1 -- start start 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf1f40 0x55739bdf2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf2860 0x55739bdfd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739bdffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bd5b400 con 0x55739bdf1f40 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bd4cbc0 con 0x55739bdf2860 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bd4a4e0 con 0x55739bdfd710 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76e1ffb700 1 -- 172.21.15.45:0/1497099157 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f76d800c7c0 con 0x7f76e40690e0 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76e9763700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf1f40 0x55739bdf2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76e9763700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf1f40 0x55739bdf2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36186/0 (socket says 172.21.15.45:36186) 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76e9763700 1 -- 172.21.15.45:0/24110721 learned_addr learned my addr 172.21.15.45:0/24110721 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76e3fff700 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739bdffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.891+0000 7f76e8f62700 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf2860 0x55739bdfd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e9763700 1 -- 172.21.15.45:0/24110721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 msgr2=0x55739bdffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e9763700 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739bdffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e9763700 1 -- 172.21.15.45:0/24110721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf2860 msgr2=0x55739bdfd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e9763700 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf2860 0x55739bdfd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e9763700 1 -- 172.21.15.45:0/24110721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739bda8870 con 0x55739bdf1f40 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e3fff700 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739bdffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e9763700 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf1f40 0x55739bdf2320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009b80 tx=0x7f76dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03ca7f51b7e417f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e1ffb700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f76d40a1710 0x7f76d40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76ceffd700 1 -- 172.21.15.45:0/24110721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc025070 con 0x55739bdf1f40 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e8f62700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f76d40a1710 0x7f76d40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76e1ffb700 1 -- 172.21.15.45:0/1497099157 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f76d8014070 con 0x7f76e40690e0 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76ceffd700 1 -- 172.21.15.45:0/24110721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76dc0093e0 con 0x55739bdf1f40 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.892+0000 7f76ceffd700 1 -- 172.21.15.45:0/24110721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00e040 con 0x55739bdf1f40 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 -- 172.21.15.45:0/24110721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf1f40 msgr2=0x55739bdf2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76e8f62700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f76d40a1710 0x7f76d40a3bd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f76d80cadc0 tx=0x7f76d8003680 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf1f40 0x55739bdf2320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009b80 tx=0x7f76dc009bb0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 -- 172.21.15.45:0/24110721 shutdown_connections 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739bdffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf1f40 0x55739bdf2320 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/24110721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf2860 0x55739bdfd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 -- 172.21.15.45:0/24110721 >> 172.21.15.45:0/24110721 conn(0x55739bd67200 msgr2=0x55739bdee3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 -- 172.21.15.45:0/24110721 shutdown_connections 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 -- 172.21.15.45:0/24110721 wait complete. 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 -- start start 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.893+0000 7f76f73b0b80 1 Processor -- start 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739bdfd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf2860 0x55739bdfa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdfd710 0x55739bdfa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bd45dd0 con 0x55739bdfd710 2023-09-23T09:41:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bce26b0 con 0x55739bdf1f40 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bce2350 con 0x55739bdf2860 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76f73b0b80 10 client.0 _ll_register_callbacks cb 0x55739bb3f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76e3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf2860 0x55739bdfa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76e9763700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739bdfd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76e3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf2860 0x55739bdfa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38302/0 (socket says 172.21.15.45:38302) 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76e9763700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739bdfd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57544/0 (socket says 172.21.15.45:57544) 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76e3fff700 1 -- 172.21.15.45:0/2763146528 learned_addr learned my addr 172.21.15.45:0/2763146528 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.894+0000 7f76e9763700 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdfd710 0x55739bdfa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76e3fff700 1 -- 172.21.15.45:0/2763146528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 msgr2=0x55739bdfd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76e3fff700 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739bdfd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76e3fff700 1 -- 172.21.15.45:0/2763146528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdfd710 msgr2=0x55739bdfa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76e3fff700 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdfd710 0x55739bdfa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76e3fff700 1 -- 172.21.15.45:0/2763146528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d0011440 con 0x55739bdf2860 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76e9763700 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdfd710 0x55739bdfa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76e3fff700 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf2860 0x55739bdfa300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76d000f3c0 tx=0x7f76d000f6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b696329fa3ed1b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76c3fff700 1 -- 172.21.15.45:0/2763146528 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0017070 con 0x55739bdf2860 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55739bde6090 con 0x55739bdf2860 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76c3fff700 1 -- 172.21.15.45:0/2763146528 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f76d0012020 con 0x55739bdf2860 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76c3fff700 1 -- 172.21.15.45:0/2763146528 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d00121e0 con 0x55739bdf2860 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76f73b0b80 10 client.26428 fetch_fsmap learned FSMap version 47 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f76e410e560 con 0x55739bdf2860 2023-09-23T09:41:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.895+0000 7f76c3fff700 1 -- 172.21.15.45:0/2763146528 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55739bde6090 con 0x55739bdf2860 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76c3fff700 1 -- 172.21.15.45:0/2763146528 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f76d0003ba0 con 0x55739bdf2860 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76f73b0b80 10 client.26428 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76f73b0b80 10 client.26428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76f73b0b80 20 client.26428 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be0d130 0x55739be0f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76f73b0b80 4 client.26428 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55739bd3edb0 con 0x55739be0d130 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76e9763700 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be0d130 0x55739be0f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76e9763700 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be0d130 0x55739be0f550 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c861af73e9087190 server_cookie=4b68a20457293bc in_seq=0 out_seq=0 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.896+0000 7f76c3fff700 10 client.26428 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76c3fff700 1 -- 172.21.15.45:0/2763146528 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55739bd3edb0 con 0x55739be0d130 2023-09-23T09:41:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76c3fff700 10 client.26428 handle_command_reply: tid=0 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 client.26428 shutdown 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf2860 msgr2=0x55739bdfa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf2860 0x55739bdfa300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76d000f3c0 tx=0x7f76d000f6d0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 shutdown_connections 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be0d130 0x55739be0f550 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdfd710 0x55739bdfa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739bdfd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2763146528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf2860 0x55739bdfa300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76c3fff700 0 client.26428 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 >> 172.21.15.45:0/2763146528 conn(0x55739bd67200 msgr2=0x55739bdedfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.897+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 shutdown_connections 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.898+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2763146528 wait complete. 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.898+0000 7f76f73b0b80 20 client.26428 trim_cache size 0 max 16384 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.922+0000 7f76f73b0b80 1 Processor -- start 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.924+0000 7f76f73b0b80 1 -- start start 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.924+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf1f40 0x55739bdfa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.924+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 0x55739be04190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.924+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdfd710 0x55739bdf9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.924+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bbc68c0 con 0x55739bdf2860 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.924+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bdf39a0 con 0x55739bdfd710 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.924+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739bdef950 con 0x55739bdf1f40 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf1f40 0x55739bdfa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf1f40 0x55739bdfa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38308/0 (socket says 172.21.15.45:38308) 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 -- 172.21.15.45:0/2159239359 learned_addr learned my addr 172.21.15.45:0/2159239359 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 -- 172.21.15.45:0/2159239359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdfd710 msgr2=0x55739bdf9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 --2- 172.21.15.45:0/2159239359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdfd710 0x55739bdf9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 -- 172.21.15.45:0/2159239359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 msgr2=0x55739be04190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 --2- 172.21.15.45:0/2159239359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 0x55739be04190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 -- 172.21.15.45:0/2159239359 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739bb70d80 con 0x55739bdf1f40 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.926+0000 7f76e9763700 1 --2- 172.21.15.45:0/2159239359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf1f40 0x55739bdfa2c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f76dc022010 tx=0x7f76dc00b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=655c8b59a810c496 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.935+0000 7f76cf7fe700 1 -- 172.21.15.45:0/2159239359 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc025040 con 0x55739bdf1f40 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.935+0000 7f76cf7fe700 1 -- 172.21.15.45:0/2159239359 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76dc0354e0 con 0x55739bdf1f40 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.935+0000 7f76cf7fe700 1 -- 172.21.15.45:0/2159239359 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc003d70 con 0x55739bdf1f40 2023-09-23T09:41:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.936+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2159239359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf1f40 msgr2=0x55739bdfa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.936+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2159239359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf1f40 0x55739bdfa2c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f76dc022010 tx=0x7f76dc00b730 comp rx=0 tx=0).stop 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.936+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2159239359 shutdown_connections 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.936+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2159239359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdfd710 0x55739bdf9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.936+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2159239359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdf1f40 0x55739bdfa2c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.936+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/2159239359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 0x55739be04190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.936+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2159239359 >> 172.21.15.45:0/2159239359 conn(0x55739bd67200 msgr2=0x55739be13eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.937+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2159239359 shutdown_connections 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.937+0000 7f76f73b0b80 1 -- 172.21.15.45:0/2159239359 wait complete. 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.937+0000 7f76f73b0b80 1 -- start start 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 1 Processor -- start 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739be00360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 0x55739be008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739be5ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739be60490 con 0x55739bdf2860 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739be60600 con 0x55739bdf1f40 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739be607c0 con 0x55739bdfd710 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.938+0000 7f76f73b0b80 10 client.0 _ll_register_callbacks cb 0x7f76d0012b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739be5ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739be5ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38314/0 (socket says 172.21.15.45:38314) 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 -- 172.21.15.45:0/615417386 learned_addr learned my addr 172.21.15.45:0/615417386 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 -- 172.21.15.45:0/615417386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 msgr2=0x55739be00360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739be00360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 -- 172.21.15.45:0/615417386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 msgr2=0x55739be008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 0x55739be008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 -- 172.21.15.45:0/615417386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55739bda8870 con 0x55739bdfd710 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76e9763700 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739be5ff50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76dc038040 tx=0x7f76dc008fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d40e6450aa0215 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76c3fff700 1 -- 172.21.15.45:0/615417386 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc025040 con 0x55739bdfd710 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76c3fff700 1 -- 172.21.15.45:0/615417386 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f76dc020d80 con 0x55739bdfd710 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.939+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76dc0093e0 con 0x55739bdfd710 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76c3fff700 1 -- 172.21.15.45:0/615417386 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76dc035a10 con 0x55739bdfd710 2023-09-23T09:41:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 10 client.26434 fetch_fsmap learned FSMap version 47 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55739bb59c00 con 0x55739bdfd710 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76c3fff700 1 -- 172.21.15.45:0/615417386 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76dc052080 con 0x55739bdfd710 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76c3fff700 1 -- 172.21.15.45:0/615417386 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f76dc031070 con 0x55739bdfd710 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 10 client.26434 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 10 client.26434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 20 client.26434 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be61970 0x55739be63d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 4 client.26434 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.940+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55739bd3ead0 con 0x55739be61970 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.943+0000 7f76e3fff700 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be61970 0x55739be63d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.948+0000 7f76c3fff700 10 client.26434 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.948+0000 7f76e3fff700 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be61970 0x55739be63d90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c0e99b8950724b server_cookie=1454167ae7cfc70f in_seq=0 out_seq=0 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76c3fff700 1 -- 172.21.15.45:0/615417386 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55739bd3ead0 con 0x55739be61970 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76c3fff700 10 client.26434 handle_command_reply: tid=0 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 client.26434 shutdown 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 msgr2=0x55739be5ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739be5ff50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76dc038040 tx=0x7f76dc008fc0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 shutdown_connections 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55739be61970 0x55739be63d90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55739bdfd710 0x55739be5ff50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55739bdf1f40 0x55739be00360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.992+0000 7f76f73b0b80 1 --2- 172.21.15.45:0/615417386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55739bdf2860 0x55739be008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.993+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 >> 172.21.15.45:0/615417386 conn(0x55739bd67200 msgr2=0x55739be088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.993+0000 7f76c3fff700 0 client.26434 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.993+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 shutdown_connections 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.993+0000 7f76f73b0b80 1 -- 172.21.15.45:0/615417386 wait complete. 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.993+0000 7f76f73b0b80 20 client.26434 trim_cache size 0 max 16384 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f76d40a1710 msgr2=0x7f76d40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f76d40a1710 0x7f76d40a3bd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f76d80cadc0 tx=0x7f76d8003680 comp rx=0 tx=0).stop 2023-09-23T09:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76e40690e0 msgr2=0x7f76e4199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76e40690e0 0x7f76e4199170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f76d800e9c0 tx=0x7f76d800ecd0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 shutdown_connections 2023-09-23T09:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f76d40a1710 0x7f76d40a3bd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4106f30 0x7f76e419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76e40690e0 0x7f76e4199170 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 --2- 172.21.15.45:0/1497099157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76e4069aa0 0x7f76e41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 >> 172.21.15.45:0/1497099157 conn(0x7f76e4077bd0 msgr2=0x7f76e410bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 shutdown_connections 2023-09-23T09:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:06.994+0000 7f76eb1c6700 1 -- 172.21.15.45:0/1497099157 wait complete. 2023-09-23T09:41:08.291 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:07 smithi045 ceph-mon[98627]: pgmap v365: 97 pgs: 97 active+clean; 6.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-09-23T09:41:08.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:07 smithi123 ceph-mon[103577]: pgmap v365: 97 pgs: 97 active+clean; 6.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-09-23T09:41:09.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:08 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:09.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:08 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:09.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:08 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:10.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:09 smithi098 ceph-mon[102386]: pgmap v366: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-09-23T09:41:10.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:09 smithi045 ceph-mon[98627]: pgmap v366: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-09-23T09:41:10.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:09 smithi123 ceph-mon[103577]: pgmap v366: 97 pgs: 97 active+clean; 6.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-09-23T09:41:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 685.0 s, 1688.8 tps, lat 59.535 ms stddev 68.719 2023-09-23T09:41:12.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:11 smithi045 ceph-mon[98627]: pgmap v367: 97 pgs: 97 active+clean; 5.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-09-23T09:41:12.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:11 smithi123 ceph-mon[103577]: pgmap v367: 97 pgs: 97 active+clean; 5.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-09-23T09:41:12.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:11 smithi098 ceph-mon[102386]: pgmap v367: 97 pgs: 97 active+clean; 5.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-09-23T09:41:14.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:13 smithi098 ceph-mon[102386]: pgmap v368: 97 pgs: 97 active+clean; 5.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-09-23T09:41:14.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:13 smithi045 ceph-mon[98627]: pgmap v368: 97 pgs: 97 active+clean; 5.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-09-23T09:41:14.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:13 smithi123 ceph-mon[103577]: pgmap v368: 97 pgs: 97 active+clean; 5.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-09-23T09:41:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:15 smithi098 ceph-mon[102386]: pgmap v369: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-09-23T09:41:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:15 smithi098 ceph-mon[102386]: osdmap e99: 12 total, 12 up, 12 in 2023-09-23T09:41:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:15 smithi045 ceph-mon[98627]: pgmap v369: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-09-23T09:41:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:15 smithi045 ceph-mon[98627]: osdmap e99: 12 total, 12 up, 12 in 2023-09-23T09:41:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:15 smithi123 ceph-mon[103577]: pgmap v369: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-09-23T09:41:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:15 smithi123 ceph-mon[103577]: osdmap e99: 12 total, 12 up, 12 in 2023-09-23T09:41:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 690.0 s, 1621.0 tps, lat 60.590 ms stddev 74.550 2023-09-23T09:41:18.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:17 smithi098 ceph-mon[102386]: pgmap v371: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-09-23T09:41:18.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:17 smithi045 ceph-mon[98627]: pgmap v371: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-09-23T09:41:18.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:17 smithi123 ceph-mon[103577]: pgmap v371: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-09-23T09:41:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.162+0000 7f90a8dc1700 1 -- 172.21.15.45:0/1337707659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a4105d90 msgr2=0x7f90a4106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.162+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/1337707659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a4105d90 0x7f90a4106170 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9094009990 tx=0x7f9094009ca0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.163+0000 7f90a8dc1700 1 -- 172.21.15.45:0/1337707659 shutdown_connections 2023-09-23T09:41:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.163+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/1337707659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100690 0x7f90a410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.163+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/1337707659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a4105d90 0x7f90a4106170 secure :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9094009990 tx=0x7f9094009ca0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.163+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/1337707659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a40ffcd0 0x7f90a4100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.163+0000 7f90a8dc1700 1 -- 172.21.15.45:0/1337707659 >> 172.21.15.45:0/1337707659 conn(0x7f90a4076e20 msgr2=0x7f90a4077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.166+0000 7f90a8dc1700 1 -- 172.21.15.45:0/1337707659 shutdown_connections 2023-09-23T09:41:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.166+0000 7f90a8dc1700 1 -- 172.21.15.45:0/1337707659 wait complete. 2023-09-23T09:41:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.167+0000 7f90a8dc1700 1 Processor -- start 2023-09-23T09:41:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a8dc1700 1 -- start start 2023-09-23T09:41:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a8dc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a40ffcd0 0x7f90a410c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a8dc1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100690 0x7f90a4108510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a8dc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4108a50 0x7f90a4108f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a8dc1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a410cf20 con 0x7f90a40ffcd0 2023-09-23T09:41:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a8dc1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4072890 con 0x7f90a4100690 2023-09-23T09:41:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a8dc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4072a50 con 0x7f90a4108a50 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a40ffcd0 0x7f90a410c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4108a50 0x7f90a4108f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a40ffcd0 0x7f90a410c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36220/0 (socket says 172.21.15.45:36220) 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a259c700 1 -- 172.21.15.45:0/842289019 learned_addr learned my addr 172.21.15.45:0/842289019 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.168+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100690 0x7f90a4108510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a1d9b700 1 -- 172.21.15.45:0/842289019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4108a50 msgr2=0x7f90a4108f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4108a50 0x7f90a4108f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a1d9b700 1 -- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a40ffcd0 msgr2=0x7f90a410c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a40ffcd0 0x7f90a410c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a1d9b700 1 -- 172.21.15.45:0/842289019 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940095f0 con 0x7f90a4100690 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a259c700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a40ffcd0 0x7f90a410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100690 0x7f90a4108510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f909800ea20 tx=0x7f909800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90937fe700 1 -- 172.21.15.45:0/842289019 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800cbd0 con 0x7f90a4100690 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90937fe700 1 -- 172.21.15.45:0/842289019 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909800ee60 con 0x7f90a4100690 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a4072d30 con 0x7f90a4100690 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90937fe700 1 -- 172.21.15.45:0/842289019 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098018680 con 0x7f90a4100690 2023-09-23T09:41:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.169+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a41b3410 con 0x7f90a4100690 2023-09-23T09:41:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.173+0000 7f90b4fabb80 1 Processor -- start 2023-09-23T09:41:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.173+0000 7f90937fe700 1 -- 172.21.15.45:0/842289019 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f9098010430 con 0x7f90a4100690 2023-09-23T09:41:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.173+0000 7f90b4fabb80 1 -- start start 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.173+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c986f40 0x55fb3c987320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.173+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c987860 0x55fb3c9921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.173+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c994b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c8f0400 con 0x55fb3c987860 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c8e1bc0 con 0x55fb3c986f40 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c8df4e0 con 0x55fb3c992710 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c986f40 0x55fb3c987320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c986f40 0x55fb3c987320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57576/0 (socket says 172.21.15.45:57576) 2023-09-23T09:41:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 -- 172.21.15.45:0/3428619279 learned_addr learned my addr 172.21.15.45:0/3428619279 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90937fe700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a1790 0x7f908c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c994b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a1790 0x7f908c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a259c700 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c987860 0x55fb3c9921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 -- 172.21.15.45:0/3428619279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 msgr2=0x55fb3c994b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c994b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 -- 172.21.15.45:0/3428619279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c987860 msgr2=0x55fb3c9921d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c987860 0x55fb3c9921d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90a2d9d700 1 -- 172.21.15.45:0/3428619279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb3c93d870 con 0x55fb3c986f40 2023-09-23T09:41:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.174+0000 7f90937fe700 1 -- 172.21.15.45:0/842289019 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9098014070 con 0x7f90a4100690 2023-09-23T09:41:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.175+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c986f40 0x55fb3c987320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f909c00a530 tx=0x7f909c00a840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ff2c8de391e217 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.175+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c994b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.175+0000 7f90a259c700 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c987860 0x55fb3c9921d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.175+0000 7f9090ff9700 1 -- 172.21.15.45:0/3428619279 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c018750 con 0x55fb3c986f40 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.175+0000 7f9090ff9700 1 -- 172.21.15.45:0/3428619279 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909c018d70 con 0x55fb3c986f40 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.175+0000 7f9090ff9700 1 -- 172.21.15.45:0/3428619279 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c017cf0 con 0x55fb3c986f40 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.176+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3428619279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c986f40 msgr2=0x55fb3c987320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.176+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c986f40 0x55fb3c987320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f909c00a530 tx=0x7f909c00a840 comp rx=0 tx=0).stop 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a1790 0x7f908c0a3c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f909c01e000 tx=0x7f909c017f30 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3428619279 shutdown_connections 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c994b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c986f40 0x55fb3c987320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3428619279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c987860 0x55fb3c9921d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3428619279 >> 172.21.15.45:0/3428619279 conn(0x55fb3c8fc200 msgr2=0x55fb3c9833c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3428619279 shutdown_connections 2023-09-23T09:41:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.177+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3428619279 wait complete. 2023-09-23T09:41:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 -- start start 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 Processor -- start 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c9920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c98f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c98f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c8dadd0 con 0x55fb3c986f40 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c8776b0 con 0x55fb3c987860 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c877350 con 0x55fb3c992710 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90b4fabb80 10 client.0 _ll_register_callbacks cb 0x55fb3c6d47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c9920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c9920d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36234/0 (socket says 172.21.15.45:36234) 2023-09-23T09:41:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.178+0000 7f90a259c700 1 -- 172.21.15.45:0/3706731926 learned_addr learned my addr 172.21.15.45:0/3706731926 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c98f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a259c700 1 -- 172.21.15.45:0/3706731926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 msgr2=0x55fb3c98f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a259c700 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c98f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a259c700 1 -- 172.21.15.45:0/3706731926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 msgr2=0x55fb3c98f350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a259c700 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c98f350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a259c700 1 -- 172.21.15.45:0/3706731926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c013040 con 0x55fb3c986f40 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c98f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90a259c700 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c9920d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f909400b650 tx=0x7f9094005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d1f6836dd8fd04 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f9089ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f9089ffb700 1 -- 172.21.15.45:0/3706731926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094004910 con 0x55fb3c986f40 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f9089ffb700 1 -- 172.21.15.45:0/3706731926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9094004a70 con 0x55fb3c986f40 2023-09-23T09:41:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.179+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb3c97b090 con 0x55fb3c986f40 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f9089ffb700 1 -- 172.21.15.45:0/3706731926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9094004c30 con 0x55fb3c986f40 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 10 client.16776 fetch_fsmap learned FSMap version 48 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55fb3c705d80 con 0x55fb3c986f40 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f9089ffb700 1 -- 172.21.15.45:0/3706731926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb3c97b090 con 0x55fb3c986f40 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f9089ffb700 1 -- 172.21.15.45:0/3706731926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9094023410 con 0x55fb3c986f40 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 10 client.16776 fetch_fsmap finished waiting for FSMap version 48 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 10 client.16776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 20 client.16776 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9a2210 0x55fb3c9a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 4 client.16776 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb3c8d3db0 con 0x55fb3c9a2210 2023-09-23T09:41:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.180+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9a2210 0x55fb3c9a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.181+0000 7f9089ffb700 10 client.16776 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.181+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9a2210 0x55fb3c9a4630 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768f2e3b0a0d4fe9 server_cookie=d53dc4773f675c44 in_seq=0 out_seq=0 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.181+0000 7f9089ffb700 1 -- 172.21.15.45:0/3706731926 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fb3c8d3db0 con 0x55fb3c9a2210 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.181+0000 7f9089ffb700 10 client.16776 handle_command_reply: tid=0 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.181+0000 7f90b4fabb80 1 client.16776 shutdown 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 msgr2=0x55fb3c9920d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c9920d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f909400b650 tx=0x7f9094005270 comp rx=0 tx=0).stop 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 shutdown_connections 2023-09-23T09:41:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9a2210 0x55fb3c9a4630 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c992710 0x55fb3c98f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c9920d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/3706731926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c98f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 >> 172.21.15.45:0/3706731926 conn(0x55fb3c8fc200 msgr2=0x55fb3c982ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f9089ffb700 0 client.16776 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 shutdown_connections 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 1 -- 172.21.15.45:0/3706731926 wait complete. 2023-09-23T09:41:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.182+0000 7f90b4fabb80 20 client.16776 trim_cache size 0 max 16384 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 Processor -- start 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 -- start start 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c991c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c987860 0x55fb3c98a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c992710 0x55fb3c98abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c75b8c0 con 0x55fb3c986f40 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c98b200 con 0x55fb3c992710 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c987ea0 con 0x55fb3c987860 2023-09-23T09:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c991c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c991c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36242/0 (socket says 172.21.15.45:36242) 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 -- 172.21.15.45:0/109523196 learned_addr learned my addr 172.21.15.45:0/109523196 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c987860 0x55fb3c98a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c992710 0x55fb3c98abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 -- 172.21.15.45:0/109523196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c987860 msgr2=0x55fb3c98a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c987860 0x55fb3c98a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 -- 172.21.15.45:0/109523196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c992710 msgr2=0x55fb3c98abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c992710 0x55fb3c98abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a259c700 1 -- 172.21.15.45:0/109523196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb3c6eec00 con 0x55fb3c986f40 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.205+0000 7f90a2d9d700 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c992710 0x55fb3c98abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.206+0000 7f90a259c700 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c991c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9094009ce0 tx=0x7f9094023990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e581a195ad505d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.206+0000 7f90917fa700 1 -- 172.21.15.45:0/109523196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094005bc0 con 0x55fb3c986f40 2023-09-23T09:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.206+0000 7f90917fa700 1 -- 172.21.15.45:0/109523196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909401e050 con 0x55fb3c986f40 2023-09-23T09:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.206+0000 7f90917fa700 1 -- 172.21.15.45:0/109523196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400f6c0 con 0x55fb3c986f40 2023-09-23T09:41:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 -- 172.21.15.45:0/109523196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 msgr2=0x55fb3c991c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c991c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9094009ce0 tx=0x7f9094023990 comp rx=0 tx=0).stop 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 -- 172.21.15.45:0/109523196 shutdown_connections 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c992710 0x55fb3c98abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c986f40 0x55fb3c991c70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/109523196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c987860 0x55fb3c98a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 -- 172.21.15.45:0/109523196 >> 172.21.15.45:0/109523196 conn(0x55fb3c8fc200 msgr2=0x55fb3c99e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 -- 172.21.15.45:0/109523196 shutdown_connections 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 -- 172.21.15.45:0/109523196 wait complete. 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.207+0000 7f90b4fabb80 1 -- start start 2023-09-23T09:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 1 Processor -- start 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c986f40 0x55fb3c997240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c997780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c992710 0x55fb3c997cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c9f5080 con 0x55fb3c992710 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c9f51f0 con 0x55fb3c987860 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3c9f53b0 con 0x55fb3c986f40 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90b4fabb80 10 client.0 _ll_register_callbacks cb 0x7f909400be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90a1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c997780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90a2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c992710 0x55fb3c997cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90a1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c997780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57598/0 (socket says 172.21.15.45:57598) 2023-09-23T09:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.208+0000 7f90a1d9b700 1 -- 172.21.15.45:0/2722532197 learned_addr learned my addr 172.21.15.45:0/2722532197 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90a1d9b700 1 -- 172.21.15.45:0/2722532197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c986f40 msgr2=0x55fb3c997240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c986f40 0x55fb3c997240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90a1d9b700 1 -- 172.21.15.45:0/2722532197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c992710 msgr2=0x55fb3c997cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c992710 0x55fb3c997cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90a1d9b700 1 -- 172.21.15.45:0/2722532197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb3c705d80 con 0x55fb3c987860 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90a1d9b700 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c997780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f909809a070 tx=0x7f909801dc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b8b62a029c7d0c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f9089ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f9089ffb700 1 -- 172.21.15.45:0/2722532197 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098025170 con 0x55fb3c987860 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f9089ffb700 1 -- 172.21.15.45:0/2722532197 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9098010e40 con 0x55fb3c987860 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f909c018d70 con 0x55fb3c987860 2023-09-23T09:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f9089ffb700 1 -- 172.21.15.45:0/2722532197 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90980cd320 con 0x55fb3c987860 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90b4fabb80 10 client.26459 fetch_fsmap learned FSMap version 48 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.209+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55fb3c72c1f0 con 0x55fb3c987860 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f9089ffb700 1 -- 172.21.15.45:0/2722532197 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909807d080 con 0x55fb3c987860 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f9089ffb700 1 -- 172.21.15.45:0/2722532197 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9098096a80 con 0x55fb3c987860 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90b4fabb80 10 client.26459 fetch_fsmap finished waiting for FSMap version 48 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90b4fabb80 10 client.26459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90b4fabb80 20 client.26459 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9f6b90 0x55fb3c9f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90b4fabb80 4 client.26459 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb3c8d3ad0 con 0x55fb3c9f6b90 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90a259c700 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9f6b90 0x55fb3c9f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f9089ffb700 10 client.26459 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.210+0000 7f90a259c700 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9f6b90 0x55fb3c9f8fb0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb514c4f9f9080fe server_cookie=877f05262a5c4583 in_seq=0 out_seq=0 2023-09-23T09:41:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f9089ffb700 1 -- 172.21.15.45:0/2722532197 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb3c8d3ad0 con 0x55fb3c9f6b90 2023-09-23T09:41:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f9089ffb700 10 client.26459 handle_command_reply: tid=0 2023-09-23T09:41:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 client.26459 shutdown 2023-09-23T09:41:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 msgr2=0x55fb3c997780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c997780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f909809a070 tx=0x7f909801dc80 comp rx=0 tx=0).stop 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 shutdown_connections 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55fb3c9f6b90 0x55fb3c9f8fb0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb3c992710 0x55fb3c997cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f9089ffb700 0 client.26459 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb3c986f40 0x55fb3c997240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 --2- 172.21.15.45:0/2722532197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb3c987860 0x55fb3c997780 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 >> 172.21.15.45:0/2722532197 conn(0x55fb3c8fc200 msgr2=0x55fb3c98a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 shutdown_connections 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 1 -- 172.21.15.45:0/2722532197 wait complete. 2023-09-23T09:41:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90b4fabb80 20 client.26459 trim_cache size 0 max 16384 2023-09-23T09:41:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:41:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:41:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:41:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:41:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a1790 msgr2=0x7f908c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a1790 0x7f908c0a3c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f909c01e000 tx=0x7f909c017f30 comp rx=0 tx=0).stop 2023-09-23T09:41:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100690 msgr2=0x7f90a4108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100690 0x7f90a4108510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f909800ea20 tx=0x7f909800ed30 comp rx=0 tx=0).stop 2023-09-23T09:41:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 shutdown_connections 2023-09-23T09:41:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4108a50 0x7f90a4108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a1790 0x7f908c0a3c50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a40ffcd0 0x7f90a410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 --2- 172.21.15.45:0/842289019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100690 0x7f90a4108510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 >> 172.21.15.45:0/842289019 conn(0x7f90a4076e20 msgr2=0x7f90a40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 shutdown_connections 2023-09-23T09:41:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:20.224+0000 7f90a8dc1700 1 -- 172.21.15.45:0/842289019 wait complete. 2023-09-23T09:41:20.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:19 smithi098 ceph-mon[102386]: pgmap v372: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 645 op/s 2023-09-23T09:41:20.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:19 smithi045 ceph-mon[98627]: pgmap v372: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 645 op/s 2023-09-23T09:41:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:19 smithi123 ceph-mon[103577]: pgmap v372: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 645 op/s 2023-09-23T09:41:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 695.0 s, 1627.1 tps, lat 62.618 ms stddev 72.036 2023-09-23T09:41:22.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:21 smithi098 ceph-mon[102386]: pgmap v373: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 676 op/s 2023-09-23T09:41:22.335 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:21 smithi045 ceph-mon[98627]: pgmap v373: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 676 op/s 2023-09-23T09:41:22.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:21 smithi123 ceph-mon[103577]: pgmap v373: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 676 op/s 2023-09-23T09:41:24.301 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:23 smithi098 ceph-mon[102386]: pgmap v374: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 676 op/s 2023-09-23T09:41:24.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:23 smithi045 ceph-mon[98627]: pgmap v374: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 676 op/s 2023-09-23T09:41:24.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:23 smithi123 ceph-mon[103577]: pgmap v374: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 676 op/s 2023-09-23T09:41:26.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:25 smithi098 ceph-mon[102386]: pgmap v375: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 688 op/s 2023-09-23T09:41:26.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:25 smithi045 ceph-mon[98627]: pgmap v375: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 688 op/s 2023-09-23T09:41:26.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:25 smithi123 ceph-mon[103577]: pgmap v375: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 688 op/s 2023-09-23T09:41:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 700.0 s, 1774.2 tps, lat 56.671 ms stddev 67.827 2023-09-23T09:41:28.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:27 smithi098 ceph-mon[102386]: pgmap v376: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 661 op/s 2023-09-23T09:41:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:27 smithi045 ceph-mon[98627]: pgmap v376: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 661 op/s 2023-09-23T09:41:28.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:27 smithi123 ceph-mon[103577]: pgmap v376: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 661 op/s 2023-09-23T09:41:29.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:29 smithi045 ceph-mon[98627]: pgmap v377: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 582 op/s 2023-09-23T09:41:29.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:29 smithi123 ceph-mon[103577]: pgmap v377: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 582 op/s 2023-09-23T09:41:29.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:29 smithi098 ceph-mon[102386]: pgmap v377: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 582 op/s 2023-09-23T09:41:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 705.0 s, 1004.8 tps, lat 98.000 ms stddev 203.647 2023-09-23T09:41:32.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:31 smithi098 ceph-mon[102386]: pgmap v378: 97 pgs: 97 active+clean; 5.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-09-23T09:41:32.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:31 smithi045 ceph-mon[98627]: pgmap v378: 97 pgs: 97 active+clean; 5.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-09-23T09:41:32.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:31 smithi123 ceph-mon[103577]: pgmap v378: 97 pgs: 97 active+clean; 5.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-09-23T09:41:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.544+0000 7fad2cea7700 1 -- 172.21.15.45:0/2586749184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 msgr2=0x7fad28108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.544+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2586749184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 0x7fad28108330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009990 tx=0x7fad1c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 -- 172.21.15.45:0/2586749184 shutdown_connections 2023-09-23T09:41:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2586749184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad28077070 0x7fad28079530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2586749184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad2810d7f0 0x7fad2810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2586749184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 0x7fad28108330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 -- 172.21.15.45:0/2586749184 >> 172.21.15.45:0/2586749184 conn(0x7fad2806cee0 msgr2=0x7fad2806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 -- 172.21.15.45:0/2586749184 shutdown_connections 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 -- 172.21.15.45:0/2586749184 wait complete. 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 Processor -- start 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.545+0000 7fad2cea7700 1 -- start start 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2cea7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad28077070 0x7fad2807ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2cea7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 0x7fad2807d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2cea7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad2810d7f0 0x7fad2807d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2cea7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281befe0 con 0x7fad28077070 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2cea7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281bf120 con 0x7fad28107eb0 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2cea7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281bf260 con 0x7fad2810d7f0 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad28077070 0x7fad2807ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad28077070 0x7fad2807ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36262/0 (socket says 172.21.15.45:36262) 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2659c700 1 -- 172.21.15.45:0/2748734958 learned_addr learned my addr 172.21.15.45:0/2748734958 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad26d9d700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad2810d7f0 0x7fad2807d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad25d9b700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 0x7fad2807d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad26d9d700 1 -- 172.21.15.45:0/2748734958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 msgr2=0x7fad2807d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad26d9d700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 0x7fad2807d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad26d9d700 1 -- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad28077070 msgr2=0x7fad2807ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad26d9d700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad28077070 0x7fad2807ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad26d9d700 1 -- 172.21.15.45:0/2748734958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0095f0 con 0x7fad2810d7f0 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad25d9b700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 0x7fad2807d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad2659c700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad28077070 0x7fad2807ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.546+0000 7fad26d9d700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad2810d7f0 0x7fad2807d8c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad20009f60 tx=0x7fad20009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.547+0000 7fad177fe700 1 -- 172.21.15.45:0/2748734958 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20018070 con 0x7fad2810d7f0 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.547+0000 7fad177fe700 1 -- 172.21.15.45:0/2748734958 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad2000f040 con 0x7fad2810d7f0 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.547+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad281bf490 con 0x7fad2810d7f0 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.547+0000 7fad177fe700 1 -- 172.21.15.45:0/2748734958 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000cd50 con 0x7fad2810d7f0 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.547+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad281bf970 con 0x7fad2810d7f0 2023-09-23T09:41:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.549+0000 7fad177fe700 1 -- 172.21.15.45:0/2748734958 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fad20003a00 con 0x7fad2810d7f0 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.550+0000 7fad177fe700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fad100a17d0 0x7fad100a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.550+0000 7fad177fe700 1 -- 172.21.15.45:0/2748734958 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad200ccd70 con 0x7fad2810d7f0 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.550+0000 7fad2659c700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fad100a17d0 0x7fad100a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.551+0000 7fad2659c700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fad100a17d0 0x7fad100a3c90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fad18011550 tx=0x7fad18009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.551+0000 7fad39091b80 1 Processor -- start 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.551+0000 7fad39091b80 1 -- start start 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad39091b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f1ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad39091b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad39091b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1b9710 0x55dc1f1bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad39091b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f117400 con 0x55dc1f1b9710 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad39091b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f108bc0 con 0x55dc1f1ae860 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad39091b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f1064e0 con 0x55dc1f1adf40 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f1ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad26d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1b9710 0x55dc1f1bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f1ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38376/0 (socket says 172.21.15.45:38376) 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad25d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 -- 172.21.15.45:0/3081185900 learned_addr learned my addr 172.21.15.45:0/3081185900 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 -- 172.21.15.45:0/3081185900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 msgr2=0x55dc1f1b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 --2- 172.21.15.45:0/3081185900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 -- 172.21.15.45:0/3081185900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1b9710 msgr2=0x55dc1f1bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 --2- 172.21.15.45:0/3081185900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1b9710 0x55dc1f1bbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 -- 172.21.15.45:0/3081185900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc1f164870 con 0x55dc1f1adf40 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.552+0000 7fad2659c700 1 --2- 172.21.15.45:0/3081185900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f1ae320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad1800e730 tx=0x7fad18014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9dee8580b57a168 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad14ff9700 1 -- 172.21.15.45:0/3081185900 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18004830 con 0x55dc1f1adf40 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad14ff9700 1 -- 172.21.15.45:0/3081185900 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad18011e90 con 0x55dc1f1adf40 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad14ff9700 1 -- 172.21.15.45:0/3081185900 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1801e680 con 0x55dc1f1adf40 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad39091b80 1 -- 172.21.15.45:0/3081185900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 msgr2=0x55dc1f1ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad39091b80 1 --2- 172.21.15.45:0/3081185900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f1ae320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad1800e730 tx=0x7fad18014000 comp rx=0 tx=0).stop 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad39091b80 1 -- 172.21.15.45:0/3081185900 shutdown_connections 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad39091b80 1 --2- 172.21.15.45:0/3081185900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1b9710 0x55dc1f1bbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad39091b80 1 --2- 172.21.15.45:0/3081185900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f1ae320 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad39091b80 1 --2- 172.21.15.45:0/3081185900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.553+0000 7fad39091b80 1 -- 172.21.15.45:0/3081185900 >> 172.21.15.45:0/3081185900 conn(0x55dc1f123200 msgr2=0x55dc1f1aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.554+0000 7fad39091b80 1 -- 172.21.15.45:0/3081185900 shutdown_connections 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.554+0000 7fad39091b80 1 -- 172.21.15.45:0/3081185900 wait complete. 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.554+0000 7fad39091b80 1 -- start start 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.554+0000 7fad39091b80 1 Processor -- start 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.554+0000 7fad39091b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1adf40 0x55dc1f1b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.554+0000 7fad39091b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.554+0000 7fad39091b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad39091b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f101dd0 con 0x55dc1f1adf40 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad39091b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f09e6b0 con 0x55dc1f1ae860 2023-09-23T09:41:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad39091b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f09e350 con 0x55dc1f1b9710 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad39091b80 10 client.0 _ll_register_callbacks cb 0x55dc1eefb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad2659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1adf40 0x55dc1f1b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1b6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38384/0 (socket says 172.21.15.45:38384) 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 -- 172.21.15.45:0/1385190454 learned_addr learned my addr 172.21.15.45:0/1385190454 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad25d9b700 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 -- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 msgr2=0x55dc1f1b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 -- 172.21.15.45:0/1385190454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1adf40 msgr2=0x55dc1f1b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1adf40 0x55dc1f1b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 -- 172.21.15.45:0/1385190454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad18017070 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad26d9d700 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1b6ee0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad20009390 tx=0x7fad20099810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b350d18430a129d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad0dffb700 1 -- 172.21.15.45:0/1385190454 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20099cd0 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad0dffb700 1 -- 172.21.15.45:0/1385190454 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad2009de20 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.555+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc1f1a2090 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad0dffb700 1 -- 172.21.15.45:0/1385190454 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad200a1fd0 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 10 client.26458 fetch_fsmap learned FSMap version 48 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55dc1ef2cd80 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad0dffb700 1 -- 172.21.15.45:0/1385190454 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc1f1a2090 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad0dffb700 1 -- 172.21.15.45:0/1385190454 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fad20065310 con 0x55dc1f1b9710 2023-09-23T09:41:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 10 client.26458 fetch_fsmap finished waiting for FSMap version 48 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 10 client.26458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 20 client.26458 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f1c9250 0x55dc1f1cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 4 client.26458 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc1f0fadb0 con 0x55dc1f1c9250 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.556+0000 7fad2659c700 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f1c9250 0x55dc1f1cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.557+0000 7fad0dffb700 10 client.26458 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.557+0000 7fad2659c700 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f1c9250 0x55dc1f1cb670 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e7284f1e747924 server_cookie=592f749c90652cd1 in_seq=0 out_seq=0 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.557+0000 7fad0dffb700 1 -- 172.21.15.45:0/1385190454 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc1f0fadb0 con 0x55dc1f1c9250 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.557+0000 7fad0dffb700 10 client.26458 handle_command_reply: tid=0 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.557+0000 7fad39091b80 1 client.26458 shutdown 2023-09-23T09:41:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 msgr2=0x55dc1f1b6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1b6ee0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad20009390 tx=0x7fad20099810 comp rx=0 tx=0).stop 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 shutdown_connections 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f1c9250 0x55dc1f1cb670 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1b6ee0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1adf40 0x55dc1f1b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 --2- 172.21.15.45:0/1385190454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1ae860 0x55dc1f1b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad0dffb700 0 client.26458 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 >> 172.21.15.45:0/1385190454 conn(0x55dc1f123200 msgr2=0x55dc1f1a9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 shutdown_connections 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 1 -- 172.21.15.45:0/1385190454 wait complete. 2023-09-23T09:41:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.558+0000 7fad39091b80 20 client.26458 trim_cache size 0 max 16384 2023-09-23T09:41:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.580+0000 7fad39091b80 1 Processor -- start 2023-09-23T09:41:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.580+0000 7fad39091b80 1 -- start start 2023-09-23T09:41:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad39091b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1adf40 0x55dc1f1c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad39091b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f1a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad39091b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad39091b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1ef828c0 con 0x55dc1f1ae860 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad39091b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f1af7a0 con 0x55dc1f1adf40 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad39091b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f1ab180 con 0x55dc1f1b9710 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad25d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f1a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1adf40 0x55dc1f1c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1adf40 0x55dc1f1c0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57634/0 (socket says 172.21.15.45:57634) 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 -- 172.21.15.45:0/2066039308 learned_addr learned my addr 172.21.15.45:0/2066039308 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad26d9d700 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 -- 172.21.15.45:0/2066039308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 msgr2=0x55dc1f1aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 -- 172.21.15.45:0/2066039308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 msgr2=0x55dc1f1a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f1a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 -- 172.21.15.45:0/2066039308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc1ef15c00 con 0x55dc1f1adf40 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad25d9b700 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f1a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.581+0000 7fad2659c700 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1adf40 0x55dc1f1c0260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad18009f50 tx=0x7fad18009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4daf5951af65dc57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad157fa700 1 -- 172.21.15.45:0/2066039308 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1802b040 con 0x55dc1f1adf40 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad157fa700 1 -- 172.21.15.45:0/2066039308 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad18037020 con 0x55dc1f1adf40 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad157fa700 1 -- 172.21.15.45:0/2066039308 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18033590 con 0x55dc1f1adf40 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 -- 172.21.15.45:0/2066039308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1adf40 msgr2=0x55dc1f1c0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1adf40 0x55dc1f1c0260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad18009f50 tx=0x7fad18009810 comp rx=0 tx=0).stop 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 -- 172.21.15.45:0/2066039308 shutdown_connections 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1b9710 0x55dc1f1aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1adf40 0x55dc1f1c0260 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 --2- 172.21.15.45:0/2066039308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f1a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 -- 172.21.15.45:0/2066039308 >> 172.21.15.45:0/2066039308 conn(0x55dc1f123200 msgr2=0x55dc1f1c5460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 -- 172.21.15.45:0/2066039308 shutdown_connections 2023-09-23T09:41:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.582+0000 7fad39091b80 1 -- 172.21.15.45:0/2066039308 wait complete. 2023-09-23T09:41:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 -- start start 2023-09-23T09:41:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 Processor -- start 2023-09-23T09:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f2154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f2159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1b9710 0x55dc1f211010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f211720 con 0x55dc1f1ae860 2023-09-23T09:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f211890 con 0x55dc1f1b9710 2023-09-23T09:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1f211a50 con 0x55dc1f1adf40 2023-09-23T09:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad39091b80 10 client.0 _ll_register_callbacks cb 0x7fad200960c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad2659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f2154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad26d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1b9710 0x55dc1f211010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad2659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f2154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38396/0 (socket says 172.21.15.45:38396) 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.583+0000 7fad2659c700 1 -- 172.21.15.45:0/3077108434 learned_addr learned my addr 172.21.15.45:0/3077108434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad25d9b700 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f2159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad2659c700 1 -- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1b9710 msgr2=0x55dc1f211010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad2659c700 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1b9710 0x55dc1f211010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad2659c700 1 -- 172.21.15.45:0/3077108434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 msgr2=0x55dc1f2159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad2659c700 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f2159f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad2659c700 1 -- 172.21.15.45:0/3077108434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2810d560 con 0x55dc1f1adf40 2023-09-23T09:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad26d9d700 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1b9710 0x55dc1f211010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad2659c700 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f2154b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fad1800e570 tx=0x7fad1800edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278ce2228671b157 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.584+0000 7fad0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.585+0000 7fad0dffb700 1 -- 172.21.15.45:0/3077108434 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18033bf0 con 0x55dc1f1adf40 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.585+0000 7fad0dffb700 1 -- 172.21.15.45:0/3077108434 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad18038030 con 0x55dc1f1adf40 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.585+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad18011e90 con 0x55dc1f1adf40 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad0dffb700 1 -- 172.21.15.45:0/3077108434 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad18033d50 con 0x55dc1f1adf40 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 10 client.26464 fetch_fsmap learned FSMap version 48 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55dc1f164870 con 0x55dc1f1adf40 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad0dffb700 1 -- 172.21.15.45:0/3077108434 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad18011e90 con 0x55dc1f1adf40 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad0dffb700 1 -- 172.21.15.45:0/3077108434 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fad18024030 con 0x55dc1f1adf40 2023-09-23T09:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 10 client.26464 fetch_fsmap finished waiting for FSMap version 48 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 20 client.26464 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f21e0d0 0x55dc1f2204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 4 client.26464 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc1f0faad0 con 0x55dc1f21e0d0 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.586+0000 7fad25d9b700 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f21e0d0 0x55dc1f2204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.587+0000 7fad0dffb700 10 client.26464 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.587+0000 7fad25d9b700 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f21e0d0 0x55dc1f2204f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e932f1f87ab1b65 server_cookie=8c456a15115bc47f in_seq=0 out_seq=0 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.601+0000 7fad0dffb700 1 -- 172.21.15.45:0/3077108434 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc1f0faad0 con 0x55dc1f21e0d0 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.601+0000 7fad0dffb700 10 client.26464 handle_command_reply: tid=0 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.601+0000 7fad39091b80 1 client.26464 shutdown 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 msgr2=0x55dc1f2154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f2154b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fad1800e570 tx=0x7fad1800edd0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 shutdown_connections 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dc1f21e0d0 0x55dc1f2204f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1f1b9710 0x55dc1f211010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad0dffb700 0 client.26464 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc1f1adf40 0x55dc1f2154b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 --2- 172.21.15.45:0/3077108434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc1f1ae860 0x55dc1f2159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 >> 172.21.15.45:0/3077108434 conn(0x55dc1f123200 msgr2=0x55dc1f1a9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 shutdown_connections 2023-09-23T09:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 1 -- 172.21.15.45:0/3077108434 wait complete. 2023-09-23T09:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad39091b80 20 client.26464 trim_cache size 0 max 16384 2023-09-23T09:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fad100a17d0 msgr2=0x7fad100a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fad100a17d0 0x7fad100a3c90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fad18011550 tx=0x7fad18009260 comp rx=0 tx=0).stop 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad2810d7f0 msgr2=0x7fad2807d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad2810d7f0 0x7fad2807d8c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad20009f60 tx=0x7fad20009f90 comp rx=0 tx=0).stop 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 shutdown_connections 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad2810d7f0 0x7fad2807d8c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fad100a17d0 0x7fad100a3c90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad28077070 0x7fad2807ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 --2- 172.21.15.45:0/2748734958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad28107eb0 0x7fad2807d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.602+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 >> 172.21.15.45:0/2748734958 conn(0x7fad2806cee0 msgr2=0x7fad28072080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.603+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 shutdown_connections 2023-09-23T09:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:33.603+0000 7fad2cea7700 1 -- 172.21.15.45:0/2748734958 wait complete. 2023-09-23T09:41:34.045 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:33 smithi098 ceph-mon[102386]: pgmap v379: 97 pgs: 97 active+clean; 5.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-09-23T09:41:34.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:33 smithi045 ceph-mon[98627]: pgmap v379: 97 pgs: 97 active+clean; 5.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-09-23T09:41:34.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:33 smithi123 ceph-mon[103577]: pgmap v379: 97 pgs: 97 active+clean; 5.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-09-23T09:41:36.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:35 smithi098 ceph-mon[102386]: pgmap v380: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 1.73k op/s 2023-09-23T09:41:36.086 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:35 smithi045 ceph-mon[98627]: pgmap v380: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 1.73k op/s 2023-09-23T09:41:36.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:35 smithi123 ceph-mon[103577]: pgmap v380: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 1.73k op/s 2023-09-23T09:41:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 710.0 s, 1251.2 tps, lat 80.129 ms stddev 94.441 2023-09-23T09:41:38.044 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:37 smithi098 ceph-mon[102386]: pgmap v381: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-09-23T09:41:38.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:41:38.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:37 smithi045 ceph-mon[98627]: pgmap v381: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-09-23T09:41:38.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:37 smithi123 ceph-mon[103577]: pgmap v381: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-09-23T09:41:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.404+0000 7f3c4239d700 1 -- 172.21.15.45:0/788274790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 msgr2=0x7f3c3c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.404+0000 7f3c4239d700 1 --2- 172.21.15.45:0/788274790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 0x7f3c3c1013d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009990 tx=0x7f3c2c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 -- 172.21.15.45:0/788274790 shutdown_connections 2023-09-23T09:41:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 --2- 172.21.15.45:0/788274790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c106110 0x7f3c3c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 --2- 172.21.15.45:0/788274790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 0x7f3c3c1013d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 --2- 172.21.15.45:0/788274790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 -- 172.21.15.45:0/788274790 >> 172.21.15.45:0/788274790 conn(0x7f3c3c0fc890 msgr2=0x7f3c3c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 -- 172.21.15.45:0/788274790 shutdown_connections 2023-09-23T09:41:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 -- 172.21.15.45:0/788274790 wait complete. 2023-09-23T09:41:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.408+0000 7f3c4239d700 1 Processor -- start 2023-09-23T09:41:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4239d700 1 -- start start 2023-09-23T09:41:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4239d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 0x7f3c3c10d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4239d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c106110 0x7f3c3c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4239d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c10a530 0x7f3c3c10a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4239d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c10afd0 con 0x7f3c3c106110 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4239d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c10b140 con 0x7f3c3c100ff0 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4239d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c10b300 con 0x7f3c3c10a530 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c10a530 0x7f3c3c10a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c10a530 0x7f3c3c10a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38414/0 (socket says 172.21.15.45:38414) 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 -- 172.21.15.45:0/2573650229 learned_addr learned my addr 172.21.15.45:0/2573650229 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c106110 0x7f3c3c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 0x7f3c3c10d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 -- 172.21.15.45:0/2573650229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 msgr2=0x7f3c3c10d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 0x7f3c3c10d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 -- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c106110 msgr2=0x7f3c3c10d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c106110 0x7f3c3c10d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.409+0000 7f3c4093a700 1 -- 172.21.15.45:0/2573650229 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c0095f0 con 0x7f3c3c10a530 2023-09-23T09:41:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.410+0000 7f3c4093a700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c10a530 0x7f3c3c10a9b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400ef30 tx=0x7f3c3400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.410+0000 7f3c397fa700 1 -- 172.21.15.45:0/2573650229 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34018070 con 0x7f3c3c10a530 2023-09-23T09:41:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.410+0000 7f3c397fa700 1 -- 172.21.15.45:0/2573650229 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3400f040 con 0x7f3c3c10a530 2023-09-23T09:41:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.410+0000 7f3c397fa700 1 -- 172.21.15.45:0/2573650229 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400cdb0 con 0x7f3c3c10a530 2023-09-23T09:41:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.410+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c1aa140 con 0x7f3c3c10a530 2023-09-23T09:41:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.410+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c1aa600 con 0x7f3c3c10a530 2023-09-23T09:41:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.413+0000 7f3c397fa700 1 -- 172.21.15.45:0/2573650229 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f3c34003a00 con 0x7f3c3c10a530 2023-09-23T09:41:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.413+0000 7f3c397fa700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c240a1710 0x7f3c240a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.414+0000 7f3c397fa700 1 -- 172.21.15.45:0/2573650229 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c340ccf40 con 0x7f3c3c10a530 2023-09-23T09:41:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.415+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c240a1710 0x7f3c240a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.415+0000 7f3c4e587b80 1 Processor -- start 2023-09-23T09:41:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4e587b80 1 -- start start 2023-09-23T09:41:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42400 0x55f6ace427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace4d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acdab460 con 0x55f6ace42400 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acd9cc20 con 0x55f6ace4dbd0 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acd9a540 con 0x55f6ace42d20 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c240a1710 0x7f3c240a3bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b2e0 tx=0x7f3c2c005fb0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42400 0x55f6ace427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57664/0 (socket says 172.21.15.45:57664) 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 -- 172.21.15.45:0/1143239447 learned_addr learned my addr 172.21.15.45:0/1143239447 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace4d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 -- 172.21.15.45:0/1143239447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 msgr2=0x55f6ace4d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace4d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 -- 172.21.15.45:0/1143239447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42400 msgr2=0x55f6ace427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42400 0x55f6ace427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.416+0000 7f3c4093a700 1 -- 172.21.15.45:0/1143239447 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6acdf88d0 con 0x55f6ace4dbd0 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.417+0000 7f3c4093a700 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4ffc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c340090a0 tx=0x7f3c34099d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=117b7f7f3ff16d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.417+0000 7f3c227fc700 1 -- 172.21.15.45:0/1143239447 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34004ad0 con 0x55f6ace4dbd0 2023-09-23T09:41:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.417+0000 7f3c227fc700 1 -- 172.21.15.45:0/1143239447 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3401b320 con 0x55f6ace4dbd0 2023-09-23T09:41:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.417+0000 7f3c227fc700 1 -- 172.21.15.45:0/1143239447 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34096610 con 0x55f6ace4dbd0 2023-09-23T09:41:38.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1143239447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 msgr2=0x55f6ace4ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4ffc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c340090a0 tx=0x7f3c34099d50 comp rx=0 tx=0).stop 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1143239447 shutdown_connections 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4ffc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42400 0x55f6ace427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1143239447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace4d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1143239447 >> 172.21.15.45:0/1143239447 conn(0x55f6acdb7260 msgr2=0x55f6ace3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1143239447 shutdown_connections 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1143239447 wait complete. 2023-09-23T09:41:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.418+0000 7f3c4e587b80 1 -- start start 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 1 Processor -- start 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42400 0x55f6ace4a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace4ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acd95e30 con 0x55f6ace42d20 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acd32710 con 0x55f6ace4dbd0 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acd323b0 con 0x55f6ace42400 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4e587b80 10 client.0 _ll_register_callbacks cb 0x55f6acb8f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42400 0x55f6ace4a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace4ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42400 0x55f6ace4a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38424/0 (socket says 172.21.15.45:38424) 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 -- 172.21.15.45:0/349770713 learned_addr learned my addr 172.21.15.45:0/349770713 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c4093a700 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 -- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 msgr2=0x55f6ace4b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 -- 172.21.15.45:0/349770713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 msgr2=0x55f6ace4ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace4ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3bfff700 1 -- 172.21.15.45:0/349770713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c1aa600 con 0x55f6ace42400 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.419+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace4ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42400 0x55f6ace4a920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c015000 tx=0x7f3c2c01a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b0e57c8092ec5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/349770713 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c031650 con 0x55f6ace42400 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/349770713 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c2c00fe40 con 0x55f6ace42400 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6ace36250 con 0x55f6ace42400 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/349770713 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c2c0317b0 con 0x55f6ace42400 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c4e587b80 10 client.26476 fetch_fsmap learned FSMap version 48 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55f6acbc0de0 con 0x55f6ace42400 2023-09-23T09:41:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.420+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/349770713 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6ace36250 con 0x55f6ace42400 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/349770713 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3c2c018520 con 0x55f6ace42400 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c4e587b80 10 client.26476 fetch_fsmap finished waiting for FSMap version 48 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c4e587b80 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c4e587b80 20 client.26476 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6ace5d740 0x55f6ace5fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c4e587b80 4 client.26476 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6acd8ee10 con 0x55f6ace5d740 2023-09-23T09:41:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6ace5d740 0x55f6ace5fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c1b7fe700 10 client.26476 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.421+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6ace5d740 0x55f6ace5fb60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4edafa5edc38d2 server_cookie=7ba03cfcbb88df30 in_seq=0 out_seq=0 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/349770713 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6acd8ee10 con 0x55f6ace5d740 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c1b7fe700 10 client.26476 handle_command_reply: tid=0 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 client.26476 shutdown 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42400 msgr2=0x55f6ace4a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42400 0x55f6ace4a920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c015000 tx=0x7f3c2c01a050 comp rx=0 tx=0).stop 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 shutdown_connections 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6ace5d740 0x55f6ace5fb60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace4dbd0 0x55f6ace4b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42400 0x55f6ace4a920 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c1b7fe700 0 client.26476 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/349770713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace4ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.422+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 >> 172.21.15.45:0/349770713 conn(0x55f6acdb7260 msgr2=0x55f6ace3e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.423+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 shutdown_connections 2023-09-23T09:41:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.423+0000 7f3c4e587b80 1 -- 172.21.15.45:0/349770713 wait complete. 2023-09-23T09:41:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.424+0000 7f3c4e587b80 20 client.26476 trim_cache size 0 max 16384 2023-09-23T09:41:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 Processor -- start 2023-09-23T09:41:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 -- start start 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace4b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace44ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace4dbd0 0x55f6ace45000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acc16920 con 0x55f6ace42d20 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ace42a30 con 0x55f6ace42400 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.448+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ace46770 con 0x55f6ace4dbd0 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace4dbd0 0x55f6ace45000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace4dbd0 0x55f6ace45000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38434/0 (socket says 172.21.15.45:38434) 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 -- 172.21.15.45:0/1674051244 learned_addr learned my addr 172.21.15.45:0/1674051244 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace4b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 -- 172.21.15.45:0/1674051244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 msgr2=0x55f6ace4b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace4b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 -- 172.21.15.45:0/1674051244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 msgr2=0x55f6ace44ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:41:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace44ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 -- 172.21.15.45:0/1674051244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ace3a480 con 0x55f6ace4dbd0 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.449+0000 7f3c4093a700 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace4dbd0 0x55f6ace45000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c34003f90 tx=0x7f3c34004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1521b0f059488e59 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c22ffd700 1 -- 172.21.15.45:0/1674051244 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3409a600 con 0x55f6ace4dbd0 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c22ffd700 1 -- 172.21.15.45:0/1674051244 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3409e050 con 0x55f6ace4dbd0 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c22ffd700 1 -- 172.21.15.45:0/1674051244 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3409a160 con 0x55f6ace4dbd0 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1674051244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace4dbd0 msgr2=0x55f6ace45000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace4dbd0 0x55f6ace45000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c34003f90 tx=0x7f3c34004dc0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1674051244 shutdown_connections 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace4dbd0 0x55f6ace45000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace4b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/1674051244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace42d20 0x55f6ace44ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1674051244 >> 172.21.15.45:0/1674051244 conn(0x55f6acdb7260 msgr2=0x55f6ace443b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1674051244 shutdown_connections 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.450+0000 7f3c4e587b80 1 -- 172.21.15.45:0/1674051244 wait complete. 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 -- start start 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 Processor -- start 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace52370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace4dbd0 0x55f6acec15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ace52f70 con 0x55f6ace4dbd0 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acec1ae0 con 0x55f6ace42400 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6acec1c20 con 0x55f6ace42d20 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.451+0000 7f3c4e587b80 10 client.0 _ll_register_callbacks cb 0x7f3c2c00bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace52370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace52370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57680/0 (socket says 172.21.15.45:57680) 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38438/0 (socket says 172.21.15.45:38438) 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3bfff700 1 -- 172.21.15.45:0/3633210794 learned_addr learned my addr 172.21.15.45:0/3633210794 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c4093a700 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace4dbd0 0x55f6acec15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 -- 172.21.15.45:0/3633210794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 msgr2=0x55f6ace52370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace52370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 -- 172.21.15.45:0/3633210794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace4dbd0 msgr2=0x55f6acec15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace4dbd0 0x55f6acec15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 -- 172.21.15.45:0/3633210794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6acdf88d0 con 0x55f6ace42d20 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c4093a700 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace4dbd0 0x55f6acec15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace528b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55f6ace60490 tx=0x7f3c30020d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e43e976a02fe05 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:41:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/3633210794 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30017070 con 0x55f6ace42d20 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c3401b320 con 0x55f6ace42d20 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/3633210794 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c30012cd0 con 0x55f6ace42d20 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/3633210794 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c300200e0 con 0x55f6ace42d20 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c4e587b80 10 client.26482 fetch_fsmap learned FSMap version 48 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.452+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55f6acba9c60 con 0x55f6ace42d20 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/3633210794 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3401b320 con 0x55f6ace42d20 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/3633210794 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3c300036f0 con 0x55f6ace42d20 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c4e587b80 10 client.26482 fetch_fsmap finished waiting for FSMap version 48 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c4e587b80 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c4e587b80 20 client.26482 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6acec3460 0x55f6acec5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c4e587b80 4 client.26482 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f6acd8eb30 con 0x55f6acec3460 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6acec3460 0x55f6acec5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c1b7fe700 10 client.26482 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.453+0000 7f3c3bfff700 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6acec3460 0x55f6acec5880 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e923ba27ac99c732 server_cookie=1fbe433a0966c0a0 in_seq=0 out_seq=0 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c1b7fe700 1 -- 172.21.15.45:0/3633210794 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f6acd8eb30 con 0x55f6acec3460 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c1b7fe700 10 client.26482 handle_command_reply: tid=0 2023-09-23T09:41:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 client.26482 shutdown 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 msgr2=0x55f6ace528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace528b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55f6ace60490 tx=0x7f3c30020d40 comp rx=0 tx=0).stop 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 shutdown_connections 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f6acec3460 0x55f6acec5880 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6ace4dbd0 0x55f6acec15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ace42400 0x55f6ace52370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 --2- 172.21.15.45:0/3633210794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ace42d20 0x55f6ace528b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c1b7fe700 0 client.26482 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 >> 172.21.15.45:0/3633210794 conn(0x55f6acdb7260 msgr2=0x55f6ace44b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.454+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 shutdown_connections 2023-09-23T09:41:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.455+0000 7f3c4e587b80 1 -- 172.21.15.45:0/3633210794 wait complete. 2023-09-23T09:41:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.455+0000 7f3c4e587b80 20 client.26482 trim_cache size 0 max 16384 2023-09-23T09:41:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:41:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:41:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.456+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c240a1710 msgr2=0x7f3c240a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.456+0000 7f3c4239d700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c240a1710 0x7f3c240a3bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b2e0 tx=0x7f3c2c005fb0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c10a530 msgr2=0x7f3c3c10a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c10a530 0x7f3c3c10a9b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400ef30 tx=0x7f3c3400c3a0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 shutdown_connections 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c3c10a530 0x7f3c3c10a9b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c3c100ff0 0x7f3c3c10d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f3c240a1710 0x7f3c240a3bd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 --2- 172.21.15.45:0/2573650229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c106110 0x7f3c3c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 >> 172.21.15.45:0/2573650229 conn(0x7f3c3c0fc890 msgr2=0x7f3c3c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 shutdown_connections 2023-09-23T09:41:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:41:38.457+0000 7f3c4239d700 1 -- 172.21.15.45:0/2573650229 wait complete. 2023-09-23T09:41:38.479 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:41:40.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:39 smithi098 ceph-mon[102386]: pgmap v382: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-09-23T09:41:40.089 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:39 smithi045 ceph-mon[98627]: pgmap v382: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-09-23T09:41:40.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:39 smithi123 ceph-mon[103577]: pgmap v382: 97 pgs: 97 active+clean; 5.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-09-23T09:41:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 715.0 s, 1548.4 tps, lat 64.794 ms stddev 69.532 2023-09-23T09:41:42.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:41 smithi098 ceph-mon[102386]: pgmap v383: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-23T09:41:42.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:41 smithi045 ceph-mon[98627]: pgmap v383: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-23T09:41:42.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:41 smithi123 ceph-mon[103577]: pgmap v383: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-23T09:41:44.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:43 smithi045 ceph-mon[98627]: pgmap v384: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2023-09-23T09:41:44.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:43 smithi123 ceph-mon[103577]: pgmap v384: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2023-09-23T09:41:44.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:43 smithi098 ceph-mon[102386]: pgmap v384: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2023-09-23T09:41:46.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:45 smithi045 ceph-mon[98627]: pgmap v385: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.61k op/s 2023-09-23T09:41:46.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:45 smithi123 ceph-mon[103577]: pgmap v385: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.61k op/s 2023-09-23T09:41:46.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:45 smithi098 ceph-mon[102386]: pgmap v385: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.61k op/s 2023-09-23T09:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 -- 172.21.15.45:0/3048744463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100ad0 msgr2=0x7f8534104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 --2- 172.21.15.45:0/3048744463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100ad0 0x7f8534104900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8524009990 tx=0x7f8524009ca0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 -- 172.21.15.45:0/3048744463 shutdown_connections 2023-09-23T09:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 --2- 172.21.15.45:0/3048744463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8534105030 0x7f853410cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 --2- 172.21.15.45:0/3048744463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8534100120 0x7f8534100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 --2- 172.21.15.45:0/3048744463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100ad0 0x7f8534104900 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 -- 172.21.15.45:0/3048744463 >> 172.21.15.45:0/3048744463 conn(0x7f8534077bf0 msgr2=0x7f8534078000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 -- 172.21.15.45:0/3048744463 shutdown_connections 2023-09-23T09:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 -- 172.21.15.45:0/3048744463 wait complete. 2023-09-23T09:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.723+0000 7f853a08c700 1 Processor -- start 2023-09-23T09:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f853a08c700 1 -- start start 2023-09-23T09:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f853a08c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100120 0x7f8534196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f853a08c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8534105030 0x7f85341974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f853a08c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f853419bb10 0x7f853419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f853a08c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853419c590 con 0x7f8534105030 2023-09-23T09:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f853a08c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853419c700 con 0x7f853419bb10 2023-09-23T09:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f853a08c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853419c8c0 con 0x7f8534100120 2023-09-23T09:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f8532ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8534105030 0x7f85341974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f85337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100120 0x7f8534196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f85337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100120 0x7f8534196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38450/0 (socket says 172.21.15.45:38450) 2023-09-23T09:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f85337fe700 1 -- 172.21.15.45:0/3003871574 learned_addr learned my addr 172.21.15.45:0/3003871574 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f8533fff700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f853419bb10 0x7f853419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f8532ffd700 1 -- 172.21.15.45:0/3003871574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100120 msgr2=0x7f8534196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f8532ffd700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100120 0x7f8534196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f8532ffd700 1 -- 172.21.15.45:0/3003871574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f853419bb10 msgr2=0x7f853419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f8532ffd700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f853419bb10 0x7f853419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f8532ffd700 1 -- 172.21.15.45:0/3003871574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240095f0 con 0x7f8534105030 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.724+0000 7f85337fe700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100120 0x7f8534196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.725+0000 7f8532ffd700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8534105030 0x7f85341974a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f853406a670 tx=0x7f8524004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.725+0000 7f8530ff9700 1 -- 172.21.15.45:0/3003871574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852401d070 con 0x7f8534105030 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.725+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85341018d0 con 0x7f8534105030 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.725+0000 7f8530ff9700 1 -- 172.21.15.45:0/3003871574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8524021410 con 0x7f8534105030 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.725+0000 7f8530ff9700 1 -- 172.21.15.45:0/3003871574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524017650 con 0x7f8534105030 2023-09-23T09:41:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.725+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8534101d80 con 0x7f8534105030 2023-09-23T09:41:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.727+0000 7f8530ff9700 1 -- 172.21.15.45:0/3003871574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f85240217f0 con 0x7f8534105030 2023-09-23T09:41:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.727+0000 7f8530ff9700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f851c0a1710 0x7f851c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.728+0000 7f8530ff9700 1 -- 172.21.15.45:0/3003871574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f85240ce5e0 con 0x7f8534105030 2023-09-23T09:41:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.728+0000 7f85337fe700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f851c0a1710 0x7f851c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.728+0000 7f85337fe700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f851c0a1710 0x7f851c0a3bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85280097f0 tx=0x7f8528012040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.728+0000 7f8546276b80 1 Processor -- start 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8546276b80 1 -- start start 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8546276b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593be6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8546276b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8546276b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8546276b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593b4f400 con 0x55b593bf1710 2023-09-23T09:41:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8546276b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593b40bc0 con 0x55b593be5f40 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8546276b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593b3e4e0 con 0x55b593be6860 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8533fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f85337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593be6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f85337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593be6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57702/0 (socket says 172.21.15.45:57702) 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f85337fe700 1 -- 172.21.15.45:0/1266150165 learned_addr learned my addr 172.21.15.45:0/1266150165 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8532ffd700 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8533fff700 1 -- 172.21.15.45:0/1266150165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 msgr2=0x55b593bf11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8533fff700 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8533fff700 1 -- 172.21.15.45:0/1266150165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 msgr2=0x55b593be6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8533fff700 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593be6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8533fff700 1 -- 172.21.15.45:0/1266150165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b593b9c870 con 0x55b593bf1710 2023-09-23T09:41:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f85337fe700 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593be6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.729+0000 7f8533fff700 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf3b00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f852c011550 tx=0x7f852c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece21f972f8b2bc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8519ffb700 1 -- 172.21.15.45:0/1266150165 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00e040 con 0x55b593bf1710 2023-09-23T09:41:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8519ffb700 1 -- 172.21.15.45:0/1266150165 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c00f040 con 0x55b593bf1710 2023-09-23T09:41:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8519ffb700 1 -- 172.21.15.45:0/1266150165 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c004360 con 0x55b593bf1710 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 -- 172.21.15.45:0/1266150165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 msgr2=0x55b593bf3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf3b00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f852c011550 tx=0x7f852c009920 comp rx=0 tx=0).stop 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 -- 172.21.15.45:0/1266150165 shutdown_connections 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf3b00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593be6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 --2- 172.21.15.45:0/1266150165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 -- 172.21.15.45:0/1266150165 >> 172.21.15.45:0/1266150165 conn(0x55b593b5b200 msgr2=0x55b593be23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 -- 172.21.15.45:0/1266150165 shutdown_connections 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.730+0000 7f8546276b80 1 -- 172.21.15.45:0/1266150165 wait complete. 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 -- start start 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 Processor -- start 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593bf1710 0x55b593beea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593b39dd0 con 0x55b593be6860 2023-09-23T09:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593ad66b0 con 0x55b593be5f40 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593ad6350 con 0x55b593bf1710 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8546276b80 10 client.0 _ll_register_callbacks cb 0x55b5939337a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8532ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8532ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bee390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36360/0 (socket says 172.21.15.45:36360) 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8532ffd700 1 -- 172.21.15.45:0/1848787666 learned_addr learned my addr 172.21.15.45:0/1848787666 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f85337fe700 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.731+0000 7f8533fff700 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593bf1710 0x55b593beea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8532ffd700 1 -- 172.21.15.45:0/1848787666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593bf1710 msgr2=0x55b593beea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8532ffd700 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593bf1710 0x55b593beea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8532ffd700 1 -- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 msgr2=0x55b593bf1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8532ffd700 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8532ffd700 1 -- 172.21.15.45:0/1848787666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852800b520 con 0x55b593be6860 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8532ffd700 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bee390 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8524000c00 tx=0x7f85240979b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67825f1ca25cc628 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8512ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8512ffd700 1 -- 172.21.15.45:0/1848787666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852409ba00 con 0x55b593be6860 2023-09-23T09:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8512ffd700 1 -- 172.21.15.45:0/1848787666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f852409bca0 con 0x55b593be6860 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b593bda090 con 0x55b593be6860 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8512ffd700 1 -- 172.21.15.45:0/1848787666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f852409af00 con 0x55b593be6860 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8546276b80 10 client.16827 fetch_fsmap learned FSMap version 49 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f852c011230 con 0x55b593be6860 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8512ffd700 1 -- 172.21.15.45:0/1848787666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b593bda090 con 0x55b593be6860 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.732+0000 7f8512ffd700 1 -- 172.21.15.45:0/1848787666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f852409abb0 con 0x55b593be6860 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f8546276b80 10 client.16827 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f8546276b80 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f8546276b80 20 client.16827 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f8546276b80 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c00de0 0x55b593c03200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f8546276b80 4 client.16827 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b593b32db0 con 0x55b593c00de0 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f85337fe700 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c00de0 0x55b593c03200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f8512ffd700 10 client.16827 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.733+0000 7f85337fe700 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c00de0 0x55b593c03200 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91a18acfc2f07c7 server_cookie=849be04c98d84f2e in_seq=0 out_seq=0 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8512ffd700 1 -- 172.21.15.45:0/1848787666 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b593b32db0 con 0x55b593c00de0 2023-09-23T09:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8512ffd700 10 client.16827 handle_command_reply: tid=0 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 client.16827 shutdown 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 msgr2=0x55b593bee390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bee390 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8524000c00 tx=0x7f85240979b0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 shutdown_connections 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c00de0 0x55b593c03200 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593bf1710 0x55b593beea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8512ffd700 0 client.16827 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 --2- 172.21.15.45:0/1848787666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bee390 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 >> 172.21.15.45:0/1848787666 conn(0x55b593b5b200 msgr2=0x55b593be1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 shutdown_connections 2023-09-23T09:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 1 -- 172.21.15.45:0/1848787666 wait complete. 2023-09-23T09:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.734+0000 7f8546276b80 20 client.16827 trim_cache size 0 max 16384 2023-09-23T09:41:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 Processor -- start 2023-09-23T09:41:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 -- start start 2023-09-23T09:41:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be5f40 0x55b593bedb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bf0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593bf1710 0x55b593be9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5939ba8c0 con 0x55b593be6860 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593bef970 con 0x55b593bf1710 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8546276b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593be3100 con 0x55b593be5f40 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8532ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bf0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f85337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be5f40 0x55b593bedb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8532ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bf0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36368/0 (socket says 172.21.15.45:36368) 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8532ffd700 1 -- 172.21.15.45:0/3329503615 learned_addr learned my addr 172.21.15.45:0/3329503615 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f8533fff700 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593bf1710 0x55b593be9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f85337fe700 1 -- 172.21.15.45:0/3329503615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593bf1710 msgr2=0x55b593be9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f85337fe700 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593bf1710 0x55b593be9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f85337fe700 1 -- 172.21.15.45:0/3329503615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 msgr2=0x55b593bf0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f85337fe700 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bf0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f85337fe700 1 -- 172.21.15.45:0/3329503615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b593964d80 con 0x55b593be5f40 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.749+0000 7f85337fe700 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be5f40 0x55b593bedb50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55b593c03ad0 tx=0x7f852801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e822394e9d4ef9c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f851a7fc700 1 -- 172.21.15.45:0/3329503615 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852802e040 con 0x55b593be5f40 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f851a7fc700 1 -- 172.21.15.45:0/3329503615 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8528016040 con 0x55b593be5f40 2023-09-23T09:41:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f851a7fc700 1 -- 172.21.15.45:0/3329503615 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280053e0 con 0x55b593be5f40 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 -- 172.21.15.45:0/3329503615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be5f40 msgr2=0x55b593bedb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be5f40 0x55b593bedb50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55b593c03ad0 tx=0x7f852801c000 comp rx=0 tx=0).stop 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 -- 172.21.15.45:0/3329503615 shutdown_connections 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593bf1710 0x55b593be9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be5f40 0x55b593bedb50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 --2- 172.21.15.45:0/3329503615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593be6860 0x55b593bf0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 -- 172.21.15.45:0/3329503615 >> 172.21.15.45:0/3329503615 conn(0x55b593b5b200 msgr2=0x55b593bfd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 -- 172.21.15.45:0/3329503615 shutdown_connections 2023-09-23T09:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.750+0000 7f8546276b80 1 -- 172.21.15.45:0/3329503615 wait complete. 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 -- start start 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 Processor -- start 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593bf68e0 con 0x55b593bf1710 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593bf6a20 con 0x55b593be5f40 2023-09-23T09:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b593bf6be0 con 0x55b593be6860 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8546276b80 10 client.0 _ll_register_callbacks cb 0x7f85240860b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f85337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8533fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f85337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57722/0 (socket says 172.21.15.45:57722) 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f85337fe700 1 -- 172.21.15.45:0/918098721 learned_addr learned my addr 172.21.15.45:0/918098721 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f8532ffd700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.751+0000 7f85337fe700 1 -- 172.21.15.45:0/918098721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 msgr2=0x55b593bf8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f85337fe700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f85337fe700 1 -- 172.21.15.45:0/918098721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 msgr2=0x55b593bf6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f85337fe700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f85337fe700 1 -- 172.21.15.45:0/918098721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852800b850 con 0x55b593be5f40 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8533fff700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8532ffd700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f85337fe700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf7c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8528009320 tx=0x7f852802f300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9635eb5f830a4dba server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8512ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8512ffd700 1 -- 172.21.15.45:0/918098721 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852802e040 con 0x55b593be5f40 2023-09-23T09:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f852c00f040 con 0x55b593be5f40 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8512ffd700 1 -- 172.21.15.45:0/918098721 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f852802f4f0 con 0x55b593be5f40 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8512ffd700 1 -- 172.21.15.45:0/918098721 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8528029790 con 0x55b593be5f40 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8546276b80 10 client.26513 fetch_fsmap learned FSMap version 49 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b59394dc00 con 0x55b593be5f40 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8512ffd700 1 -- 172.21.15.45:0/918098721 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c00f040 con 0x55b593be5f40 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8512ffd700 1 -- 172.21.15.45:0/918098721 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8528037440 con 0x55b593be5f40 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8546276b80 10 client.26513 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8546276b80 10 client.26513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.752+0000 7f8546276b80 20 client.26513 populate_metadata read hostname 'smithi045' 2023-09-23T09:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.753+0000 7f8546276b80 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c55db0 0x55b593c581d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.753+0000 7f8546276b80 4 client.26513 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.753+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b593b32ad0 con 0x55b593c55db0 2023-09-23T09:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.753+0000 7f8532ffd700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c55db0 0x55b593c581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.753+0000 7f8532ffd700 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c55db0 0x55b593c581d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13f109b61bcd064 server_cookie=3a776993018b7cc3 in_seq=0 out_seq=0 2023-09-23T09:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.753+0000 7f8512ffd700 10 client.26513 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8512ffd700 1 -- 172.21.15.45:0/918098721 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b593b32ad0 con 0x55b593c55db0 2023-09-23T09:41:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8512ffd700 10 client.26513 handle_command_reply: tid=0 2023-09-23T09:41:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 client.26513 shutdown 2023-09-23T09:41:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 msgr2=0x55b593bf7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf7c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8528009320 tx=0x7f852802f300 comp rx=0 tx=0).stop 2023-09-23T09:41:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 shutdown_connections 2023-09-23T09:41:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b593c55db0 0x55b593c581d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b593bf1710 0x55b593bf6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b593be5f40 0x55b593bf7c10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8512ffd700 0 client.26513 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 --2- 172.21.15.45:0/918098721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b593be6860 0x55b593bf8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 >> 172.21.15.45:0/918098721 conn(0x55b593b5b200 msgr2=0x55b593be9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 shutdown_connections 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 1 -- 172.21.15.45:0/918098721 wait complete. 2023-09-23T09:41:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.774+0000 7f8546276b80 20 client.26513 trim_cache size 0 max 16384 2023-09-23T09:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f851c0a1710 msgr2=0x7f851c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f851c0a1710 0x7f851c0a3bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85280097f0 tx=0x7f8528012040 comp rx=0 tx=0).stop 2023-09-23T09:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8534105030 msgr2=0x7f85341974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8534105030 0x7f85341974a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f853406a670 tx=0x7f8524004730 comp rx=0 tx=0).stop 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 shutdown_connections 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f853419bb10 0x7f853419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8534100120 0x7f8534196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f851c0a1710 0x7f851c0a3bd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 --2- 172.21.15.45:0/3003871574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8534105030 0x7f85341974a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 >> 172.21.15.45:0/3003871574 conn(0x7f8534077bf0 msgr2=0x7f8534068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 shutdown_connections 2023-09-23T09:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:41:46.775+0000 7f853a08c700 1 -- 172.21.15.45:0/3003871574 wait complete. 2023-09-23T09:41:46.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 720.0 s, 454.0 tps, lat 100.664 ms stddev 107.544 2023-09-23T09:41:47.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:46 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:47.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:46 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:47.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:46 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:41:48.083 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:47 smithi045 ceph-mon[98627]: pgmap v386: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-09-23T09:41:48.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:47 smithi123 ceph-mon[103577]: pgmap v386: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-09-23T09:41:48.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:47 smithi098 ceph-mon[102386]: pgmap v386: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-09-23T09:41:50.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:49 smithi045 ceph-mon[98627]: pgmap v387: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 884 op/s 2023-09-23T09:41:50.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:49 smithi123 ceph-mon[103577]: pgmap v387: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 884 op/s 2023-09-23T09:41:50.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:49 smithi098 ceph-mon[102386]: pgmap v387: 97 pgs: 97 active+clean; 5.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 884 op/s 2023-09-23T09:41:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 725.0 s, 843.2 tps, lat 177.877 ms stddev 501.247 2023-09-23T09:41:52.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:51 smithi045 ceph-mon[98627]: pgmap v388: 97 pgs: 97 active+clean; 5.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2023-09-23T09:41:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:51 smithi123 ceph-mon[103577]: pgmap v388: 97 pgs: 97 active+clean; 5.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2023-09-23T09:41:52.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:51 smithi098 ceph-mon[102386]: pgmap v388: 97 pgs: 97 active+clean; 5.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2023-09-23T09:41:54.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:53 smithi123 ceph-mon[103577]: pgmap v389: 97 pgs: 97 active+clean; 5.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 964 op/s 2023-09-23T09:41:54.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:53 smithi098 ceph-mon[102386]: pgmap v389: 97 pgs: 97 active+clean; 5.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 964 op/s 2023-09-23T09:41:54.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:53 smithi045 ceph-mon[98627]: pgmap v389: 97 pgs: 97 active+clean; 5.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 964 op/s 2023-09-23T09:41:56.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:55 smithi123 ceph-mon[103577]: pgmap v390: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-09-23T09:41:56.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:55 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:41:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:55 smithi098 ceph-mon[102386]: pgmap v390: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-09-23T09:41:56.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:55 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:41:56.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:55 smithi045 ceph-mon[98627]: pgmap v390: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-09-23T09:41:56.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:55 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:41:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 730.0 s, 408.8 tps, lat 256.455 ms stddev 773.872 2023-09-23T09:41:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:57 smithi123 ceph-mon[103577]: pgmap v391: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2023-09-23T09:41:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:57 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:41:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:57 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:41:58.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:57 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:41:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:57 smithi098 ceph-mon[102386]: pgmap v391: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2023-09-23T09:41:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:57 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:41:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:57 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:41:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:57 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:41:58.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:57 smithi045 ceph-mon[98627]: pgmap v391: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2023-09-23T09:41:58.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:57 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:41:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:57 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:41:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:57 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 -- 172.21.15.45:0/4092930317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc105050 msgr2=0x7ff6fc10cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 --2- 172.21.15.45:0/4092930317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc105050 0x7ff6fc10cbd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400b0d0 tx=0x7ff6f400b3e0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 -- 172.21.15.45:0/4092930317 shutdown_connections 2023-09-23T09:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 --2- 172.21.15.45:0/4092930317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc105050 0x7ff6fc10cbd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 --2- 172.21.15.45:0/4092930317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6fc100160 0x7ff6fc100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 --2- 172.21.15.45:0/4092930317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc100b10 0x7ff6fc104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 -- 172.21.15.45:0/4092930317 >> 172.21.15.45:0/4092930317 conn(0x7ff6fc076840 msgr2=0x7ff6fc076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 -- 172.21.15.45:0/4092930317 shutdown_connections 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.062+0000 7ff70287f700 1 -- 172.21.15.45:0/4092930317 wait complete. 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 Processor -- start 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 -- start start 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc100160 0x7ff6fc101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc105050 0x7ff6fc1020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6fc1a3170 0x7ff6fc1a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc1a5aa0 con 0x7ff6fc100160 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc1a5be0 con 0x7ff6fc105050 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.063+0000 7ff70287f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc1a5d50 con 0x7ff6fc1a3170 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc100160 0x7ff6fc101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc100160 0x7ff6fc101ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36384/0 (socket says 172.21.15.45:36384) 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6fbfff700 1 -- 172.21.15.45:0/2250915957 learned_addr learned my addr 172.21.15.45:0/2250915957 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff700e1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6fc1a3170 0x7ff6fc1a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6fb7fe700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc105050 0x7ff6fc1020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff700e1c700 1 -- 172.21.15.45:0/2250915957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc105050 msgr2=0x7ff6fc1020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff700e1c700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc105050 0x7ff6fc1020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff700e1c700 1 -- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc100160 msgr2=0x7ff6fc101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff700e1c700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc100160 0x7ff6fc101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff700e1c700 1 -- 172.21.15.45:0/2250915957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f4009d00 con 0x7ff6fc1a3170 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc100160 0x7ff6fc101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff700e1c700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6fc1a3170 0x7ff6fc1a5560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400b770 tx=0x7ff6f4007870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6f97fa700 1 -- 172.21.15.45:0/2250915957 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400e040 con 0x7ff6fc1a3170 2023-09-23T09:42:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc1a5fd0 con 0x7ff6fc1a3170 2023-09-23T09:42:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6f97fa700 1 -- 172.21.15.45:0/2250915957 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f401d930 con 0x7ff6fc1a3170 2023-09-23T09:42:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff6f97fa700 1 -- 172.21.15.45:0/2250915957 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f401b820 con 0x7ff6fc1a3170 2023-09-23T09:42:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.064+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc1a6450 con 0x7ff6fc1a3170 2023-09-23T09:42:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.068+0000 7ff6f97fa700 1 -- 172.21.15.45:0/2250915957 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7ff6f401d470 con 0x7ff6fc1a3170 2023-09-23T09:42:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.068+0000 7ff6f97fa700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff6e40a1710 0x7ff6e40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff6f97fa700 1 -- 172.21.15.45:0/2250915957 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff6f40ce600 con 0x7ff6fc1a3170 2023-09-23T09:42:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff6e40a1710 0x7ff6e40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 Processor -- start 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 -- start start 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff6e40a1710 0x7ff6e40a3bd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0011550 tx=0x7ff6f0009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a1e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d197c400 con 0x55e2d1a1e710 2023-09-23T09:42:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d196dbc0 con 0x55e2d1a13860 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.069+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d196b4e0 con 0x55e2d1a12f40 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff700e1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a13320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38498/0 (socket says 172.21.15.45:38498) 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3577133455 learned_addr learned my addr 172.21.15.45:0/3577133455 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6fb7fe700 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a1e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff700e1c700 1 -- 172.21.15.45:0/3577133455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 msgr2=0x55e2d1a13320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff700e1c700 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a13320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff700e1c700 1 -- 172.21.15.45:0/3577133455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 msgr2=0x55e2d1a1e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff700e1c700 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a1e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff700e1c700 1 -- 172.21.15.45:0/3577133455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2d19c9870 con 0x55e2d1a1e710 2023-09-23T09:42:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff700e1c700 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a20b00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40cbd10 tx=0x7ff6f4006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61c242d6aa8753b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6e27fc700 1 -- 172.21.15.45:0/3577133455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f409b5c0 con 0x55e2d1a1e710 2023-09-23T09:42:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6e27fc700 1 -- 172.21.15.45:0/3577133455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f409f710 con 0x55e2d1a1e710 2023-09-23T09:42:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.070+0000 7ff6e27fc700 1 -- 172.21.15.45:0/3577133455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4099a40 con 0x55e2d1a1e710 2023-09-23T09:42:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3577133455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 msgr2=0x55e2d1a20b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a20b00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40cbd10 tx=0x7ff6f4006060 comp rx=0 tx=0).stop 2023-09-23T09:42:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3577133455 shutdown_connections 2023-09-23T09:42:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a20b00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3577133455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a1e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3577133455 >> 172.21.15.45:0/3577133455 conn(0x55e2d1988200 msgr2=0x55e2d1a0f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3577133455 shutdown_connections 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.071+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3577133455 wait complete. 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 -- start start 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 Processor -- start 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a1e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a1b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d1966dd0 con 0x55e2d1a13860 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d19036b0 con 0x55e2d1a12f40 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d1903350 con 0x55e2d1a1e710 2023-09-23T09:42:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff70ea69b80 10 client.0 _ll_register_callbacks cb 0x55e2d17607a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a1e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a1e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57748/0 (socket says 172.21.15.45:57748) 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3768972992 learned_addr learned my addr 172.21.15.45:0/3768972992 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.072+0000 7ff700e1c700 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3768972992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 msgr2=0x55e2d1a1bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3768972992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 msgr2=0x55e2d1a1b440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a1b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3768972992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f0017070 con 0x55e2d1a12f40 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a1e120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009cf0 tx=0x7ff6f00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a8987110bf35d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:42:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6db7fe700 1 -- 172.21.15.45:0/3768972992 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0022070 con 0x55e2d1a12f40 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6db7fe700 1 -- 172.21.15.45:0/3768972992 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff6f001dbb0 con 0x55e2d1a12f40 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2d1a07090 con 0x55e2d1a12f40 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff6db7fe700 1 -- 172.21.15.45:0/3768972992 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6f001ddb0 con 0x55e2d1a12f40 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff70ea69b80 10 client.26525 fetch_fsmap learned FSMap version 49 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.073+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7ff6fc1a6450 con 0x55e2d1a12f40 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff6db7fe700 1 -- 172.21.15.45:0/3768972992 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2d1a07090 con 0x55e2d1a12f40 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff6db7fe700 1 -- 172.21.15.45:0/3768972992 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff6f001c520 con 0x55e2d1a12f40 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff70ea69b80 10 client.26525 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff70ea69b80 10 client.26525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff70ea69b80 20 client.26525 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a2e0c0 0x55e2d1a304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff70ea69b80 4 client.26525 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:42:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2d195fdb0 con 0x55e2d1a2e0c0 2023-09-23T09:42:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.074+0000 7ff6fb7fe700 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a2e0c0 0x55e2d1a304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.075+0000 7ff6db7fe700 10 client.26525 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.075+0000 7ff6fb7fe700 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a2e0c0 0x55e2d1a304e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e2d3c551f0a43c server_cookie=af21634ab57893dd in_seq=0 out_seq=0 2023-09-23T09:42:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.076+0000 7ff6db7fe700 1 -- 172.21.15.45:0/3768972992 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2d195fdb0 con 0x55e2d1a2e0c0 2023-09-23T09:42:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.076+0000 7ff6db7fe700 10 client.26525 handle_command_reply: tid=0 2023-09-23T09:42:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.076+0000 7ff70ea69b80 1 client.26525 shutdown 2023-09-23T09:42:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 msgr2=0x55e2d1a1e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a1e120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009cf0 tx=0x7ff6f00047c0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 shutdown_connections 2023-09-23T09:42:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a2e0c0 0x55e2d1a304e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff6db7fe700 0 client.26525 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a1e120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3768972992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a1b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 >> 172.21.15.45:0/3768972992 conn(0x55e2d1988200 msgr2=0x55e2d1a0ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 shutdown_connections 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3768972992 wait complete. 2023-09-23T09:42:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.077+0000 7ff70ea69b80 20 client.26525 trim_cache size 0 max 16384 2023-09-23T09:42:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.095+0000 7ff70ea69b80 1 Processor -- start 2023-09-23T09:42:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.095+0000 7ff70ea69b80 1 -- start start 2023-09-23T09:42:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.095+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a15eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.095+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a25100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d17e78c0 con 0x55e2d1a1e710 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d1a256b0 con 0x55e2d1a13860 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d1a19ac0 con 0x55e2d1a12f40 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff700e1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a15eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a25100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a15eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38510/0 (socket says 172.21.15.45:38510) 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a25100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57756/0 (socket says 172.21.15.45:57756) 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff700e1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36408/0 (socket says 172.21.15.45:36408) 2023-09-23T09:42:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3646102573 learned_addr learned my addr 172.21.15.45:0/3646102573 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3646102573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 msgr2=0x55e2d1a25100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a25100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3646102573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 msgr2=0x55e2d1a1aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 -- 172.21.15.45:0/3646102573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2d1791d80 con 0x55e2d1a12f40 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff700e1c700 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a15eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000c00 tx=0x7ff6f0009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d9cc6cde90d93a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.096+0000 7ff6e2ffd700 1 -- 172.21.15.45:0/3646102573 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0022030 con 0x55e2d1a12f40 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff6e2ffd700 1 -- 172.21.15.45:0/3646102573 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f001ca40 con 0x55e2d1a12f40 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff6e2ffd700 1 -- 172.21.15.45:0/3646102573 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0004d70 con 0x55e2d1a12f40 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3646102573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 msgr2=0x55e2d1a15eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a15eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000c00 tx=0x7ff6f0009ec0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3646102573 shutdown_connections 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a1e710 0x55e2d1a1aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a12f40 0x55e2d1a15eb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/3646102573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a13860 0x55e2d1a25100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3646102573 >> 172.21.15.45:0/3646102573 conn(0x55e2d1988200 msgr2=0x55e2d1a0e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3646102573 shutdown_connections 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.097+0000 7ff70ea69b80 1 -- 172.21.15.45:0/3646102573 wait complete. 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 -- start start 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 Processor -- start 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a24d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a23070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d1a23af0 con 0x55e2d1a13860 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d1a23c60 con 0x55e2d1a12f40 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2d1a23e20 con 0x55e2d1a1e710 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff70ea69b80 10 client.0 _ll_register_callbacks cb 0x7ff6f000e980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a23070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a24d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a23070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36412/0 (socket says 172.21.15.45:36412) 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a24d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57760/0 (socket says 172.21.15.45:57760) 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff6fb7fe700 1 -- 172.21.15.45:0/4029697574 learned_addr learned my addr 172.21.15.45:0/4029697574 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.098+0000 7ff700e1c700 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6fbfff700 1 -- 172.21.15.45:0/4029697574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 msgr2=0x55e2d1a235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a235b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6fbfff700 1 -- 172.21.15.45:0/4029697574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 msgr2=0x55e2d1a23070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a23070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6fbfff700 1 -- 172.21.15.45:0/4029697574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2d19c9870 con 0x55e2d1a12f40 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6fb7fe700 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a23070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6fbfff700 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a24d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6f001c850 tx=0x7ff6f001df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba9e547f770477f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6db7fe700 1 -- 172.21.15.45:0/4029697574 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0022040 con 0x55e2d1a12f40 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6f409f710 con 0x55e2d1a12f40 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6db7fe700 1 -- 172.21.15.45:0/4029697574 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff6f0023070 con 0x55e2d1a12f40 2023-09-23T09:42:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff6db7fe700 1 -- 172.21.15.45:0/4029697574 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6f0036620 con 0x55e2d1a12f40 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff70ea69b80 10 client.26531 fetch_fsmap learned FSMap version 49 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.099+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7ff6fc1a6450 con 0x55e2d1a12f40 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff6db7fe700 1 -- 172.21.15.45:0/4029697574 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f409f710 con 0x55e2d1a12f40 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff6db7fe700 1 -- 172.21.15.45:0/4029697574 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff6f0041860 con 0x55e2d1a12f40 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff70ea69b80 10 client.26531 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff70ea69b80 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff70ea69b80 20 client.26531 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a82fe0 0x55e2d1a85400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff70ea69b80 4 client.26531 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:42:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2d195fad0 con 0x55e2d1a82fe0 2023-09-23T09:42:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.100+0000 7ff6fb7fe700 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a82fe0 0x55e2d1a85400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.101+0000 7ff6fb7fe700 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a82fe0 0x55e2d1a85400 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d99539384191056 server_cookie=d377c4396a0b8e86 in_seq=0 out_seq=0 2023-09-23T09:42:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.101+0000 7ff6db7fe700 10 client.26531 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff6db7fe700 1 -- 172.21.15.45:0/4029697574 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2d195fad0 con 0x55e2d1a82fe0 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff6db7fe700 10 client.26531 handle_command_reply: tid=0 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 client.26531 shutdown 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 msgr2=0x55e2d1a24d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a24d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6f001c850 tx=0x7ff6f001df60 comp rx=0 tx=0).stop 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 shutdown_connections 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e2d1a82fe0 0x55e2d1a85400 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2d1a1e710 0x55e2d1a235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff6db7fe700 0 client.26531 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2d1a12f40 0x55e2d1a24d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 --2- 172.21.15.45:0/4029697574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2d1a13860 0x55e2d1a23070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 >> 172.21.15.45:0/4029697574 conn(0x55e2d1988200 msgr2=0x55e2d1a0ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 shutdown_connections 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 1 -- 172.21.15.45:0/4029697574 wait complete. 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.126+0000 7ff70ea69b80 20 client.26531 trim_cache size 0 max 16384 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:42:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:42:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff6e40a1710 msgr2=0x7ff6e40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff6e40a1710 0x7ff6e40a3bd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0011550 tx=0x7ff6f0009260 comp rx=0 tx=0).stop 2023-09-23T09:42:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6fc1a3170 msgr2=0x7ff6fc1a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6fc1a3170 0x7ff6fc1a5560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400b770 tx=0x7ff6f4007870 comp rx=0 tx=0).stop 2023-09-23T09:42:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 shutdown_connections 2023-09-23T09:42:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6fc1a3170 0x7ff6fc1a5560 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff6e40a1710 0x7ff6e40a3bd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6fc100160 0x7ff6fc101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 --2- 172.21.15.45:0/2250915957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc105050 0x7ff6fc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 >> 172.21.15.45:0/2250915957 conn(0x7ff6fc076840 msgr2=0x7ff6fc068250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 shutdown_connections 2023-09-23T09:42:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:00.127+0000 7ff70287f700 1 -- 172.21.15.45:0/2250915957 wait complete. 2023-09-23T09:42:00.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:41:59 smithi098 ceph-mon[102386]: pgmap v392: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 911 op/s 2023-09-23T09:42:00.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:41:59 smithi045 ceph-mon[98627]: pgmap v392: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 911 op/s 2023-09-23T09:42:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:41:59 smithi123 ceph-mon[103577]: pgmap v392: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 911 op/s 2023-09-23T09:42:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 735.0 s, 1151.2 tps, lat 86.096 ms stddev 105.971 2023-09-23T09:42:02.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:01 smithi098 ceph-mon[102386]: pgmap v393: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-09-23T09:42:02.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:01 smithi045 ceph-mon[98627]: pgmap v393: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-09-23T09:42:02.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:01 smithi123 ceph-mon[103577]: pgmap v393: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-09-23T09:42:04.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:03 smithi098 ceph-mon[102386]: pgmap v394: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 536 op/s 2023-09-23T09:42:04.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:03 smithi045 ceph-mon[98627]: pgmap v394: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 536 op/s 2023-09-23T09:42:04.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:03 smithi123 ceph-mon[103577]: pgmap v394: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 536 op/s 2023-09-23T09:42:06.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:05 smithi098 ceph-mon[102386]: pgmap v395: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 746 op/s 2023-09-23T09:42:06.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:05 smithi098 ceph-mon[102386]: osdmap e100: 12 total, 12 up, 12 in 2023-09-23T09:42:06.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:05 smithi123 ceph-mon[103577]: pgmap v395: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 746 op/s 2023-09-23T09:42:06.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:05 smithi123 ceph-mon[103577]: osdmap e100: 12 total, 12 up, 12 in 2023-09-23T09:42:06.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:05 smithi045 ceph-mon[98627]: pgmap v395: 97 pgs: 97 active+clean; 6.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 746 op/s 2023-09-23T09:42:06.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:05 smithi045 ceph-mon[98627]: osdmap e100: 12 total, 12 up, 12 in 2023-09-23T09:42:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 740.0 s, 1011.3 tps, lat 98.732 ms stddev 127.133 2023-09-23T09:42:08.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:07 smithi098 ceph-mon[102386]: pgmap v397: 97 pgs: 97 active+clean; 6.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 947 op/s 2023-09-23T09:42:08.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:07 smithi045 ceph-mon[98627]: pgmap v397: 97 pgs: 97 active+clean; 6.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 947 op/s 2023-09-23T09:42:08.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:07 smithi123 ceph-mon[103577]: pgmap v397: 97 pgs: 97 active+clean; 6.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 947 op/s 2023-09-23T09:42:08.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:42:08.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.768+0000 7ff9cac97700 1 -- 172.21.15.45:0/4210919014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f940 msgr2=0x7ff9c410d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.768+0000 7ff9cac97700 1 --2- 172.21.15.45:0/4210919014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f940 0x7ff9c410d5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.769+0000 7ff9cac97700 1 -- 172.21.15.45:0/4210919014 shutdown_connections 2023-09-23T09:42:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.769+0000 7ff9cac97700 1 --2- 172.21.15.45:0/4210919014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c411a600 0x7ff9c411c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.769+0000 7ff9cac97700 1 --2- 172.21.15.45:0/4210919014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c410f4d0 0x7ff9c410d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.769+0000 7ff9cac97700 1 --2- 172.21.15.45:0/4210919014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f940 0x7ff9c410d5a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.769+0000 7ff9cac97700 1 -- 172.21.15.45:0/4210919014 >> 172.21.15.45:0/4210919014 conn(0x7ff9c406d0e0 msgr2=0x7ff9c406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.769+0000 7ff9cac97700 1 -- 172.21.15.45:0/4210919014 shutdown_connections 2023-09-23T09:42:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.770+0000 7ff9cac97700 1 -- 172.21.15.45:0/4210919014 wait complete. 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.770+0000 7ff9cac97700 1 Processor -- start 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9cac97700 1 -- start start 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9cac97700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f4d0 0x7ff9c41158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9cac97700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c410f940 0x7ff9c4115e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9cac97700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c411a600 0x7ff9c41a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9cac97700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a97d0 con 0x7ff9c410f940 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9cac97700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a9940 con 0x7ff9c411a600 2023-09-23T09:42:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9cac97700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a9b00 con 0x7ff9c410f4d0 2023-09-23T09:42:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9c9234700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c411a600 0x7ff9c41a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f4d0 0x7ff9c41158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9c9234700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c411a600 0x7ff9c41a9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57776/0 (socket says 172.21.15.45:57776) 2023-09-23T09:42:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f4d0 0x7ff9c41158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38534/0 (socket says 172.21.15.45:38534) 2023-09-23T09:42:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9c9234700 1 -- 172.21.15.45:0/94157861 learned_addr learned my addr 172.21.15.45:0/94157861 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.771+0000 7ff9c3fff700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c410f940 0x7ff9c4115e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c9234700 1 -- 172.21.15.45:0/94157861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f4d0 msgr2=0x7ff9c41158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c9234700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f4d0 0x7ff9c41158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c9234700 1 -- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c410f940 msgr2=0x7ff9c4115e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c9234700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c410f940 0x7ff9c4115e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c9234700 1 -- 172.21.15.45:0/94157861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80095f0 con 0x7ff9c411a600 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c3fff700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c410f940 0x7ff9c4115e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c9234700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c411a600 0x7ff9c41a9290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc041fd0 tx=0x7ff9bc03f3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c1ffb700 1 -- 172.21.15.45:0/94157861 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc01a2e0 con 0x7ff9c411a600 2023-09-23T09:42:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c1ffb700 1 -- 172.21.15.45:0/94157861 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc004bb0 con 0x7ff9c411a600 2023-09-23T09:42:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c40730b0 con 0x7ff9c411a600 2023-09-23T09:42:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.772+0000 7ff9c1ffb700 1 -- 172.21.15.45:0/94157861 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc004020 con 0x7ff9c411a600 2023-09-23T09:42:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.773+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c40735c0 con 0x7ff9c411a600 2023-09-23T09:42:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.776+0000 7ff9c1ffb700 1 -- 172.21.15.45:0/94157861 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7ff9bc004d20 con 0x7ff9c411a600 2023-09-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.777+0000 7ff9c1ffb700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9ac0a1860 0x7ff9ac0a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.777+0000 7ff9c1ffb700 1 -- 172.21.15.45:0/94157861 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff9bc0ffde0 con 0x7ff9c411a600 2023-09-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.777+0000 7ff9d6e81b80 1 Processor -- start 2023-09-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.777+0000 7ff9d6e81b80 1 -- start start 2023-09-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.777+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9ac0a1860 0x7ff9ac0a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e5079b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079bbbd0 0x55e5079bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e507919460 con 0x55e5079b0d20 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50790ac20 con 0x55e5079bbbd0 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e507908540 con 0x55e5079b0400 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e5079b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e5079b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38538/0 (socket says 172.21.15.45:38538) 2023-09-23T09:42:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 -- 172.21.15.45:0/2633058215 learned_addr learned my addr 172.21.15.45:0/2633058215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c3fff700 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c9234700 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079bbbd0 0x55e5079bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9ac0a1860 0x7ff9ac0a3d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009cb0 tx=0x7ff9b4009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 -- 172.21.15.45:0/2633058215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079bbbd0 msgr2=0x55e5079bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079bbbd0 0x55e5079bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 -- 172.21.15.45:0/2633058215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 msgr2=0x55e5079bb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079bb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.778+0000 7ff9c8a33700 1 -- 172.21.15.45:0/2633058215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5079668d0 con 0x55e5079b0400 2023-09-23T09:42:08.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.779+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e5079b07e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009fd0 tx=0x7ff9b401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c86c8203784cdd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.779+0000 7ff9aaffd700 1 -- 172.21.15.45:0/2633058215 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b401ce30 con 0x55e5079b0400 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.779+0000 7ff9aaffd700 1 -- 172.21.15.45:0/2633058215 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b4004cf0 con 0x55e5079b0400 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.779+0000 7ff9aaffd700 1 -- 172.21.15.45:0/2633058215 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4024680 con 0x55e5079b0400 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2633058215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 msgr2=0x55e5079b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e5079b07e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009fd0 tx=0x7ff9b401c000 comp rx=0 tx=0).stop 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2633058215 shutdown_connections 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079bbbd0 0x55e5079bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e5079b07e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2633058215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2633058215 >> 172.21.15.45:0/2633058215 conn(0x55e507925260 msgr2=0x55e5079ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2633058215 shutdown_connections 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2633058215 wait complete. 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.780+0000 7ff9d6e81b80 1 -- start start 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 1 Processor -- start 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0400 0x55e5079b8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e507903e30 con 0x55e5079b0d20 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5078a0710 con 0x55e5079b0400 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5078a03b0 con 0x55e5079bbbd0 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9d6e81b80 10 client.0 _ll_register_callbacks cb 0x55e5076fd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0400 0x55e5079b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c9234700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0400 0x55e5079b8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57788/0 (socket says 172.21.15.45:57788) 2023-09-23T09:42:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 -- 172.21.15.45:0/1843639399 learned_addr learned my addr 172.21.15.45:0/1843639399 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c9234700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38548/0 (socket says 172.21.15.45:38548) 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 -- 172.21.15.45:0/1843639399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 msgr2=0x55e5079b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 -- 172.21.15.45:0/1843639399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 msgr2=0x55e5079b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c8a33700 1 -- 172.21.15.45:0/1843639399 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4017050 con 0x55e5079b0400 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c9234700 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.781+0000 7ff9c3fff700 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0400 0x55e5079b8920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4026040 tx=0x7ff9b4005380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7cb8b41891c3346 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9a3fff700 1 -- 172.21.15.45:0/1843639399 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4005060 con 0x55e5079b0400 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5079a4250 con 0x55e5079b0400 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9a3fff700 1 -- 172.21.15.45:0/1843639399 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff9b4003680 con 0x55e5079b0400 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9a3fff700 1 -- 172.21.15.45:0/1843639399 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b4003840 con 0x55e5079b0400 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 10 client.26543 fetch_fsmap learned FSMap version 49 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e50772ede0 con 0x55e5079b0400 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9a3fff700 1 -- 172.21.15.45:0/1843639399 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5079a4250 con 0x55e5079b0400 2023-09-23T09:42:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9a3fff700 1 -- 172.21.15.45:0/1843639399 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff9b402d970 con 0x55e5079b0400 2023-09-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 10 client.26543 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 10 client.26543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 20 client.26543 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e5079cb5f0 0x55e5079cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 4 client.26543 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.782+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5078fce10 con 0x55e5079cb5f0 2023-09-23T09:42:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.783+0000 7ff9c3fff700 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e5079cb5f0 0x55e5079cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.784+0000 7ff9a3fff700 10 client.26543 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.784+0000 7ff9c3fff700 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e5079cb5f0 0x55e5079cda10 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4cc0e7b2e6828d8 server_cookie=10d61cf3eeb58d74 in_seq=0 out_seq=0 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9a3fff700 1 -- 172.21.15.45:0/1843639399 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e5078fce10 con 0x55e5079cb5f0 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9a3fff700 10 client.26543 handle_command_reply: tid=0 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 client.26543 shutdown 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0400 msgr2=0x55e5079b8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0400 0x55e5079b8920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4026040 tx=0x7ff9b4005380 comp rx=0 tx=0).stop 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 shutdown_connections 2023-09-23T09:42:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e5079cb5f0 0x55e5079cda10 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0400 0x55e5079b8920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9a3fff700 0 client.26543 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1843639399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0d20 0x55e5079b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 >> 172.21.15.45:0/1843639399 conn(0x55e507925260 msgr2=0x55e5079ac320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 shutdown_connections 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.785+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1843639399 wait complete. 2023-09-23T09:42:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.786+0000 7ff9d6e81b80 20 client.26543 trim_cache size 0 max 16384 2023-09-23T09:42:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 Processor -- start 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 -- start start 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0400 0x55e5079b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e5079c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e507784920 con 0x55e5079b0400 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5079b1100 con 0x55e5079b0d20 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5079b6ea0 con 0x55e5079bbbd0 2023-09-23T09:42:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0400 0x55e5079b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e5079c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c9234700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e5079c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57796/0 (socket says 172.21.15.45:57796) 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0400 0x55e5079b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36446/0 (socket says 172.21.15.45:36446) 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c3fff700 1 -- 172.21.15.45:0/1200916864 learned_addr learned my addr 172.21.15.45:0/1200916864 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c8a33700 1 -- 172.21.15.45:0/1200916864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 msgr2=0x55e5079b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c8a33700 1 -- 172.21.15.45:0/1200916864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 msgr2=0x55e5079c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e5079c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c8a33700 1 -- 172.21.15.45:0/1200916864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e507717c60 con 0x55e5079b0400 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.806+0000 7ff9c3fff700 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e5079c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.807+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0400 0x55e5079b3370 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005460 tx=0x7ff9b4009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b690f022f0f092e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.807+0000 7ff9ab7fe700 1 -- 172.21.15.45:0/1200916864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40092d0 con 0x55e5079b0400 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.807+0000 7ff9ab7fe700 1 -- 172.21.15.45:0/1200916864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b4005550 con 0x55e5079b0400 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.807+0000 7ff9ab7fe700 1 -- 172.21.15.45:0/1200916864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4024700 con 0x55e5079b0400 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.807+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1200916864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0400 msgr2=0x55e5079b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.807+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0400 0x55e5079b3370 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005460 tx=0x7ff9b4009770 comp rx=0 tx=0).stop 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1200916864 shutdown_connections 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079bbbd0 0x55e5079b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079b0400 0x55e5079b3370 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/1200916864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e5079c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1200916864 >> 172.21.15.45:0/1200916864 conn(0x55e507925260 msgr2=0x55e5079b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1200916864 shutdown_connections 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/1200916864 wait complete. 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 -- start start 2023-09-23T09:42:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.808+0000 7ff9d6e81b80 1 Processor -- start 2023-09-23T09:42:08.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e507a16dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e507a17300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079bbbd0 0x55e5079c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5079c2340 con 0x55e5079bbbd0 2023-09-23T09:42:08.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5079c24b0 con 0x55e5079b0d20 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5079c2670 con 0x55e5079b0400 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 10 client.0 _ll_register_callbacks cb 0x7ff9b4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079bbbd0 0x55e5079c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e507a17300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e507a16dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e507a17300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57804/0 (socket says 172.21.15.45:57804) 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079bbbd0 0x55e5079c1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36456/0 (socket says 172.21.15.45:36456) 2023-09-23T09:42:08.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c8a33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e507a16dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38558/0 (socket says 172.21.15.45:38558) 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 -- 172.21.15.45:0/2949986247 learned_addr learned my addr 172.21.15.45:0/2949986247 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 -- 172.21.15.45:0/2949986247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 msgr2=0x55e507a16dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e507a16dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 -- 172.21.15.45:0/2949986247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 msgr2=0x55e507a17300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e507a17300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 -- 172.21.15.45:0/2949986247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c411d2f0 con 0x55e5079bbbd0 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9c9234700 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079bbbd0 0x55e5079c1c00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc03f7b0 tx=0x7ff9bc0cd220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68bad273dfe0c5b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9a3fff700 1 -- 172.21.15.45:0/2949986247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0d4de0 con 0x55e5079bbbd0 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9a3fff700 1 -- 172.21.15.45:0/2949986247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff9bc0e53e0 con 0x55e5079bbbd0 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.809+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9b4004cf0 con 0x55e5079bbbd0 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9a3fff700 1 -- 172.21.15.45:0/2949986247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9bc04c310 con 0x55e5079bbbd0 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 10 client.16866 fetch_fsmap learned FSMap version 49 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e50772ede0 con 0x55e5079bbbd0 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9a3fff700 1 -- 172.21.15.45:0/2949986247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b4004cf0 con 0x55e5079bbbd0 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9a3fff700 1 -- 172.21.15.45:0/2949986247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff9bc003a00 con 0x55e5079bbbd0 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 10 client.16866 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:42:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 20 client.16866 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e507a1fde0 0x55e507a22200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 4 client.16866 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e5078fcb30 con 0x55e507a1fde0 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.810+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e507a1fde0 0x55e507a22200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9a3fff700 10 client.16866 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9c8a33700 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e507a1fde0 0x55e507a22200 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450a2685a6a6c081 server_cookie=90730fcd225ad7f8 in_seq=0 out_seq=0 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9a3fff700 1 -- 172.21.15.45:0/2949986247 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e5078fcb30 con 0x55e507a1fde0 2023-09-23T09:42:08.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9a3fff700 10 client.16866 handle_command_reply: tid=0 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 client.16866 shutdown 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079bbbd0 msgr2=0x55e5079c1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079bbbd0 0x55e5079c1c00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc03f7b0 tx=0x7ff9bc0cd220 comp rx=0 tx=0).stop 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 shutdown_connections 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55e507a1fde0 0x55e507a22200 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5079bbbd0 0x55e5079c1c00 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9a3fff700 0 client.16866 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5079b0400 0x55e507a16dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 --2- 172.21.15.45:0/2949986247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5079b0d20 0x55e507a17300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.812+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 >> 172.21.15.45:0/2949986247 conn(0x55e507925260 msgr2=0x55e5079b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:08.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 shutdown_connections 2023-09-23T09:42:08.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9d6e81b80 1 -- 172.21.15.45:0/2949986247 wait complete. 2023-09-23T09:42:08.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9d6e81b80 20 client.16866 trim_cache size 0 max 16384 2023-09-23T09:42:08.824 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:42:08.824 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:42:08.825 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9ac0a1860 msgr2=0x7ff9ac0a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9ac0a1860 0x7ff9ac0a3d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009cb0 tx=0x7ff9b4009260 comp rx=0 tx=0).stop 2023-09-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c411a600 msgr2=0x7ff9c41a9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:08.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c411a600 0x7ff9c41a9290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc041fd0 tx=0x7ff9bc03f3b0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 shutdown_connections 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c411a600 0x7ff9c41a9290 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9c410f4d0 0x7ff9c41158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9ac0a1860 0x7ff9ac0a3d20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 --2- 172.21.15.45:0/94157861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c410f940 0x7ff9c4115e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 >> 172.21.15.45:0/94157861 conn(0x7ff9c406d0e0 msgr2=0x7ff9c411a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 shutdown_connections 2023-09-23T09:42:08.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:08.813+0000 7ff9cac97700 1 -- 172.21.15.45:0/94157861 wait complete. 2023-09-23T09:42:08.829 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:42:10.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:09 smithi098 ceph-mon[102386]: pgmap v398: 97 pgs: 97 active+clean; 6.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 946 op/s 2023-09-23T09:42:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:09 smithi045 ceph-mon[98627]: pgmap v398: 97 pgs: 97 active+clean; 6.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 946 op/s 2023-09-23T09:42:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:09 smithi123 ceph-mon[103577]: pgmap v398: 97 pgs: 97 active+clean; 6.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 946 op/s 2023-09-23T09:42:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 745.0 s, 1335.1 tps, lat 75.247 ms stddev 81.364 2023-09-23T09:42:12.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:11 smithi098 ceph-mon[102386]: pgmap v399: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 974 op/s 2023-09-23T09:42:12.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:11 smithi045 ceph-mon[98627]: pgmap v399: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 974 op/s 2023-09-23T09:42:12.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:11 smithi123 ceph-mon[103577]: pgmap v399: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 974 op/s 2023-09-23T09:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.902+0000 7f90a9b33700 1 -- 172.21.15.45:0/216652082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4105050 msgr2=0x7f90a410cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.902+0000 7f90a9b33700 1 --2- 172.21.15.45:0/216652082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4105050 0x7f90a410cbd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f90940099c0 tx=0x7f9094009cd0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.904+0000 7f90a9b33700 1 -- 172.21.15.45:0/216652082 shutdown_connections 2023-09-23T09:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.904+0000 7f90a9b33700 1 --2- 172.21.15.45:0/216652082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4105050 0x7f90a410cbd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.904+0000 7f90a9b33700 1 --2- 172.21.15.45:0/216652082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4100160 0x7f90a4100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.904+0000 7f90a9b33700 1 --2- 172.21.15.45:0/216652082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a4100b10 0x7f90a4104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.904+0000 7f90a9b33700 1 -- 172.21.15.45:0/216652082 >> 172.21.15.45:0/216652082 conn(0x7f90a4076840 msgr2=0x7f90a4076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.914+0000 7f90a9b33700 1 -- 172.21.15.45:0/216652082 shutdown_connections 2023-09-23T09:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.925+0000 7f90a9b33700 1 -- 172.21.15.45:0/216652082 wait complete. 2023-09-23T09:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.925+0000 7f90a9b33700 1 Processor -- start 2023-09-23T09:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.926+0000 7f90a9b33700 1 -- start start 2023-09-23T09:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a9b33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4100160 0x7f90a41153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a9b33700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4105050 0x7f90a41158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a9b33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a410f2e0 0x7f90a410f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a9b33700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a410fed0 con 0x7f90a410f2e0 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a9b33700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4110040 con 0x7f90a4105050 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a9b33700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40708f0 con 0x7f90a4100160 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a410f2e0 0x7f90a410f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a410f2e0 0x7f90a410f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36466/0 (socket says 172.21.15.45:36466) 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.927+0000 7f90a3fff700 1 -- 172.21.15.45:0/3056700019 learned_addr learned my addr 172.21.15.45:0/3056700019 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a3fff700 1 -- 172.21.15.45:0/3056700019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4100160 msgr2=0x7f90a41153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a3fff700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4100160 0x7f90a41153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a3fff700 1 -- 172.21.15.45:0/3056700019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4105050 msgr2=0x7f90a41158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a3fff700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4105050 0x7f90a41158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a3fff700 1 -- 172.21.15.45:0/3056700019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940095f0 con 0x7f90a410f2e0 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a3fff700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a410f2e0 0x7f90a410f790 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9094006010 tx=0x7f9094003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a0ff9700 1 -- 172.21.15.45:0/3056700019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909401d070 con 0x7f90a410f2e0 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a0ff9700 1 -- 172.21.15.45:0/3056700019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9094004040 con 0x7f90a410f2e0 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a4070b70 con 0x7f90a410f2e0 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a0ff9700 1 -- 172.21.15.45:0/3056700019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094005290 con 0x7f90a410f2e0 2023-09-23T09:42:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.928+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a4070fc0 con 0x7f90a410f2e0 2023-09-23T09:42:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.932+0000 7f90a0ff9700 1 -- 172.21.15.45:0/3056700019 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f90940047e0 con 0x7f90a410f2e0 2023-09-23T09:42:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.932+0000 7f90a0ff9700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a18a0 0x7f908c0a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90a0ff9700 1 -- 172.21.15.45:0/3056700019 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f90940ce070 con 0x7f90a410f2e0 2023-09-23T09:42:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 Processor -- start 2023-09-23T09:42:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 -- start start 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15b6860 0x564ae15c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15c1710 0x564ae15c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae151f400 con 0x564ae15b6860 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae1510bc0 con 0x564ae15b5f40 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae150e4e0 con 0x564ae15c1710 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.933+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15b6860 0x564ae15c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15b6860 0x564ae15c11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36474/0 (socket says 172.21.15.45:36474) 2023-09-23T09:42:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1419163421 learned_addr learned my addr 172.21.15.45:0/1419163421 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a3fff700 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15c1710 0x564ae15c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1419163421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15c1710 msgr2=0x564ae15c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15c1710 0x564ae15c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1419163421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 msgr2=0x564ae15b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1419163421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ae156c870 con 0x564ae15b6860 2023-09-23T09:42:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.934+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15b6860 0x564ae15c11d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f909c009d30 tx=0x7f909c00b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275ee79af1f3ef7d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.940+0000 7f90a37fe700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a18a0 0x7f908c0a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.946+0000 7f9091ffb700 1 -- 172.21.15.45:0/1419163421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c00bea0 con 0x564ae15b6860 2023-09-23T09:42:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.946+0000 7f9091ffb700 1 -- 172.21.15.45:0/1419163421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909c0138d0 con 0x564ae15b6860 2023-09-23T09:42:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.946+0000 7f9091ffb700 1 -- 172.21.15.45:0/1419163421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c012950 con 0x564ae15b6860 2023-09-23T09:42:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.947+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1419163421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15b6860 msgr2=0x564ae15c11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.947+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15b6860 0x564ae15c11d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f909c009d30 tx=0x7f909c00b6f0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.947+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1419163421 shutdown_connections 2023-09-23T09:42:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.947+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15c1710 0x564ae15c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.947+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.947+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1419163421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15b6860 0x564ae15c11d0 secure :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f909c009d30 tx=0x7f909c00b6f0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.947+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1419163421 >> 172.21.15.45:0/1419163421 conn(0x564ae152b200 msgr2=0x564ae15b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.949+0000 7f90a37fe700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a18a0 0x7f908c0a3d60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9098005fd0 tx=0x7f9098005f00 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.954+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1419163421 shutdown_connections 2023-09-23T09:42:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.954+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1419163421 wait complete. 2023-09-23T09:42:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 -- start start 2023-09-23T09:42:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 Processor -- start 2023-09-23T09:42:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15c1710 0x564ae15beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15bf0c0 0x564ae15cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae1509dd0 con 0x564ae15c1710 2023-09-23T09:42:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae14a66b0 con 0x564ae15b5f40 2023-09-23T09:42:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae14a6350 con 0x564ae15bf0c0 2023-09-23T09:42:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.955+0000 7f90b5d1db80 10 client.0 _ll_register_callbacks cb 0x564ae13037a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15c1710 0x564ae15beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15c1710 0x564ae15beb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36482/0 (socket says 172.21.15.45:36482) 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1102094194 learned_addr learned my addr 172.21.15.45:0/1102094194 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1102094194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15bf0c0 msgr2=0x564ae15cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15bf0c0 0x564ae15cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1102094194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 msgr2=0x564ae15be640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.956+0000 7f90a2ffd700 1 -- 172.21.15.45:0/1102094194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c009990 con 0x564ae15c1710 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.957+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15c1710 0x564ae15beb80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f909c016040 tx=0x7f909c0136f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a6a8413249972e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.957+0000 7f9086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.958+0000 7f9086ffd700 1 -- 172.21.15.45:0/1102094194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c00bea0 con 0x564ae15c1710 2023-09-23T09:42:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.958+0000 7f9086ffd700 1 -- 172.21.15.45:0/1102094194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f909c012700 con 0x564ae15c1710 2023-09-23T09:42:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.958+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ae15aa090 con 0x564ae15c1710 2023-09-23T09:42:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.959+0000 7f9086ffd700 1 -- 172.21.15.45:0/1102094194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f909c0128c0 con 0x564ae15c1710 2023-09-23T09:42:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.959+0000 7f90b5d1db80 10 client.16875 fetch_fsmap learned FSMap version 49 2023-09-23T09:42:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.959+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564ae1334d80 con 0x564ae15c1710 2023-09-23T09:42:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.960+0000 7f9086ffd700 1 -- 172.21.15.45:0/1102094194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909c02b030 con 0x564ae15c1710 2023-09-23T09:42:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f9086ffd700 1 -- 172.21.15.45:0/1102094194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f909c008030 con 0x564ae15c1710 2023-09-23T09:42:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f90b5d1db80 10 client.16875 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:42:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f90b5d1db80 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f90b5d1db80 20 client.16875 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae15d1730 0x564ae15d3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f90b5d1db80 4 client.16875 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:42:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ae1502db0 con 0x564ae15d1730 2023-09-23T09:42:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.961+0000 7f90a37fe700 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae15d1730 0x564ae15d3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.962+0000 7f9086ffd700 10 client.16875 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.962+0000 7f90a37fe700 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae15d1730 0x564ae15d3b50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80096dfad60348c9 server_cookie=ab5991245d2c5f01 in_seq=0 out_seq=0 2023-09-23T09:42:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f9086ffd700 1 -- 172.21.15.45:0/1102094194 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564ae1502db0 con 0x564ae15d1730 2023-09-23T09:42:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f9086ffd700 10 client.16875 handle_command_reply: tid=0 2023-09-23T09:42:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 client.16875 shutdown 2023-09-23T09:42:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15c1710 msgr2=0x564ae15beb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15c1710 0x564ae15beb80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f909c016040 tx=0x7f909c0136f0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 shutdown_connections 2023-09-23T09:42:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae15d1730 0x564ae15d3b50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15bf0c0 0x564ae15cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15be640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f9086ffd700 0 client.16875 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1102094194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15c1710 0x564ae15beb80 secure :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f909c016040 tx=0x7f909c0136f0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.963+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 >> 172.21.15.45:0/1102094194 conn(0x564ae152b200 msgr2=0x564ae15b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.964+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 shutdown_connections 2023-09-23T09:42:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.964+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1102094194 wait complete. 2023-09-23T09:42:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.964+0000 7f90b5d1db80 20 client.16875 trim_cache size 0 max 16384 2023-09-23T09:42:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.990+0000 7f90b5d1db80 1 Processor -- start 2023-09-23T09:42:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.990+0000 7f90b5d1db80 1 -- start start 2023-09-23T09:42:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15bf0c0 0x564ae15b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15ae240 0x564ae15cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae138a8c0 con 0x564ae15bf0c0 2023-09-23T09:42:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae15b3100 con 0x564ae15b5f40 2023-09-23T09:42:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae15c87a0 con 0x564ae15ae240 2023-09-23T09:42:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15ae240 0x564ae15cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15ae240 0x564ae15cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38596/0 (socket says 172.21.15.45:38596) 2023-09-23T09:42:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90a3fff700 1 -- 172.21.15.45:0/504630555 learned_addr learned my addr 172.21.15.45:0/504630555 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15bf0c0 0x564ae15b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.991+0000 7f90a37fe700 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.992+0000 7f90a3fff700 1 -- 172.21.15.45:0/504630555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 msgr2=0x564ae15c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.992+0000 7f90a3fff700 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.992+0000 7f90a3fff700 1 -- 172.21.15.45:0/504630555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15bf0c0 msgr2=0x564ae15b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.992+0000 7f90a3fff700 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15bf0c0 0x564ae15b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.992+0000 7f90a3fff700 1 -- 172.21.15.45:0/504630555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ae131dc00 con 0x564ae15ae240 2023-09-23T09:42:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:13.992+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15bf0c0 0x564ae15b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:42:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.095+0000 7f90a3fff700 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15ae240 0x564ae15cd0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9094009fd0 tx=0x7f90940975b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb3da0f72931fb4c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.097+0000 7f90927fc700 1 -- 172.21.15.45:0/504630555 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940cbea0 con 0x564ae15ae240 2023-09-23T09:42:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.097+0000 7f90927fc700 1 -- 172.21.15.45:0/504630555 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909409b840 con 0x564ae15ae240 2023-09-23T09:42:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.097+0000 7f90927fc700 1 -- 172.21.15.45:0/504630555 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909409b600 con 0x564ae15ae240 2023-09-23T09:42:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.098+0000 7f90b5d1db80 1 -- 172.21.15.45:0/504630555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15ae240 msgr2=0x564ae15cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.098+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15ae240 0x564ae15cd0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9094009fd0 tx=0x7f90940975b0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 -- 172.21.15.45:0/504630555 shutdown_connections 2023-09-23T09:42:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15ae240 0x564ae15cd0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15b5f40 0x564ae15c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/504630555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15bf0c0 0x564ae15b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 -- 172.21.15.45:0/504630555 >> 172.21.15.45:0/504630555 conn(0x564ae152b200 msgr2=0x564ae15cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 -- 172.21.15.45:0/504630555 shutdown_connections 2023-09-23T09:42:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 -- 172.21.15.45:0/504630555 wait complete. 2023-09-23T09:42:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.099+0000 7f90b5d1db80 1 -- start start 2023-09-23T09:42:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.100+0000 7f90b5d1db80 1 Processor -- start 2023-09-23T09:42:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15ae240 0x564ae161eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15b5f40 0x564ae161f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90b5d1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15bf0c0 0x564ae16194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae1619c00 con 0x564ae15ae240 2023-09-23T09:42:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae1619d70 con 0x564ae15bf0c0 2023-09-23T09:42:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90b5d1db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ae1619f30 con 0x564ae15b5f40 2023-09-23T09:42:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90b5d1db80 10 client.0 _ll_register_callbacks cb 0x7f909c009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15ae240 0x564ae161eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15ae240 0x564ae161eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36492/0 (socket says 172.21.15.45:36492) 2023-09-23T09:42:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90a37fe700 1 -- 172.21.15.45:0/1216767475 learned_addr learned my addr 172.21.15.45:0/1216767475 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.101+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15b5f40 0x564ae161f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.102+0000 7f90a37fe700 1 -- 172.21.15.45:0/1216767475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15b5f40 msgr2=0x564ae161f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.102+0000 7f90a37fe700 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15b5f40 0x564ae161f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.102+0000 7f90a37fe700 1 -- 172.21.15.45:0/1216767475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15bf0c0 msgr2=0x564ae16194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.102+0000 7f90a37fe700 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15bf0c0 0x564ae16194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.102+0000 7f90a37fe700 1 -- 172.21.15.45:0/1216767475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a4070fc0 con 0x564ae15ae240 2023-09-23T09:42:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.102+0000 7f90a37fe700 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15ae240 0x564ae161eef0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9098012040 tx=0x7f90980058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bd02880ef1b9ea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.104+0000 7f9086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.104+0000 7f9086ffd700 1 -- 172.21.15.45:0/1216767475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098004040 con 0x564ae15ae240 2023-09-23T09:42:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.105+0000 7f9086ffd700 1 -- 172.21.15.45:0/1216767475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9098017040 con 0x564ae15ae240 2023-09-23T09:42:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.105+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f909c0138d0 con 0x564ae15ae240 2023-09-23T09:42:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.105+0000 7f9086ffd700 1 -- 172.21.15.45:0/1216767475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90980041a0 con 0x564ae15ae240 2023-09-23T09:42:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.105+0000 7f90b5d1db80 10 client.16881 fetch_fsmap learned FSMap version 49 2023-09-23T09:42:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.105+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564ae135b1f0 con 0x564ae15ae240 2023-09-23T09:42:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.106+0000 7f9086ffd700 1 -- 172.21.15.45:0/1216767475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909c0138d0 con 0x564ae15ae240 2023-09-23T09:42:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.106+0000 7f9086ffd700 1 -- 172.21.15.45:0/1216767475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9098033070 con 0x564ae15ae240 2023-09-23T09:42:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.107+0000 7f90b5d1db80 10 client.16881 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T09:42:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.107+0000 7f90b5d1db80 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.107+0000 7f90b5d1db80 20 client.16881 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.107+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae1626c40 0x564ae1629060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.107+0000 7f90b5d1db80 4 client.16881 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:42:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.107+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ae1502ad0 con 0x564ae1626c40 2023-09-23T09:42:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.107+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae1626c40 0x564ae1629060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.108+0000 7f9086ffd700 10 client.16881 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.108+0000 7f90a2ffd700 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae1626c40 0x564ae1629060 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562ed2235f1247c3 server_cookie=61dfcfa151f3142d in_seq=0 out_seq=0 2023-09-23T09:42:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.141+0000 7f9086ffd700 1 -- 172.21.15.45:0/1216767475 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ae1502ad0 con 0x564ae1626c40 2023-09-23T09:42:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.141+0000 7f9086ffd700 10 client.16881 handle_command_reply: tid=0 2023-09-23T09:42:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.141+0000 7f90b5d1db80 1 client.16881 shutdown 2023-09-23T09:42:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.142+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15ae240 msgr2=0x564ae161eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.142+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15ae240 0x564ae161eef0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9098012040 tx=0x7f90980058e0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 shutdown_connections 2023-09-23T09:42:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564ae1626c40 0x564ae1629060 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ae15bf0c0 0x564ae16194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ae15ae240 0x564ae161eef0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 --2- 172.21.15.45:0/1216767475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ae15b5f40 0x564ae161f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 >> 172.21.15.45:0/1216767475 conn(0x564ae152b200 msgr2=0x564ae15bff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f9086ffd700 0 client.16881 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 shutdown_connections 2023-09-23T09:42:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 1 -- 172.21.15.45:0/1216767475 wait complete. 2023-09-23T09:42:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.143+0000 7f90b5d1db80 20 client.16881 trim_cache size 0 max 16384 2023-09-23T09:42:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:42:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:42:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:42:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:42:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a18a0 msgr2=0x7f908c0a3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a18a0 0x7f908c0a3d60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9098005fd0 tx=0x7f9098005f00 comp rx=0 tx=0).stop 2023-09-23T09:42:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a410f2e0 msgr2=0x7f90a410f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a410f2e0 0x7f90a410f790 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9094006010 tx=0x7f9094003680 comp rx=0 tx=0).stop 2023-09-23T09:42:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 shutdown_connections 2023-09-23T09:42:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f908c0a18a0 0x7f908c0a3d60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90a410f2e0 0x7f90a410f790 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a4100160 0x7f90a41153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 --2- 172.21.15.45:0/3056700019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90a4105050 0x7f90a41158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 >> 172.21.15.45:0/3056700019 conn(0x7f90a4076840 msgr2=0x7f90a40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 shutdown_connections 2023-09-23T09:42:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:14.144+0000 7f90a9b33700 1 -- 172.21.15.45:0/3056700019 wait complete. 2023-09-23T09:42:14.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:13 smithi098 ceph-mon[102386]: pgmap v400: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 974 op/s 2023-09-23T09:42:14.335 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:13 smithi045 ceph-mon[98627]: pgmap v400: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 974 op/s 2023-09-23T09:42:14.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:13 smithi123 ceph-mon[103577]: pgmap v400: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 974 op/s 2023-09-23T09:42:16.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:15 smithi098 ceph-mon[102386]: pgmap v401: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 873 op/s 2023-09-23T09:42:16.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:15 smithi098 ceph-mon[102386]: osdmap e101: 12 total, 12 up, 12 in 2023-09-23T09:42:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:15 smithi123 ceph-mon[103577]: pgmap v401: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 873 op/s 2023-09-23T09:42:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:15 smithi123 ceph-mon[103577]: osdmap e101: 12 total, 12 up, 12 in 2023-09-23T09:42:16.349 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:15 smithi045 ceph-mon[98627]: pgmap v401: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 873 op/s 2023-09-23T09:42:16.349 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:15 smithi045 ceph-mon[98627]: osdmap e101: 12 total, 12 up, 12 in 2023-09-23T09:42:16.887 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 750.0 s, 1120.8 tps, lat 89.217 ms stddev 110.150 2023-09-23T09:42:18.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:17 smithi098 ceph-mon[102386]: pgmap v403: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 737 op/s 2023-09-23T09:42:18.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:17 smithi045 ceph-mon[98627]: pgmap v403: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 737 op/s 2023-09-23T09:42:18.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:17 smithi123 ceph-mon[103577]: pgmap v403: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 737 op/s 2023-09-23T09:42:20.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:19 smithi098 ceph-mon[102386]: pgmap v404: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 764 op/s 2023-09-23T09:42:20.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:19 smithi045 ceph-mon[98627]: pgmap v404: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 764 op/s 2023-09-23T09:42:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:19 smithi123 ceph-mon[103577]: pgmap v404: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 764 op/s 2023-09-23T09:42:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 755.0 s, 1520.9 tps, lat 65.711 ms stddev 75.943 2023-09-23T09:42:22.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:21 smithi098 ceph-mon[102386]: pgmap v405: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-09-23T09:42:22.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:21 smithi045 ceph-mon[98627]: pgmap v405: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-09-23T09:42:22.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:21 smithi123 ceph-mon[103577]: pgmap v405: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-09-23T09:42:24.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:24 smithi098 ceph-mon[102386]: pgmap v406: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-09-23T09:42:24.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:24 smithi045 ceph-mon[98627]: pgmap v406: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-09-23T09:42:24.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:24 smithi123 ceph-mon[103577]: pgmap v406: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-09-23T09:42:26.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:26 smithi098 ceph-mon[102386]: pgmap v407: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-09-23T09:42:26.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:26 smithi045 ceph-mon[98627]: pgmap v407: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-09-23T09:42:26.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:26 smithi123 ceph-mon[103577]: pgmap v407: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-09-23T09:42:26.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 760.0 s, 1633.2 tps, lat 61.502 ms stddev 74.137 2023-09-23T09:42:27.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:27 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:42:27.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:27 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:42:27.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:27 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:42:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 -- 172.21.15.45:0/1509001479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068210 msgr2=0x7f62f4068690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 --2- 172.21.15.45:0/1509001479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068210 0x7f62f4068690 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f62e00099c0 tx=0x7f62e0009cd0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 -- 172.21.15.45:0/1509001479 shutdown_connections 2023-09-23T09:42:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 --2- 172.21.15.45:0/1509001479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068bd0 0x7f62f410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 --2- 172.21.15.45:0/1509001479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f4105da0 0x7f62f4106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 --2- 172.21.15.45:0/1509001479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068210 0x7f62f4068690 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 -- 172.21.15.45:0/1509001479 >> 172.21.15.45:0/1509001479 conn(0x7f62f4076840 msgr2=0x7f62f4076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 -- 172.21.15.45:0/1509001479 shutdown_connections 2023-09-23T09:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 -- 172.21.15.45:0/1509001479 wait complete. 2023-09-23T09:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 Processor -- start 2023-09-23T09:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 -- start start 2023-09-23T09:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068210 0x7f62f4196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068bd0 0x7f62f4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f4105da0 0x7f62f419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.343+0000 7f62faa6a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f419bf90 con 0x7f62f4068bd0 2023-09-23T09:42:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62faa6a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f419c100 con 0x7f62f4068210 2023-09-23T09:42:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62faa6a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f419c2c0 con 0x7f62f4105da0 2023-09-23T09:42:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068bd0 0x7f62f4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068210 0x7f62f4196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f9007700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f4105da0 0x7f62f419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068210 0x7f62f4196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57856/0 (socket says 172.21.15.45:57856) 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 -- 172.21.15.45:0/903914791 learned_addr learned my addr 172.21.15.45:0/903914791 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f3fff700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068bd0 0x7f62f4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36508/0 (socket says 172.21.15.45:36508) 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 -- 172.21.15.45:0/903914791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f4105da0 msgr2=0x7f62f419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f4105da0 0x7f62f419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 -- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068bd0 msgr2=0x7f62f4197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068bd0 0x7f62f4197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 -- 172.21.15.45:0/903914791 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00095f0 con 0x7f62f4068210 2023-09-23T09:42:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f9007700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f4105da0 0x7f62f419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f3fff700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068bd0 0x7f62f4197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f8806700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068210 0x7f62f4196e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f62e800e9a0 tx=0x7f62e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.344+0000 7f62f1ffb700 1 -- 172.21.15.45:0/903914791 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8009790 con 0x7f62f4068210 2023-09-23T09:42:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.345+0000 7f62f1ffb700 1 -- 172.21.15.45:0/903914791 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e8010430 con 0x7f62f4068210 2023-09-23T09:42:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.345+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f419c5a0 con 0x7f62f4068210 2023-09-23T09:42:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.345+0000 7f62f1ffb700 1 -- 172.21.15.45:0/903914791 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8018680 con 0x7f62f4068210 2023-09-23T09:42:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.345+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f41a3de0 con 0x7f62f4068210 2023-09-23T09:42:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.347+0000 7f62f1ffb700 1 -- 172.21.15.45:0/903914791 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f62e80105a0 con 0x7f62f4068210 2023-09-23T09:42:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.348+0000 7f62f1ffb700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62e40a1790 0x7f62e40a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.348+0000 7f62f1ffb700 1 -- 172.21.15.45:0/903914791 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62e8014070 con 0x7f62f4068210 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.348+0000 7f62f3fff700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62e40a1790 0x7f62e40a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.349+0000 7f62f3fff700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62e40a1790 0x7f62e40a3c50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009990 tx=0x7f62e001a040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.349+0000 7f6306c54b80 1 Processor -- start 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f6306c54b80 1 -- start start 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008cf73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d0c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f6306c54b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008c5f400 con 0x564008cf6f50 2023-09-23T09:42:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f6306c54b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008c50bc0 con 0x564008d057a0 2023-09-23T09:42:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f6306c54b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008c4e4e0 con 0x564008cf7910 2023-09-23T09:42:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008cf73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f8806700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008cf73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36516/0 (socket says 172.21.15.45:36516) 2023-09-23T09:42:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f3fff700 1 -- 172.21.15.45:0/2099920694 learned_addr learned my addr 172.21.15.45:0/2099920694 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f9007700 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d0c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f8806700 1 -- 172.21.15.45:0/2099920694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 msgr2=0x564008d0c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f8806700 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d0c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f8806700 1 -- 172.21.15.45:0/2099920694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 msgr2=0x564008cf73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f8806700 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008cf73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.350+0000 7f62f8806700 1 -- 172.21.15.45:0/2099920694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564008cac870 con 0x564008d057a0 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.351+0000 7f62f8806700 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d05b80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cabe0 tx=0x7f62e809a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80f012ee1f5352b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.351+0000 7f62deffd700 1 -- 172.21.15.45:0/2099920694 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e809a400 con 0x564008d057a0 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.351+0000 7f62deffd700 1 -- 172.21.15.45:0/2099920694 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e809e550 con 0x564008d057a0 2023-09-23T09:42:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.351+0000 7f62deffd700 1 -- 172.21.15.45:0/2099920694 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e809a660 con 0x564008d057a0 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 -- 172.21.15.45:0/2099920694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 msgr2=0x564008d05b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d05b80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cabe0 tx=0x7f62e809a030 comp rx=0 tx=0).stop 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 -- 172.21.15.45:0/2099920694 shutdown_connections 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d0c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d05b80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2099920694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008cf73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 -- 172.21.15.45:0/2099920694 >> 172.21.15.45:0/2099920694 conn(0x564008c6b200 msgr2=0x564008cf21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 -- 172.21.15.45:0/2099920694 shutdown_connections 2023-09-23T09:42:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 -- 172.21.15.45:0/2099920694 wait complete. 2023-09-23T09:42:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 -- start start 2023-09-23T09:42:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.352+0000 7f6306c54b80 1 Processor -- start 2023-09-23T09:42:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf6f50 0x564008d097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d0a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f6306c54b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008c49dd0 con 0x564008d057a0 2023-09-23T09:42:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f6306c54b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008be66b0 con 0x564008cf6f50 2023-09-23T09:42:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f6306c54b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008be6350 con 0x564008cf7910 2023-09-23T09:42:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f6306c54b80 10 client.0 _ll_register_callbacks cb 0x564008a437a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f62f9007700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d0a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f62f9007700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d0a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36524/0 (socket says 172.21.15.45:36524) 2023-09-23T09:42:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f62f9007700 1 -- 172.21.15.45:0/86448770 learned_addr learned my addr 172.21.15.45:0/86448770 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f62f8806700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf6f50 0x564008d097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.353+0000 7f62f3fff700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62f9007700 1 -- 172.21.15.45:0/86448770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 msgr2=0x564008d09d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62f9007700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d09d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62f9007700 1 -- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf6f50 msgr2=0x564008d097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62f9007700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf6f50 0x564008d097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62f9007700 1 -- 172.21.15.45:0/86448770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f41a3de0 con 0x564008d057a0 2023-09-23T09:42:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62f3fff700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62f9007700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d0a260 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f62ec004730 tx=0x7f62ec003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872df66c1864d56c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62d3fff700 1 -- 172.21.15.45:0/86448770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec01e400 con 0x564008d057a0 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f62d3fff700 1 -- 172.21.15.45:0/86448770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62ec016400 con 0x564008d057a0 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.354+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564008cea090 con 0x564008d057a0 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f62d3fff700 1 -- 172.21.15.45:0/86448770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62ec0165c0 con 0x564008d057a0 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 10 client.16893 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x564008a74d80 con 0x564008d057a0 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f62d3fff700 1 -- 172.21.15.45:0/86448770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564008cea090 con 0x564008d057a0 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f62d3fff700 1 -- 172.21.15.45:0/86448770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f62ec01c070 con 0x564008d057a0 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 10 client.16893 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 20 client.16893 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d19c80 0x564008d1c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 4 client.16893 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:42:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564008c42db0 con 0x564008d19c80 2023-09-23T09:42:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.355+0000 7f62f8806700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d19c80 0x564008d1c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f62d3fff700 10 client.16893 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f62f8806700 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d19c80 0x564008d1c0a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4b36f49edf5dad server_cookie=de0da9712dab8392 in_seq=0 out_seq=0 2023-09-23T09:42:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f62d3fff700 1 -- 172.21.15.45:0/86448770 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564008c42db0 con 0x564008d19c80 2023-09-23T09:42:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f62d3fff700 10 client.16893 handle_command_reply: tid=0 2023-09-23T09:42:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f6306c54b80 1 client.16893 shutdown 2023-09-23T09:42:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 msgr2=0x564008d0a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f6306c54b80 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d0a260 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f62ec004730 tx=0x7f62ec003f80 comp rx=0 tx=0).stop 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 shutdown_connections 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f6306c54b80 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d19c80 0x564008d1c0a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.356+0000 7f6306c54b80 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d0a260 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.357+0000 7f6306c54b80 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf6f50 0x564008d097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.357+0000 7f62d3fff700 0 client.16893 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.357+0000 7f6306c54b80 1 --2- 172.21.15.45:0/86448770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.357+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 >> 172.21.15.45:0/86448770 conn(0x564008c6b200 msgr2=0x564008cf1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.357+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 shutdown_connections 2023-09-23T09:42:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.357+0000 7f6306c54b80 1 -- 172.21.15.45:0/86448770 wait complete. 2023-09-23T09:42:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.357+0000 7f6306c54b80 20 client.16893 trim_cache size 0 max 16384 2023-09-23T09:42:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 Processor -- start 2023-09-23T09:42:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 -- start start 2023-09-23T09:42:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf6f50 0x564008cf7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf7910 0x564008d07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d10b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008aca8c0 con 0x564008d057a0 2023-09-23T09:42:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008d11090 con 0x564008cf7910 2023-09-23T09:42:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f6306c54b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008d17200 con 0x564008cf6f50 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d10b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d10b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36530/0 (socket says 172.21.15.45:36530) 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf7910 0x564008d07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 -- 172.21.15.45:0/2572693311 learned_addr learned my addr 172.21.15.45:0/2572693311 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f8806700 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf6f50 0x564008cf7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 -- 172.21.15.45:0/2572693311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf6f50 msgr2=0x564008cf7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf6f50 0x564008cf7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 -- 172.21.15.45:0/2572693311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf7910 msgr2=0x564008d07420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf7910 0x564008d07420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f9007700 1 -- 172.21.15.45:0/2572693311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564008a5dc00 con 0x564008d057a0 2023-09-23T09:42:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.374+0000 7f62f3fff700 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf7910 0x564008d07420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.375+0000 7f62f9007700 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d10b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f62ec01ea50 tx=0x7f62ec00e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc6b27b1cc58eeb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.375+0000 7f62df7fe700 1 -- 172.21.15.45:0/2572693311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec01e400 con 0x564008d057a0 2023-09-23T09:42:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.375+0000 7f62df7fe700 1 -- 172.21.15.45:0/2572693311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ec00f040 con 0x564008d057a0 2023-09-23T09:42:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.375+0000 7f62df7fe700 1 -- 172.21.15.45:0/2572693311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec007d50 con 0x564008d057a0 2023-09-23T09:42:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.376+0000 7f6306c54b80 1 -- 172.21.15.45:0/2572693311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 msgr2=0x564008d10b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.376+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d10b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f62ec01ea50 tx=0x7f62ec00e880 comp rx=0 tx=0).stop 2023-09-23T09:42:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.377+0000 7f6306c54b80 1 -- 172.21.15.45:0/2572693311 shutdown_connections 2023-09-23T09:42:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.377+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008d057a0 0x564008d10b50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.377+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf6f50 0x564008cf7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.377+0000 7f6306c54b80 1 --2- 172.21.15.45:0/2572693311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008cf7910 0x564008d07420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.377+0000 7f6306c54b80 1 -- 172.21.15.45:0/2572693311 >> 172.21.15.45:0/2572693311 conn(0x564008c6b200 msgr2=0x564008d152a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.377+0000 7f6306c54b80 1 -- 172.21.15.45:0/2572693311 shutdown_connections 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.377+0000 7f6306c54b80 1 -- 172.21.15.45:0/2572693311 wait complete. 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 -- start start 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 Processor -- start 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008d61c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d62150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008d62840 con 0x564008cf6f50 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008d6cfe0 con 0x564008d057a0 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564008d6d120 con 0x564008cf7910 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f6306c54b80 10 client.0 _ll_register_callbacks cb 0x564008a437a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f62f8806700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008d61c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f62f9007700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f62f9007700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d6caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57886/0 (socket says 172.21.15.45:57886) 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f62f8806700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008d61c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36532/0 (socket says 172.21.15.45:36532) 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d62150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.378+0000 7f62f8806700 1 -- 172.21.15.45:0/1933478006 learned_addr learned my addr 172.21.15.45:0/1933478006 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62f8806700 1 -- 172.21.15.45:0/1933478006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 msgr2=0x564008d62150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62f8806700 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d62150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62f8806700 1 -- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 msgr2=0x564008d6caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62f8806700 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d6caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62f8806700 1 -- 172.21.15.45:0/1933478006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f410e570 con 0x564008cf6f50 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62f9007700 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62f8806700 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008d61c10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cac40 tx=0x7f62e8096110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0044c50744fbdb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62d3fff700 1 -- 172.21.15.45:0/1933478006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80a2810 con 0x564008cf6f50 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62d3fff700 1 -- 172.21.15.45:0/1933478006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62e80b2040 con 0x564008cf6f50 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62e809e550 con 0x564008cf6f50 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62d3fff700 1 -- 172.21.15.45:0/1933478006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62e809e170 con 0x564008cf6f50 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 10 client.16899 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x564008a74d80 con 0x564008cf6f50 2023-09-23T09:42:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62d3fff700 1 -- 172.21.15.45:0/1933478006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e809e550 con 0x564008cf6f50 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f62d3fff700 1 -- 172.21.15.45:0/1933478006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f62e80cae60 con 0x564008cf6f50 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 10 client.16899 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 20 client.16899 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d6e540 0x564008d70960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 4 client.16899 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.379+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564008c42ad0 con 0x564008d6e540 2023-09-23T09:42:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.380+0000 7f62f3fff700 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d6e540 0x564008d70960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.380+0000 7f62d3fff700 10 client.16899 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.380+0000 7f62f3fff700 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d6e540 0x564008d70960 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192d852c124e5e32 server_cookie=62eea5ecb7e5a996 in_seq=0 out_seq=0 2023-09-23T09:42:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f62d3fff700 1 -- 172.21.15.45:0/1933478006 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564008c42ad0 con 0x564008d6e540 2023-09-23T09:42:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f62d3fff700 10 client.16899 handle_command_reply: tid=0 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 client.16899 shutdown 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 msgr2=0x564008d61c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008d61c10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cac40 tx=0x7f62e8096110 comp rx=0 tx=0).stop 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 shutdown_connections 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x564008d6e540 0x564008d70960 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564008d057a0 0x564008d6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f62d3fff700 0 client.16899 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564008cf6f50 0x564008d61c10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 --2- 172.21.15.45:0/1933478006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564008cf7910 0x564008d62150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 >> 172.21.15.45:0/1933478006 conn(0x564008c6b200 msgr2=0x564008d04960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 shutdown_connections 2023-09-23T09:42:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.401+0000 7f6306c54b80 1 -- 172.21.15.45:0/1933478006 wait complete. 2023-09-23T09:42:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f6306c54b80 20 client.16899 trim_cache size 0 max 16384 2023-09-23T09:42:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:42:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:42:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:42:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:42:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62e40a1790 msgr2=0x7f62e40a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62e40a1790 0x7f62e40a3c50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009990 tx=0x7f62e001a040 comp rx=0 tx=0).stop 2023-09-23T09:42:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068210 msgr2=0x7f62f4196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068210 0x7f62f4196e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f62e800e9a0 tx=0x7f62e800ecb0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 shutdown_connections 2023-09-23T09:42:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f4105da0 0x7f62f419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62f4068210 0x7f62f4196e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62e40a1790 0x7f62e40a3c50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 --2- 172.21.15.45:0/903914791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62f4068bd0 0x7f62f4197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 >> 172.21.15.45:0/903914791 conn(0x7f62f4076840 msgr2=0x7f62f40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 shutdown_connections 2023-09-23T09:42:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:27.402+0000 7f62faa6a700 1 -- 172.21.15.45:0/903914791 wait complete. 2023-09-23T09:42:28.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:28 smithi098 ceph-mon[102386]: pgmap v408: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:42:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:28 smithi045 ceph-mon[98627]: pgmap v408: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:42:28.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:28 smithi123 ceph-mon[103577]: pgmap v408: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:42:30.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:30 smithi098 ceph-mon[102386]: pgmap v409: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-09-23T09:42:30.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:30 smithi045 ceph-mon[98627]: pgmap v409: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-09-23T09:42:30.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:30 smithi123 ceph-mon[103577]: pgmap v409: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-09-23T09:42:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 765.0 s, 1600.8 tps, lat 62.272 ms stddev 73.301 2023-09-23T09:42:32.301 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:32 smithi098 ceph-mon[102386]: pgmap v410: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-23T09:42:32.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:32 smithi045 ceph-mon[98627]: pgmap v410: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-23T09:42:32.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:32 smithi123 ceph-mon[103577]: pgmap v410: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-23T09:42:34.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:34 smithi098 ceph-mon[102386]: pgmap v411: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.4 MiB/s wr, 464 op/s 2023-09-23T09:42:34.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:34 smithi045 ceph-mon[98627]: pgmap v411: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.4 MiB/s wr, 464 op/s 2023-09-23T09:42:34.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:34 smithi123 ceph-mon[103577]: pgmap v411: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.4 MiB/s wr, 464 op/s 2023-09-23T09:42:36.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:36 smithi098 ceph-mon[102386]: pgmap v412: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 616 op/s 2023-09-23T09:42:36.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:36 smithi045 ceph-mon[98627]: pgmap v412: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 616 op/s 2023-09-23T09:42:36.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:36 smithi123 ceph-mon[103577]: pgmap v412: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 616 op/s 2023-09-23T09:42:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 770.0 s, 1415.4 tps, lat 70.256 ms stddev 83.028 2023-09-23T09:42:38.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:38 smithi098 ceph-mon[102386]: pgmap v413: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 701 op/s 2023-09-23T09:42:38.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:38 smithi045 ceph-mon[98627]: pgmap v413: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 701 op/s 2023-09-23T09:42:38.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:38 smithi123 ceph-mon[103577]: pgmap v413: 97 pgs: 97 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 701 op/s 2023-09-23T09:42:38.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:42:39.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.113+0000 7f5a8d89d700 1 -- 172.21.15.45:0/4215580566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88103210 msgr2=0x7f5a881035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.113+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/4215580566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88103210 0x7f5a881035f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b0a0 tx=0x7f5a8000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.115+0000 7f5a8d89d700 1 -- 172.21.15.45:0/4215580566 shutdown_connections 2023-09-23T09:42:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.115+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/4215580566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88108330 0x7f5a8810feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.115+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/4215580566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88103210 0x7f5a881035f0 secure :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b0a0 tx=0x7f5a8000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.115+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/4215580566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a88103bc0 0x7f5a88107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.115+0000 7f5a8d89d700 1 -- 172.21.15.45:0/4215580566 >> 172.21.15.45:0/4215580566 conn(0x7f5a880fea70 msgr2=0x7f5a88100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.116+0000 7f5a8d89d700 1 -- 172.21.15.45:0/4215580566 shutdown_connections 2023-09-23T09:42:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.116+0000 7f5a8d89d700 1 -- 172.21.15.45:0/4215580566 wait complete. 2023-09-23T09:42:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 Processor -- start 2023-09-23T09:42:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 -- start start 2023-09-23T09:42:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88108330 0x7f5a88072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88072ac0 0x7f5a8806b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a8806bad0 0x7f5a8806bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a880702b0 con 0x7f5a88072ac0 2023-09-23T09:42:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88070420 con 0x7f5a8806bad0 2023-09-23T09:42:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.117+0000 7f5a8d89d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a880705e0 con 0x7f5a88108330 2023-09-23T09:42:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.118+0000 7f5a867fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88072ac0 0x7f5a8806b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.118+0000 7f5a867fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88072ac0 0x7f5a8806b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36548/0 (socket says 172.21.15.45:36548) 2023-09-23T09:42:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.118+0000 7f5a867fc700 1 -- 172.21.15.45:0/1025360528 learned_addr learned my addr 172.21.15.45:0/1025360528 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:39.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.119+0000 7f5a867fc700 1 -- 172.21.15.45:0/1025360528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88108330 msgr2=0x7f5a88072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.119+0000 7f5a867fc700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88108330 0x7f5a88072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.119+0000 7f5a867fc700 1 -- 172.21.15.45:0/1025360528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a8806bad0 msgr2=0x7f5a8806bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.119+0000 7f5a867fc700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a8806bad0 0x7f5a8806bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.119+0000 7f5a867fc700 1 -- 172.21.15.45:0/1025360528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80009d00 con 0x7f5a88072ac0 2023-09-23T09:42:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.119+0000 7f5a867fc700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88072ac0 0x7f5a8806b590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800e970 tx=0x7f5a7800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.120+0000 7f5a8c89b700 1 -- 172.21.15.45:0/1025360528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78009790 con 0x7f5a88072ac0 2023-09-23T09:42:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.120+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a880708c0 con 0x7f5a88072ac0 2023-09-23T09:42:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.120+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a88070e20 con 0x7f5a88072ac0 2023-09-23T09:42:39.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.125+0000 7f5a99a87b80 1 Processor -- start 2023-09-23T09:42:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a99a87b80 1 -- start start 2023-09-23T09:42:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b82037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8216ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 0x5630b82197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b816c460 con 0x5630b8203400 2023-09-23T09:42:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b815dc20 con 0x5630b8203d20 2023-09-23T09:42:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b815b540 con 0x5630b82173e0 2023-09-23T09:42:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a8c89b700 1 -- 172.21.15.45:0/1025360528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a78004cf0 con 0x7f5a88072ac0 2023-09-23T09:42:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a8c89b700 1 -- 172.21.15.45:0/1025360528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a780055e0 con 0x7f5a88072ac0 2023-09-23T09:42:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a8c89b700 1 -- 172.21.15.45:0/1025360528 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f5a78021440 con 0x7f5a88072ac0 2023-09-23T09:42:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b82037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.131+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b82037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36550/0 (socket says 172.21.15.45:36550) 2023-09-23T09:42:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a877fe700 1 -- 172.21.15.45:0/437112639 learned_addr learned my addr 172.21.15.45:0/437112639 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a877fe700 1 -- 172.21.15.45:0/437112639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 msgr2=0x5630b82197d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a877fe700 1 --2- 172.21.15.45:0/437112639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 0x5630b82197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a877fe700 1 -- 172.21.15.45:0/437112639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 msgr2=0x5630b8216ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:42:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a877fe700 1 --2- 172.21.15.45:0/437112639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8216ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a877fe700 1 -- 172.21.15.45:0/437112639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630b81b98d0 con 0x5630b8203400 2023-09-23T09:42:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a8c89b700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5a700a18d0 0x7f5a700a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.132+0000 7f5a877fe700 1 --2- 172.21.15.45:0/437112639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b82037e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009c50 tx=0x7f5a7c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289d5f90c5653e3a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.133+0000 7f5a8c89b700 1 -- 172.21.15.45:0/1025360528 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5a78014070 con 0x7f5a88072ac0 2023-09-23T09:42:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.133+0000 7f5a6dffb700 1 -- 172.21.15.45:0/437112639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00eb60 con 0x5630b8203400 2023-09-23T09:42:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.133+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5a700a18d0 0x7f5a700a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.133+0000 7f5a6dffb700 1 -- 172.21.15.45:0/437112639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a7c004cf0 con 0x5630b8203400 2023-09-23T09:42:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.133+0000 7f5a6dffb700 1 -- 172.21.15.45:0/437112639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c005690 con 0x5630b8203400 2023-09-23T09:42:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.134+0000 7f5a99a87b80 1 -- 172.21.15.45:0/437112639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 msgr2=0x5630b82037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.134+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/437112639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b82037e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009c50 tx=0x7f5a7c009f60 comp rx=0 tx=0).stop 2023-09-23T09:42:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.142+0000 7f5a99a87b80 1 -- 172.21.15.45:0/437112639 shutdown_connections 2023-09-23T09:42:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.142+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/437112639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 0x5630b82197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.142+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/437112639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b82037e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.142+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/437112639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8216ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.142+0000 7f5a99a87b80 1 -- 172.21.15.45:0/437112639 >> 172.21.15.45:0/437112639 conn(0x5630b8178260 msgr2=0x5630b81ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.142+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5a700a18d0 0x7f5a700a3d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b070 tx=0x7f5a80006000 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:39.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.148+0000 7f5a99a87b80 1 -- 172.21.15.45:0/437112639 shutdown_connections 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.148+0000 7f5a99a87b80 1 -- 172.21.15.45:0/437112639 wait complete. 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 -- start start 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 Processor -- start 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203400 0x5630b8214220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203d20 0x5630b8214760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b82173e0 0x5630b8214ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b8156e30 con 0x5630b82173e0 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b80f3710 con 0x5630b8203400 2023-09-23T09:42:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.149+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b80f33b0 con 0x5630b8203d20 2023-09-23T09:42:39.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a99a87b80 10 client.0 _ll_register_callbacks cb 0x5630b7f50800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b82173e0 0x5630b8214ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b82173e0 0x5630b8214ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36562/0 (socket says 172.21.15.45:36562) 2023-09-23T09:42:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 -- 172.21.15.45:0/4114375961 learned_addr learned my addr 172.21.15.45:0/4114375961 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 -- 172.21.15.45:0/4114375961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203d20 msgr2=0x5630b8214760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:42:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203d20 0x5630b8214760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 -- 172.21.15.45:0/4114375961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203400 msgr2=0x5630b8214220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203400 0x5630b8214220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.150+0000 7f5a877fe700 1 -- 172.21.15.45:0/4114375961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c009900 con 0x5630b82173e0 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a877fe700 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b82173e0 0x5630b8214ca0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009460 tx=0x7f5a7c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea22a5a768f4dfb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a66ffd700 1 -- 172.21.15.45:0/4114375961 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00eb70 con 0x5630b82173e0 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a66ffd700 1 -- 172.21.15.45:0/4114375961 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5a7c014070 con 0x5630b82173e0 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630b81f7250 con 0x5630b82173e0 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a66ffd700 1 -- 172.21.15.45:0/4114375961 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a7c00ecd0 con 0x5630b82173e0 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a99a87b80 10 client.16911 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.151+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5630b7f81de0 con 0x5630b82173e0 2023-09-23T09:42:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.152+0000 7f5a66ffd700 1 -- 172.21.15.45:0/4114375961 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5630b81f7250 con 0x5630b82173e0 2023-09-23T09:42:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.153+0000 7f5a66ffd700 1 -- 172.21.15.45:0/4114375961 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a7c003710 con 0x5630b82173e0 2023-09-23T09:42:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.153+0000 7f5a99a87b80 10 client.16911 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.153+0000 7f5a99a87b80 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.153+0000 7f5a99a87b80 20 client.16911 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.153+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8226d70 0x5630b8229190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.153+0000 7f5a99a87b80 4 client.16911 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:42:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.153+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630b814fe10 con 0x5630b8226d70 2023-09-23T09:42:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.155+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8226d70 0x5630b8229190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.155+0000 7f5a66ffd700 10 client.16911 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:39.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.155+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8226d70 0x5630b8229190 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15e4155e0b15c85 server_cookie=fc612eccaf0e4346 in_seq=0 out_seq=0 2023-09-23T09:42:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.156+0000 7f5a66ffd700 1 -- 172.21.15.45:0/4114375961 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5630b814fe10 con 0x5630b8226d70 2023-09-23T09:42:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.156+0000 7f5a66ffd700 10 client.16911 handle_command_reply: tid=0 2023-09-23T09:42:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.156+0000 7f5a99a87b80 1 client.16911 shutdown 2023-09-23T09:42:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b82173e0 msgr2=0x5630b8214ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b82173e0 0x5630b8214ca0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009460 tx=0x7f5a7c009490 comp rx=0 tx=0).stop 2023-09-23T09:42:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 shutdown_connections 2023-09-23T09:42:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8226d70 0x5630b8229190 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b82173e0 0x5630b8214ca0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203400 0x5630b8214220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/4114375961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203d20 0x5630b8214760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 >> 172.21.15.45:0/4114375961 conn(0x5630b8178260 msgr2=0x5630b81ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a66ffd700 0 client.16911 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 shutdown_connections 2023-09-23T09:42:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.157+0000 7f5a99a87b80 1 -- 172.21.15.45:0/4114375961 wait complete. 2023-09-23T09:42:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.158+0000 7f5a99a87b80 20 client.16911 trim_cache size 0 max 16384 2023-09-23T09:42:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.190+0000 7f5a99a87b80 1 Processor -- start 2023-09-23T09:42:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.190+0000 7f5a99a87b80 1 -- start start 2023-09-23T09:42:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b821df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8214ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 0x5630b8211490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b7fd7920 con 0x5630b8203400 2023-09-23T09:42:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b82044d0 con 0x5630b8203d20 2023-09-23T09:42:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b820f090 con 0x5630b82173e0 2023-09-23T09:42:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8214ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8214ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57918/0 (socket says 172.21.15.45:57918) 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 -- 172.21.15.45:0/711345204 learned_addr learned my addr 172.21.15.45:0/711345204 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b821df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a877fe700 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 0x5630b8211490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 -- 172.21.15.45:0/711345204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 msgr2=0x5630b8211490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 0x5630b8211490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 -- 172.21.15.45:0/711345204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 msgr2=0x5630b821df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b821df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a867fc700 1 -- 172.21.15.45:0/711345204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630b822dda0 con 0x5630b8203d20 2023-09-23T09:42:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.191+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b821df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:42:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.192+0000 7f5a867fc700 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8214ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a78098060 tx=0x7f5a78005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=132f07d49c9a5641 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.192+0000 7f5a6e7fc700 1 -- 172.21.15.45:0/711345204 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7809b130 con 0x5630b8203d20 2023-09-23T09:42:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.192+0000 7f5a6e7fc700 1 -- 172.21.15.45:0/711345204 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a7809af50 con 0x5630b8203d20 2023-09-23T09:42:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.192+0000 7f5a6e7fc700 1 -- 172.21.15.45:0/711345204 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800cbd0 con 0x5630b8203d20 2023-09-23T09:42:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.193+0000 7f5a99a87b80 1 -- 172.21.15.45:0/711345204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 msgr2=0x5630b8214ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.193+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8214ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a78098060 tx=0x7f5a78005800 comp rx=0 tx=0).stop 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.193+0000 7f5a99a87b80 1 -- 172.21.15.45:0/711345204 shutdown_connections 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.193+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b82173e0 0x5630b8211490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.193+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203400 0x5630b821df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.193+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/711345204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b8203d20 0x5630b8214ad0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.193+0000 7f5a99a87b80 1 -- 172.21.15.45:0/711345204 >> 172.21.15.45:0/711345204 conn(0x5630b8178260 msgr2=0x5630b820f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.194+0000 7f5a99a87b80 1 -- 172.21.15.45:0/711345204 shutdown_connections 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.194+0000 7f5a99a87b80 1 -- 172.21.15.45:0/711345204 wait complete. 2023-09-23T09:42:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.194+0000 7f5a99a87b80 1 -- start start 2023-09-23T09:42:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 1 Processor -- start 2023-09-23T09:42:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203400 0x5630b821b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203d20 0x5630b821be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b82173e0 0x5630b8279b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b821c730 con 0x5630b8203d20 2023-09-23T09:42:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b827a0c0 con 0x5630b82173e0 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b8206630 con 0x5630b8203400 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a99a87b80 10 client.0 _ll_register_callbacks cb 0x7f5a7c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a867fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203d20 0x5630b821be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a86ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203400 0x5630b821b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a867fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203d20 0x5630b821be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36572/0 (socket says 172.21.15.45:36572) 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a867fc700 1 -- 172.21.15.45:0/558053135 learned_addr learned my addr 172.21.15.45:0/558053135 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.195+0000 7f5a877fe700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b82173e0 0x5630b8279b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a86ffd700 1 -- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b82173e0 msgr2=0x5630b8279b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b82173e0 0x5630b8279b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a86ffd700 1 -- 172.21.15.45:0/558053135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203d20 msgr2=0x5630b821be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203d20 0x5630b821be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a86ffd700 1 -- 172.21.15.45:0/558053135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a88070e20 con 0x5630b8203400 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a877fe700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b82173e0 0x5630b8279b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a867fc700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203d20 0x5630b821be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a86ffd700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203400 0x5630b821b930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a80008000 tx=0x7f5a800099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d36a368c76c1219 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a66ffd700 1 -- 172.21.15.45:0/558053135 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000e040 con 0x5630b8203400 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a66ffd700 1 -- 172.21.15.45:0/558053135 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5a80022040 con 0x5630b8203400 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a7c004cf0 con 0x5630b8203400 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a66ffd700 1 -- 172.21.15.45:0/558053135 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a8000bea0 con 0x5630b8203400 2023-09-23T09:42:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a99a87b80 10 client.26575 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.196+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5630b7f6ac60 con 0x5630b8203400 2023-09-23T09:42:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a66ffd700 1 -- 172.21.15.45:0/558053135 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a8003f080 con 0x5630b8203400 2023-09-23T09:42:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a66ffd700 1 -- 172.21.15.45:0/558053135 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a80003c70 con 0x5630b8203400 2023-09-23T09:42:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a99a87b80 10 client.26575 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a99a87b80 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a99a87b80 20 client.26575 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8207e00 0x5630b820a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a99a87b80 4 client.26575 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.197+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630b814fb30 con 0x5630b8207e00 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.198+0000 7f5a867fc700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8207e00 0x5630b820a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a867fc700 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8207e00 0x5630b820a220 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7f2f3631efa9dc server_cookie=2102a94c67677920 in_seq=0 out_seq=0 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a66ffd700 10 client.26575 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a66ffd700 1 -- 172.21.15.45:0/558053135 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5630b814fb30 con 0x5630b8207e00 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a66ffd700 10 client.26575 handle_command_reply: tid=0 2023-09-23T09:42:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 client.26575 shutdown 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203400 msgr2=0x5630b821b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203400 0x5630b821b930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a80008000 tx=0x7f5a800099e0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 shutdown_connections 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5630b8207e00 0x5630b820a220 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630b82173e0 0x5630b8279b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b8203400 0x5630b821b930 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 --2- 172.21.15.45:0/558053135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b8203d20 0x5630b821be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a66ffd700 0 client.26575 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 >> 172.21.15.45:0/558053135 conn(0x5630b8178260 msgr2=0x5630b82129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 shutdown_connections 2023-09-23T09:42:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 1 -- 172.21.15.45:0/558053135 wait complete. 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.199+0000 7f5a99a87b80 20 client.26575 trim_cache size 0 max 16384 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:42:39.205 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:42:39.206 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:42:39.206 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:42:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:42:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5a700a18d0 msgr2=0x7f5a700a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5a700a18d0 0x7f5a700a3d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b070 tx=0x7f5a80006000 comp rx=0 tx=0).stop 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88072ac0 msgr2=0x7f5a8806b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88072ac0 0x7f5a8806b590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800e970 tx=0x7f5a7800ed30 comp rx=0 tx=0).stop 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 shutdown_connections 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a8806bad0 0x7f5a8806bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88108330 0x7f5a88072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5a700a18d0 0x7f5a700a3d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 --2- 172.21.15.45:0/1025360528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a88072ac0 0x7f5a8806b590 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 >> 172.21.15.45:0/1025360528 conn(0x7f5a880fea70 msgr2=0x7f5a88106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 shutdown_connections 2023-09-23T09:42:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:42:39.200+0000 7f5a8d89d700 1 -- 172.21.15.45:0/1025360528 wait complete. 2023-09-23T09:42:39.218 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:42:40.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:40 smithi098 ceph-mon[102386]: pgmap v414: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 601 op/s 2023-09-23T09:42:40.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:40 smithi045 ceph-mon[98627]: pgmap v414: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 601 op/s 2023-09-23T09:42:40.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:40 smithi123 ceph-mon[103577]: pgmap v414: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 601 op/s 2023-09-23T09:42:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 -- 172.21.15.45:0/693017610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e40ffaf0 msgr2=0x7f34e40fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 --2- 172.21.15.45:0/693017610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e40ffaf0 0x7f34e40fff30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00b0a0 tx=0x7f34dc00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 -- 172.21.15.45:0/693017610 shutdown_connections 2023-09-23T09:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 --2- 172.21.15.45:0/693017610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e40ffaf0 0x7f34e40fff30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 --2- 172.21.15.45:0/693017610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e4100e70 0x7f34e4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 --2- 172.21.15.45:0/693017610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4101790 0x7f34e4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 -- 172.21.15.45:0/693017610 >> 172.21.15.45:0/693017610 conn(0x7f34e4074b90 msgr2=0x7f34e4074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 -- 172.21.15.45:0/693017610 shutdown_connections 2023-09-23T09:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.569+0000 7f34eaeea700 1 -- 172.21.15.45:0/693017610 wait complete. 2023-09-23T09:42:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.570+0000 7f34eaeea700 1 Processor -- start 2023-09-23T09:42:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.570+0000 7f34eaeea700 1 -- start start 2023-09-23T09:42:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34eaeea700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e40ffaf0 0x7f34e4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34eaeea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4100e70 0x7f34e4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34eaeea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e4101790 0x7f34e419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34eaeea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e419e330 con 0x7f34e4100e70 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34eaeea700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e419e4a0 con 0x7f34e40ffaf0 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34eaeea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e419e660 con 0x7f34e4101790 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4100e70 0x7f34e4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4100e70 0x7f34e4199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36584/0 (socket says 172.21.15.45:36584) 2023-09-23T09:42:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 -- 172.21.15.45:0/775414852 learned_addr learned my addr 172.21.15.45:0/775414852 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e8c86700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e40ffaf0 0x7f34e4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 -- 172.21.15.45:0/775414852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e4101790 msgr2=0x7f34e419dcf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:42:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e4101790 0x7f34e419dcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 -- 172.21.15.45:0/775414852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e40ffaf0 msgr2=0x7f34e4199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e40ffaf0 0x7f34e4199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.571+0000 7f34e3fff700 1 -- 172.21.15.45:0/775414852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d8009520 con 0x7f34e4100e70 2023-09-23T09:42:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.572+0000 7f34e3fff700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4100e70 0x7f34e4199680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b550 tx=0x7f34d000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.573+0000 7f34e1ffb700 1 -- 172.21.15.45:0/775414852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0011830 con 0x7f34e4100e70 2023-09-23T09:42:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.573+0000 7f34e1ffb700 1 -- 172.21.15.45:0/775414852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d0011e50 con 0x7f34e4100e70 2023-09-23T09:42:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.573+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc009d00 con 0x7f34e4100e70 2023-09-23T09:42:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.573+0000 7f34e1ffb700 1 -- 172.21.15.45:0/775414852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000f2e0 con 0x7f34e4100e70 2023-09-23T09:42:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.574+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e41a5fa0 con 0x7f34e4100e70 2023-09-23T09:42:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.577+0000 7f34e1ffb700 1 -- 172.21.15.45:0/775414852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f34d0010560 con 0x7f34e4100e70 2023-09-23T09:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.578+0000 7f34e1ffb700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f34d40a1810 0x7f34d40a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.578+0000 7f34e8c86700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f34d40a1810 0x7f34d40a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.578+0000 7f34e1ffb700 1 -- 172.21.15.45:0/775414852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f34d00ccdb0 con 0x7f34e4100e70 2023-09-23T09:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.578+0000 7f34e8c86700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f34d40a1810 0x7f34d40a3cd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009d80 tx=0x7f34d8009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.578+0000 7f34f70d4b80 1 Processor -- start 2023-09-23T09:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34f70d4b80 1 -- start start 2023-09-23T09:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa7363320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa7370b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa72cc400 con 0x557aa736e710 2023-09-23T09:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa72bdbc0 con 0x557aa7362f40 2023-09-23T09:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa72bb4e0 con 0x557aa7363860 2023-09-23T09:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e8c86700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa7363320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e8c86700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa7363320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57940/0 (socket says 172.21.15.45:57940) 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e9487700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa7370b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e8c86700 1 -- 172.21.15.45:0/1472075009 learned_addr learned my addr 172.21.15.45:0/1472075009 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e9487700 1 -- 172.21.15.45:0/1472075009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 msgr2=0x557aa736e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e9487700 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e9487700 1 -- 172.21.15.45:0/1472075009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 msgr2=0x557aa7363320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e9487700 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa7363320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e9487700 1 -- 172.21.15.45:0/1472075009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa7319870 con 0x557aa736e710 2023-09-23T09:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.579+0000 7f34e8c86700 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa7363320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.580+0000 7f34e9487700 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa7370b00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00b070 tx=0x7f34dc009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7da78e1512d51e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.580+0000 7f34ceffd700 1 -- 172.21.15.45:0/1472075009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00e040 con 0x557aa736e710 2023-09-23T09:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.580+0000 7f34ceffd700 1 -- 172.21.15.45:0/1472075009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34dc00bcb0 con 0x557aa736e710 2023-09-23T09:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.580+0000 7f34ceffd700 1 -- 172.21.15.45:0/1472075009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc005040 con 0x557aa736e710 2023-09-23T09:42:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.581+0000 7f34f70d4b80 1 -- 172.21.15.45:0/1472075009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 msgr2=0x557aa7370b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.581+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa7370b00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00b070 tx=0x7f34dc009740 comp rx=0 tx=0).stop 2023-09-23T09:42:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.582+0000 7f34f70d4b80 1 -- 172.21.15.45:0/1472075009 shutdown_connections 2023-09-23T09:42:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.582+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa7370b00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.582+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa7363320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.582+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/1472075009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.582+0000 7f34f70d4b80 1 -- 172.21.15.45:0/1472075009 >> 172.21.15.45:0/1472075009 conn(0x557aa72d8200 msgr2=0x557aa735f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.582+0000 7f34f70d4b80 1 -- 172.21.15.45:0/1472075009 shutdown_connections 2023-09-23T09:42:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.582+0000 7f34f70d4b80 1 -- 172.21.15.45:0/1472075009 wait complete. 2023-09-23T09:42:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 -- start start 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 Processor -- start 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa736e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa736ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa72b6dd0 con 0x557aa736e710 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa72536b0 con 0x557aa7362f40 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa7253350 con 0x557aa7363860 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34f70d4b80 10 client.0 _ll_register_callbacks cb 0x557aa70b07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34e9487700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa736ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34e9487700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa736ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36600/0 (socket says 172.21.15.45:36600) 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34e9487700 1 -- 172.21.15.45:0/4159825091 learned_addr learned my addr 172.21.15.45:0/4159825091 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34e3fff700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.583+0000 7f34e8c86700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa736e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34e9487700 1 -- 172.21.15.45:0/4159825091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 msgr2=0x557aa736b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34e9487700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34e9487700 1 -- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 msgr2=0x557aa736e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34e9487700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa736e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34e9487700 1 -- 172.21.15.45:0/4159825091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d8011cc0 con 0x557aa736e710 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34e8c86700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa736e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34e9487700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa736ba10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0096e0 tx=0x7f34dc004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9ddbfe9737e467 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34c3fff700 1 -- 172.21.15.45:0/4159825091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00e040 con 0x557aa736e710 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34c3fff700 1 -- 172.21.15.45:0/4159825091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f34dc00be20 con 0x557aa736e710 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557aa7357090 con 0x557aa736e710 2023-09-23T09:42:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34c3fff700 1 -- 172.21.15.45:0/4159825091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34dc0074a0 con 0x557aa736e710 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34f70d4b80 10 client.16926 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f34e410a2e0 con 0x557aa736e710 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34c3fff700 1 -- 172.21.15.45:0/4159825091 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557aa7357090 con 0x557aa736e710 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34c3fff700 1 -- 172.21.15.45:0/4159825091 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f34dc021020 con 0x557aa736e710 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34f70d4b80 10 client.16926 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.584+0000 7f34f70d4b80 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.585+0000 7f34f70d4b80 20 client.16926 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.585+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa737dc60 0x557aa7380080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.585+0000 7f34f70d4b80 4 client.16926 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:42:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.585+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aa72afdb0 con 0x557aa737dc60 2023-09-23T09:42:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.585+0000 7f34e8c86700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa737dc60 0x557aa7380080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.586+0000 7f34c3fff700 10 client.16926 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.586+0000 7f34e8c86700 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa737dc60 0x557aa7380080 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2adc3a0715926e6 server_cookie=3fd9ba06df221fe in_seq=0 out_seq=0 2023-09-23T09:42:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.589+0000 7f34c3fff700 1 -- 172.21.15.45:0/4159825091 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557aa72afdb0 con 0x557aa737dc60 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.589+0000 7f34c3fff700 10 client.16926 handle_command_reply: tid=0 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.589+0000 7f34f70d4b80 1 client.16926 shutdown 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 msgr2=0x557aa736ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa736ba10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0096e0 tx=0x7f34dc004d70 comp rx=0 tx=0).stop 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 shutdown_connections 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa737dc60 0x557aa7380080 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa736e710 0x557aa736ba10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7362f40 0x557aa736e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34c3fff700 0 client.16926 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4159825091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa7363860 0x557aa736b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 >> 172.21.15.45:0/4159825091 conn(0x557aa72d8200 msgr2=0x557aa735ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 shutdown_connections 2023-09-23T09:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.590+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4159825091 wait complete. 2023-09-23T09:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.591+0000 7f34f70d4b80 20 client.16926 trim_cache size 0 max 16384 2023-09-23T09:42:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 Processor -- start 2023-09-23T09:42:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 -- start start 2023-09-23T09:42:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa7375200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 0x557aa7379950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa7366690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa71378c0 con 0x557aa7362f40 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa7360100 con 0x557aa7363860 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa7375740 con 0x557aa736e710 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34e8c86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa7375200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34e3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 0x557aa7379950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34e8c86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa7375200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36604/0 (socket says 172.21.15.45:36604) 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34e9487700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa7366690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.613+0000 7f34e8c86700 1 -- 172.21.15.45:0/4017514899 learned_addr learned my addr 172.21.15.45:0/4017514899 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34e8c86700 1 -- 172.21.15.45:0/4017514899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 msgr2=0x557aa7366690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34e8c86700 1 --2- 172.21.15.45:0/4017514899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa7366690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34e8c86700 1 -- 172.21.15.45:0/4017514899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 msgr2=0x557aa7379950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34e8c86700 1 --2- 172.21.15.45:0/4017514899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 0x557aa7379950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34e8c86700 1 -- 172.21.15.45:0/4017514899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa70e1d80 con 0x557aa7362f40 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34e8c86700 1 --2- 172.21.15.45:0/4017514899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa7375200 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005c50 tx=0x7f34d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba36cbb5de17d0cf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34cf7fe700 1 -- 172.21.15.45:0/4017514899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80274d0 con 0x557aa7362f40 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34cf7fe700 1 -- 172.21.15.45:0/4017514899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d8025040 con 0x557aa7362f40 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.614+0000 7f34cf7fe700 1 -- 172.21.15.45:0/4017514899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80277a0 con 0x557aa7362f40 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.615+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4017514899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 msgr2=0x557aa7375200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.615+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4017514899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa7375200 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005c50 tx=0x7f34d8014000 comp rx=0 tx=0).stop 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.615+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4017514899 shutdown_connections 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.615+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4017514899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa7366690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.615+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4017514899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa7375200 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.615+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/4017514899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 0x557aa7379950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.615+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4017514899 >> 172.21.15.45:0/4017514899 conn(0x557aa72d8200 msgr2=0x557aa736dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.616+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4017514899 shutdown_connections 2023-09-23T09:42:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.616+0000 7f34f70d4b80 1 -- 172.21.15.45:0/4017514899 wait complete. 2023-09-23T09:42:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.616+0000 7f34f70d4b80 1 -- start start 2023-09-23T09:42:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 1 Processor -- start 2023-09-23T09:42:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa73c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 0x557aa73c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa73c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa73ca2f0 con 0x557aa7362f40 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa73ca460 con 0x557aa7363860 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa73ca620 con 0x557aa736e710 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34f70d4b80 10 client.0 _ll_register_callbacks cb 0x7f34d80099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34e8c86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa73c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34e8c86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa73c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36610/0 (socket says 172.21.15.45:36610) 2023-09-23T09:42:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34e8c86700 1 -- 172.21.15.45:0/3564436502 learned_addr learned my addr 172.21.15.45:0/3564436502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.617+0000 7f34e9487700 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa73c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34e8c86700 1 -- 172.21.15.45:0/3564436502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 msgr2=0x557aa73c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34e8c86700 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa73c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34e8c86700 1 -- 172.21.15.45:0/3564436502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 msgr2=0x557aa73c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34e8c86700 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 0x557aa73c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34e8c86700 1 -- 172.21.15.45:0/3564436502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d8011e50 con 0x557aa7362f40 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34e8c86700 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa73c5c30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f34d8004ff0 tx=0x7f34d8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a753b1fab2475abc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:42:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34c3fff700 1 -- 172.21.15.45:0/3564436502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8004750 con 0x557aa7362f40 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34c3fff700 1 -- 172.21.15.45:0/3564436502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f34d8018070 con 0x557aa7362f40 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34dc00bcb0 con 0x557aa7362f40 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34c3fff700 1 -- 172.21.15.45:0/3564436502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34d80048b0 con 0x557aa7362f40 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34f70d4b80 10 client.16932 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.618+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557aa70cac00 con 0x557aa7362f40 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34c3fff700 1 -- 172.21.15.45:0/3564436502 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d8043020 con 0x557aa7362f40 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34c3fff700 1 -- 172.21.15.45:0/3564436502 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f34d802d020 con 0x557aa7362f40 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34f70d4b80 10 client.16932 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34f70d4b80 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34f70d4b80 20 client.16932 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa73d2fe0 0x557aa73d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34f70d4b80 4 client.16932 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:42:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557aa72afad0 con 0x557aa73d2fe0 2023-09-23T09:42:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34e3fff700 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa73d2fe0 0x557aa73d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34c3fff700 10 client.16932 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.619+0000 7f34e3fff700 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa73d2fe0 0x557aa73d5400 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37cd83424bcd683 server_cookie=79482fb97f4b7cc5 in_seq=0 out_seq=0 2023-09-23T09:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.661+0000 7f34c3fff700 1 -- 172.21.15.45:0/3564436502 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557aa72afad0 con 0x557aa73d2fe0 2023-09-23T09:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.661+0000 7f34c3fff700 10 client.16932 handle_command_reply: tid=0 2023-09-23T09:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.661+0000 7f34f70d4b80 1 client.16932 shutdown 2023-09-23T09:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 msgr2=0x557aa73c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa73c5c30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f34d8004ff0 tx=0x7f34d8005020 comp rx=0 tx=0).stop 2023-09-23T09:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 shutdown_connections 2023-09-23T09:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557aa73d2fe0 0x557aa73d5400 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557aa736e710 0x557aa73c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557aa7362f40 0x557aa73c5c30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 --2- 172.21.15.45:0/3564436502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557aa7363860 0x557aa73c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 >> 172.21.15.45:0/3564436502 conn(0x557aa72d8200 msgr2=0x557aa7379950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34c3fff700 0 client.16932 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 shutdown_connections 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.662+0000 7f34f70d4b80 1 -- 172.21.15.45:0/3564436502 wait complete. 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34f70d4b80 20 client.16932 trim_cache size 0 max 16384 2023-09-23T09:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f34d40a1810 msgr2=0x7f34d40a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f34d40a1810 0x7f34d40a3cd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009d80 tx=0x7f34d8009260 comp rx=0 tx=0).stop 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4100e70 msgr2=0x7f34e4199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4100e70 0x7f34e4199680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b550 tx=0x7f34d000b910 comp rx=0 tx=0).stop 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 shutdown_connections 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e4101790 0x7f34e419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e40ffaf0 0x7f34e4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f34d40a1810 0x7f34d40a3cd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 --2- 172.21.15.45:0/775414852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e4100e70 0x7f34e4199680 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 >> 172.21.15.45:0/775414852 conn(0x7f34e4074b90 msgr2=0x7f34e40fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 shutdown_connections 2023-09-23T09:42:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:40.663+0000 7f34eaeea700 1 -- 172.21.15.45:0/775414852 wait complete. 2023-09-23T09:42:41.885 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 775.0 s, 2053.4 tps, lat 49.010 ms stddev 60.596 2023-09-23T09:42:42.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:42 smithi098 ceph-mon[102386]: pgmap v415: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 902 op/s 2023-09-23T09:42:42.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:42 smithi045 ceph-mon[98627]: pgmap v415: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 902 op/s 2023-09-23T09:42:42.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:42 smithi123 ceph-mon[103577]: pgmap v415: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 902 op/s 2023-09-23T09:42:44.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:44 smithi098 ceph-mon[102386]: pgmap v416: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 652 op/s 2023-09-23T09:42:44.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:44 smithi045 ceph-mon[98627]: pgmap v416: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 652 op/s 2023-09-23T09:42:44.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:44 smithi123 ceph-mon[103577]: pgmap v416: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 652 op/s 2023-09-23T09:42:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:46 smithi045 ceph-mon[98627]: pgmap v417: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 786 op/s 2023-09-23T09:42:46.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:46 smithi123 ceph-mon[103577]: pgmap v417: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 786 op/s 2023-09-23T09:42:46.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:46 smithi098 ceph-mon[102386]: pgmap v417: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 786 op/s 2023-09-23T09:42:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 780.0 s, 1348.8 tps, lat 73.905 ms stddev 90.843 2023-09-23T09:42:48.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:48 smithi045 ceph-mon[98627]: pgmap v418: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 774 op/s 2023-09-23T09:42:48.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:48 smithi123 ceph-mon[103577]: pgmap v418: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 774 op/s 2023-09-23T09:42:48.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:48 smithi098 ceph-mon[102386]: pgmap v418: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 774 op/s 2023-09-23T09:42:50.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:50 smithi045 ceph-mon[98627]: pgmap v419: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.6 MiB/s wr, 609 op/s 2023-09-23T09:42:50.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:50 smithi123 ceph-mon[103577]: pgmap v419: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.6 MiB/s wr, 609 op/s 2023-09-23T09:42:50.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:50 smithi098 ceph-mon[102386]: pgmap v419: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.6 MiB/s wr, 609 op/s 2023-09-23T09:42:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 785.0 s, 1309.8 tps, lat 76.203 ms stddev 88.353 2023-09-23T09:42:52.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:51 smithi098 ceph-mon[102386]: pgmap v420: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 842 op/s 2023-09-23T09:42:52.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:51 smithi045 ceph-mon[98627]: pgmap v420: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 842 op/s 2023-09-23T09:42:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:51 smithi123 ceph-mon[103577]: pgmap v420: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 842 op/s 2023-09-23T09:42:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 -- 172.21.15.45:0/538009110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f116410d010 msgr2=0x7f1164114b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 --2- 172.21.15.45:0/538009110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f116410d010 0x7f1164114b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f115c009f90 tx=0x7f115c00b330 comp rx=0 tx=0).stop 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 -- 172.21.15.45:0/538009110 shutdown_connections 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 --2- 172.21.15.45:0/538009110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f116410d010 0x7f1164114b70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 --2- 172.21.15.45:0/538009110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 0x7f1164108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 --2- 172.21.15.45:0/538009110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11641092d0 0x7f116410c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 -- 172.21.15.45:0/538009110 >> 172.21.15.45:0/538009110 conn(0x7f116406cee0 msgr2=0x7f116406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 -- 172.21.15.45:0/538009110 shutdown_connections 2023-09-23T09:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.859+0000 7f1168bc1700 1 -- 172.21.15.45:0/538009110 wait complete. 2023-09-23T09:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 Processor -- start 2023-09-23T09:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 -- start start 2023-09-23T09:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 0x7f1164116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f116410d010 0x7f11641167d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1164116d10 0x7f11641b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11641b26b0 con 0x7f1164108920 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11641a38d0 con 0x7f1164116d10 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.860+0000 7f1168bc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11641a3a90 con 0x7f116410d010 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 0x7f1164116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 0x7f1164116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36624/0 (socket says 172.21.15.45:36624) 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 -- 172.21.15.45:0/2994026931 learned_addr learned my addr 172.21.15.45:0/2994026931 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f1159d9b700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f116410d010 0x7f11641167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f1162d9d700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1164116d10 0x7f11641b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 -- 172.21.15.45:0/2994026931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f116410d010 msgr2=0x7f11641167d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f116410d010 0x7f11641167d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 -- 172.21.15.45:0/2994026931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1164116d10 msgr2=0x7f11641b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1164116d10 0x7f11641b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 -- 172.21.15.45:0/2994026931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154009520 con 0x7f1164108920 2023-09-23T09:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f1162d9d700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1164116d10 0x7f11641b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f116259c700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 0x7f1164116290 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f115400eb30 tx=0x7f115400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f115bfff700 1 -- 172.21.15.45:0/2994026931 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154009880 con 0x7f1164108920 2023-09-23T09:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f115bfff700 1 -- 172.21.15.45:0/2994026931 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11540044e0 con 0x7f1164108920 2023-09-23T09:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.861+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c0095f0 con 0x7f1164108920 2023-09-23T09:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.862+0000 7f115bfff700 1 -- 172.21.15.45:0/2994026931 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154003d80 con 0x7f1164108920 2023-09-23T09:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.862+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c009c40 con 0x7f1164108920 2023-09-23T09:42:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.865+0000 7f115bfff700 1 -- 172.21.15.45:0/2994026931 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f1154003f20 con 0x7f1164108920 2023-09-23T09:42:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.865+0000 7f115bfff700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f11500a1710 0x7f11500a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.866+0000 7f115bfff700 1 -- 172.21.15.45:0/2994026931 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1154014070 con 0x7f1164108920 2023-09-23T09:42:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.866+0000 7f1159d9b700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f11500a1710 0x7f11500a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.866+0000 7f1159d9b700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f11500a1710 0x7f11500a3bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1164117c90 tx=0x7f114c009310 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 Processor -- start 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 -- start start 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68823a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b6882451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b688247b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6881a3400 con 0x55b68823a860 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b688194bc0 con 0x55b688239f40 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.876+0000 7f1174dabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6881924e0 con 0x55b688245710 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68823a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f1162d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b688247b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68823a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:57982/0 (socket says 172.21.15.45:57982) 2023-09-23T09:42:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 -- 172.21.15.45:0/1890718331 learned_addr learned my addr 172.21.15.45:0/1890718331 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f1159d9b700 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b6882451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 -- 172.21.15.45:0/1890718331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 msgr2=0x55b688247b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b688247b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 -- 172.21.15.45:0/1890718331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 msgr2=0x55b6882451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b6882451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 -- 172.21.15.45:0/1890718331 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6881f0870 con 0x55b688239f40 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f116259c700 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68823a320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f11540a2630 tx=0x7f115409a1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f833afce31f382b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f1158d99700 1 -- 172.21.15.45:0/1890718331 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115409aed0 con 0x55b688239f40 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f1158d99700 1 -- 172.21.15.45:0/1890718331 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115409a7d0 con 0x55b688239f40 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.877+0000 7f1158d99700 1 -- 172.21.15.45:0/1890718331 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115401fec0 con 0x55b688239f40 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.878+0000 7f1174dabb80 1 -- 172.21.15.45:0/1890718331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 msgr2=0x55b68823a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.878+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68823a320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f11540a2630 tx=0x7f115409a1b0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.880+0000 7f1174dabb80 1 -- 172.21.15.45:0/1890718331 shutdown_connections 2023-09-23T09:42:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.880+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b688247b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.880+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68823a320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.880+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1890718331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b6882451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.880+0000 7f1174dabb80 1 -- 172.21.15.45:0/1890718331 >> 172.21.15.45:0/1890718331 conn(0x55b6881af200 msgr2=0x55b6882363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.895+0000 7f1174dabb80 1 -- 172.21.15.45:0/1890718331 shutdown_connections 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.901+0000 7f1174dabb80 1 -- 172.21.15.45:0/1890718331 wait complete. 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.901+0000 7f1174dabb80 1 -- start start 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.901+0000 7f1174dabb80 1 Processor -- start 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688239f40 0x55b6882424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688245710 0x55b688242f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f1174dabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68818ddd0 con 0x55b688245710 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f1174dabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68812a6b0 con 0x55b68823a860 2023-09-23T09:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f1174dabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68812a350 con 0x55b688239f40 2023-09-23T09:42:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f1174dabb80 10 client.0 _ll_register_callbacks cb 0x55b687f877a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.902+0000 7f116259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688239f40 0x55b6882424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.903+0000 7f116259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688239f40 0x55b6882424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38744/0 (socket says 172.21.15.45:38744) 2023-09-23T09:42:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.903+0000 7f116259c700 1 -- 172.21.15.45:0/2581375612 learned_addr learned my addr 172.21.15.45:0/2581375612 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.903+0000 7f1162d9d700 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688245710 0x55b688242f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.909+0000 7f116259c700 1 -- 172.21.15.45:0/2581375612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 msgr2=0x55b688242a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:42:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.909+0000 7f116259c700 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.909+0000 7f116259c700 1 -- 172.21.15.45:0/2581375612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688245710 msgr2=0x55b688242f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.909+0000 7f116259c700 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688245710 0x55b688242f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.909+0000 7f116259c700 1 -- 172.21.15.45:0/2581375612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c009c40 con 0x55b688239f40 2023-09-23T09:42:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.924+0000 7f116259c700 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688239f40 0x55b6882424e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f115409a760 tx=0x7f1154005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131da2b3b0d337a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.924+0000 7f1149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:42:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.930+0000 7f1149ffb700 1 -- 172.21.15.45:0/2581375612 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115400cdd0 con 0x55b688239f40 2023-09-23T09:42:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.930+0000 7f1149ffb700 1 -- 172.21.15.45:0/2581375612 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f11540b5070 con 0x55b688239f40 2023-09-23T09:42:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.930+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b68822e090 con 0x55b688239f40 2023-09-23T09:42:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.941+0000 7f1149ffb700 1 -- 172.21.15.45:0/2581375612 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11540a2b60 con 0x55b688239f40 2023-09-23T09:42:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.943+0000 7f1174dabb80 10 client.26602 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.943+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55b687fb8d80 con 0x55b688239f40 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.943+0000 7f1149ffb700 1 -- 172.21.15.45:0/2581375612 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115407e020 con 0x55b688239f40 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.943+0000 7f1149ffb700 1 -- 172.21.15.45:0/2581375612 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1154099cc0 con 0x55b688239f40 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.954+0000 7f1174dabb80 10 client.26602 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.954+0000 7f1174dabb80 10 client.26602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.954+0000 7f1174dabb80 20 client.26602 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.955+0000 7f1174dabb80 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b688276e20 0x55b688279240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.955+0000 7f1174dabb80 4 client.26602 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:42:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.955+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b688186db0 con 0x55b688276e20 2023-09-23T09:42:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.955+0000 7f1159d9b700 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b688276e20 0x55b688279240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.955+0000 7f1159d9b700 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b688276e20 0x55b688279240 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8192a86e00974a40 server_cookie=44611654727efeaa in_seq=0 out_seq=0 2023-09-23T09:42:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.955+0000 7f1149ffb700 10 client.26602 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.956+0000 7f1149ffb700 1 -- 172.21.15.45:0/2581375612 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b688186db0 con 0x55b688276e20 2023-09-23T09:42:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.956+0000 7f1149ffb700 10 client.26602 handle_command_reply: tid=0 2023-09-23T09:42:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.957+0000 7f1174dabb80 1 client.26602 shutdown 2023-09-23T09:42:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.965+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688239f40 msgr2=0x55b6882424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.965+0000 7f1174dabb80 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688239f40 0x55b6882424e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f115409a760 tx=0x7f1154005b40 comp rx=0 tx=0).stop 2023-09-23T09:42:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.967+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 shutdown_connections 2023-09-23T09:42:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.967+0000 7f1174dabb80 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b688276e20 0x55b688279240 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.967+0000 7f1174dabb80 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688245710 0x55b688242f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.967+0000 7f1174dabb80 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688239f40 0x55b6882424e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.967+0000 7f1174dabb80 1 --2- 172.21.15.45:0/2581375612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.967+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 >> 172.21.15.45:0/2581375612 conn(0x55b6881af200 msgr2=0x55b688235f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.967+0000 7f1149ffb700 0 client.26602 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.978+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 shutdown_connections 2023-09-23T09:42:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.979+0000 7f1174dabb80 1 -- 172.21.15.45:0/2581375612 wait complete. 2023-09-23T09:42:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:53.984+0000 7f1174dabb80 20 client.26602 trim_cache size 0 max 16384 2023-09-23T09:42:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 Processor -- start 2023-09-23T09:42:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 -- start start 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688239f40 0x55b68823cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68823f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68800e8c0 con 0x55b688239f40 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68823aea0 con 0x55b68823a860 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f1174dabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68823d320 con 0x55b688245710 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.002+0000 7f116259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688239f40 0x55b68823cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688239f40 0x55b68823cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36646/0 (socket says 172.21.15.45:36646) 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 -- 172.21.15.45:0/3650727658 learned_addr learned my addr 172.21.15.45:0/3650727658 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f1159d9b700 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f1162d9d700 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68823f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 -- 172.21.15.45:0/3650727658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 msgr2=0x55b68823f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68823f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 -- 172.21.15.45:0/3650727658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 msgr2=0x55b688242bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 -- 172.21.15.45:0/3650727658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b687fa1c00 con 0x55b688239f40 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f1159d9b700 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f116259c700 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688239f40 0x55b68823cde0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f11540cb530 tx=0x7f115409a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa2cbfd13753613 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f115959a700 1 -- 172.21.15.45:0/3650727658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115409a130 con 0x55b688239f40 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f115959a700 1 -- 172.21.15.45:0/3650727658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115407e070 con 0x55b688239f40 2023-09-23T09:42:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.003+0000 7f115959a700 1 -- 172.21.15.45:0/3650727658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115400cdd0 con 0x55b688239f40 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 -- 172.21.15.45:0/3650727658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688239f40 msgr2=0x55b68823cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688239f40 0x55b68823cde0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f11540cb530 tx=0x7f115409a080 comp rx=0 tx=0).stop 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 -- 172.21.15.45:0/3650727658 shutdown_connections 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68823f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b688239f40 0x55b68823cde0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 --2- 172.21.15.45:0/3650727658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b68823a860 0x55b688242bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 -- 172.21.15.45:0/3650727658 >> 172.21.15.45:0/3650727658 conn(0x55b6881af200 msgr2=0x55b6882424a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 -- 172.21.15.45:0/3650727658 shutdown_connections 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 -- 172.21.15.45:0/3650727658 wait complete. 2023-09-23T09:42:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 -- start start 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.004+0000 7f1174dabb80 1 Processor -- start 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68824bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b68824c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1174dabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68824a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1174dabb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68824a930 con 0x55b68823a860 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1174dabb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68824aaa0 con 0x55b688239f40 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1174dabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68824ac60 con 0x55b688245710 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1174dabb80 10 client.0 _ll_register_callbacks cb 0x7f114c009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f116259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68824bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1162d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68824a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f116259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68824bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58006/0 (socket says 172.21.15.45:58006) 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f116259c700 1 -- 172.21.15.45:0/1081583741 learned_addr learned my addr 172.21.15.45:0/1081583741 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:42:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.005+0000 7f1159d9b700 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b68824c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1162d9d700 1 -- 172.21.15.45:0/1081583741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 msgr2=0x55b68824bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1162d9d700 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68824bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1162d9d700 1 -- 172.21.15.45:0/1081583741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 msgr2=0x55b68824c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:42:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1162d9d700 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b68824c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1162d9d700 1 -- 172.21.15.45:0/1081583741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c017070 con 0x55b688245710 2023-09-23T09:42:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1162d9d700 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68824a240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55b68823ae10 tx=0x7f115c003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5748f549909ce3e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:42:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:42:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.006+0000 7f1149ffb700 1 -- 172.21.15.45:0/1081583741 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c00e040 con 0x55b688245710 2023-09-23T09:42:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.007+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f115409a7d0 con 0x55b688245710 2023-09-23T09:42:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.007+0000 7f1149ffb700 1 -- 172.21.15.45:0/1081583741 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f115c00be00 con 0x55b688245710 2023-09-23T09:42:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.007+0000 7f1149ffb700 1 -- 172.21.15.45:0/1081583741 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f115c005040 con 0x55b688245710 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.007+0000 7f1174dabb80 10 client.26608 fetch_fsmap learned FSMap version 50 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.007+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55b687fb8d80 con 0x55b688245710 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1149ffb700 1 -- 172.21.15.45:0/1081583741 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115c032080 con 0x55b688245710 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1149ffb700 1 -- 172.21.15.45:0/1081583741 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f115c00b050 con 0x55b688245710 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1174dabb80 10 client.26608 fetch_fsmap finished waiting for FSMap version 50 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1174dabb80 10 client.26608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1174dabb80 20 client.26608 populate_metadata read hostname 'smithi045' 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b6882cbda0 0x55b6882ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1174dabb80 4 client.26608 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b688186ad0 con 0x55b6882cbda0 2023-09-23T09:42:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.008+0000 7f116259c700 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b6882cbda0 0x55b6882ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:42:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.009+0000 7f1149ffb700 10 client.26608 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.009+0000 7f116259c700 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b6882cbda0 0x55b6882ce1c0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cec1378ea354dc4 server_cookie=541d07db2b6c3d2 in_seq=0 out_seq=0 2023-09-23T09:42:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.036+0000 7f1149ffb700 1 -- 172.21.15.45:0/1081583741 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b688186ad0 con 0x55b6882cbda0 2023-09-23T09:42:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.036+0000 7f1149ffb700 10 client.26608 handle_command_reply: tid=0 2023-09-23T09:42:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 client.26608 shutdown 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 msgr2=0x55b68824a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68824a240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55b68823ae10 tx=0x7f115c003ce0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 shutdown_connections 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55b6882cbda0 0x55b6882ce1c0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b688245710 0x55b68824a240 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b688239f40 0x55b68824bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 --2- 172.21.15.45:0/1081583741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68823a860 0x55b68824c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 >> 172.21.15.45:0/1081583741 conn(0x55b6881af200 msgr2=0x55b688242d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.037+0000 7f1149ffb700 0 client.26608 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:42:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.039+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 shutdown_connections 2023-09-23T09:42:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.039+0000 7f1174dabb80 1 -- 172.21.15.45:0/1081583741 wait complete. 2023-09-23T09:42:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1174dabb80 20 client.26608 trim_cache size 0 max 16384 2023-09-23T09:42:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:42:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:42:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:42:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:42:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f11500a1710 msgr2=0x7f11500a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f11500a1710 0x7f11500a3bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1164117c90 tx=0x7f114c009310 comp rx=0 tx=0).stop 2023-09-23T09:42:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 msgr2=0x7f1164116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:42:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 0x7f1164116290 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f115400eb30 tx=0x7f115400ee40 comp rx=0 tx=0).stop 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 shutdown_connections 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1164116d10 0x7f11641b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f11500a1710 0x7f11500a3bd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1164108920 0x7f1164116290 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 --2- 172.21.15.45:0/2994026931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f116410d010 0x7f11641167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.040+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 >> 172.21.15.45:0/2994026931 conn(0x7f116406cee0 msgr2=0x7f1164070750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.041+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 shutdown_connections 2023-09-23T09:42:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:42:54.041+0000 7f1168bc1700 1 -- 172.21.15.45:0/2994026931 wait complete. 2023-09-23T09:42:54.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:53 smithi098 ceph-mon[102386]: pgmap v421: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 530 op/s 2023-09-23T09:42:54.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:53 smithi045 ceph-mon[98627]: pgmap v421: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 530 op/s 2023-09-23T09:42:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:53 smithi123 ceph-mon[103577]: pgmap v421: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 530 op/s 2023-09-23T09:42:56.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:55 smithi045 ceph-mon[98627]: pgmap v422: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 648 op/s 2023-09-23T09:42:56.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:55 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:42:56.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:55 smithi123 ceph-mon[103577]: pgmap v422: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 648 op/s 2023-09-23T09:42:56.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:55 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:42:56.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:55 smithi098 ceph-mon[102386]: pgmap v422: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 648 op/s 2023-09-23T09:42:56.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:55 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:42:56.882 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 790.0 s, 1633.8 tps, lat 61.928 ms stddev 71.015 2023-09-23T09:42:58.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:57 smithi045 ceph-mon[98627]: pgmap v423: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 647 op/s 2023-09-23T09:42:58.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:57 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:42:58.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:57 smithi123 ceph-mon[103577]: pgmap v423: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 647 op/s 2023-09-23T09:42:58.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:57 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:42:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:57 smithi098 ceph-mon[102386]: pgmap v423: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 647 op/s 2023-09-23T09:42:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:57 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:42:59.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:58 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:58 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:58 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:58 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:58 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:58 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:58 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:58 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:58 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:58 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:58 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:58 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:58 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:58 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:58 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:58 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:58 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:42:59.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:58 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:43:00.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:59 smithi098 ceph-mon[102386]: pgmap v424: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 532 op/s 2023-09-23T09:43:00.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:59 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:43:00.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:59 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:43:00.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:42:59 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:43:00.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:59 smithi045 ceph-mon[98627]: pgmap v424: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 532 op/s 2023-09-23T09:43:00.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:59 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:43:00.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:59 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:43:00.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:42:59 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:43:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:59 smithi123 ceph-mon[103577]: pgmap v424: 97 pgs: 97 active+clean; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 532 op/s 2023-09-23T09:43:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:59 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:43:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:59 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:43:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:42:59 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:43:01.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:01 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:01.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:01 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:01.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:01 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 795.0 s, 700.0 tps, lat 141.061 ms stddev 264.069 2023-09-23T09:43:02.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:02 smithi045 ceph-mon[98627]: pgmap v425: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.58k op/s 2023-09-23T09:43:02.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:02 smithi123 ceph-mon[103577]: pgmap v425: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.58k op/s 2023-09-23T09:43:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:02 smithi098 ceph-mon[102386]: pgmap v425: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.58k op/s 2023-09-23T09:43:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:03 smithi098 ceph-mon[102386]: pgmap v426: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-09-23T09:43:03.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:03 smithi045 ceph-mon[98627]: pgmap v426: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-09-23T09:43:03.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:03 smithi123 ceph-mon[103577]: pgmap v426: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-09-23T09:43:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:05 smithi098 ceph-mon[102386]: osdmap e102: 12 total, 12 up, 12 in 2023-09-23T09:43:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:05 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:05 smithi098 ceph-mon[102386]: pgmap v428: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 25 MiB/s wr, 1.91k op/s 2023-09-23T09:43:05.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:05 smithi045 ceph-mon[98627]: osdmap e102: 12 total, 12 up, 12 in 2023-09-23T09:43:05.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:05 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:05.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:05 smithi045 ceph-mon[98627]: pgmap v428: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 25 MiB/s wr, 1.91k op/s 2023-09-23T09:43:05.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:05 smithi123 ceph-mon[103577]: osdmap e102: 12 total, 12 up, 12 in 2023-09-23T09:43:05.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:05 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:05.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:05 smithi123 ceph-mon[103577]: pgmap v428: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 25 MiB/s wr, 1.91k op/s 2023-09-23T09:43:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 800.0 s, 979.4 tps, lat 101.857 ms stddev 163.142 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.236+0000 7f823c851700 1 -- 172.21.15.45:0/3559535420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 msgr2=0x7f8238115160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.236+0000 7f823c851700 1 --2- 172.21.15.45:0/3559535420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f8238115160 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8228009990 tx=0x7f8228009ca0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.236+0000 7f823c851700 1 -- 172.21.15.45:0/3559535420 shutdown_connections 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.236+0000 7f823c851700 1 --2- 172.21.15.45:0/3559535420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f8238115160 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.236+0000 7f823c851700 1 --2- 172.21.15.45:0/3559535420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82381097c0 0x7f8238109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.236+0000 7f823c851700 1 --2- 172.21.15.45:0/3559535420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f823810a0e0 0x7f8238112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.236+0000 7f823c851700 1 -- 172.21.15.45:0/3559535420 >> 172.21.15.45:0/3559535420 conn(0x7f823806cee0 msgr2=0x7f823806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.237+0000 7f823c851700 1 -- 172.21.15.45:0/3559535420 shutdown_connections 2023-09-23T09:43:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.237+0000 7f823c851700 1 -- 172.21.15.45:0/3559535420 wait complete. 2023-09-23T09:43:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.237+0000 7f823c851700 1 Processor -- start 2023-09-23T09:43:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.237+0000 7f823c851700 1 -- start start 2023-09-23T09:43:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f823c851700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82381097c0 0x7f8238115b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f823c851700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f823810a0e0 0x7f8238116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f823c851700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f82381ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f823c851700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381adfa0 con 0x7f8238112d70 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f823c851700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381ae0e0 con 0x7f823810a0e0 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f823c851700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381ae250 con 0x7f82381097c0 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f8236d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f82381ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f8236d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f82381ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36666/0 (socket says 172.21.15.45:36666) 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f8236d9d700 1 -- 172.21.15.45:0/4018611130 learned_addr learned my addr 172.21.15.45:0/4018611130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f8235d9b700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f823810a0e0 0x7f8238116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.238+0000 7f823659c700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82381097c0 0x7f8238115b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.239+0000 7f8236d9d700 1 -- 172.21.15.45:0/4018611130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82381097c0 msgr2=0x7f8238115b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.239+0000 7f8236d9d700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82381097c0 0x7f8238115b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.239+0000 7f8236d9d700 1 -- 172.21.15.45:0/4018611130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f823810a0e0 msgr2=0x7f8238116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.239+0000 7f8236d9d700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f823810a0e0 0x7f8238116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.239+0000 7f8236d9d700 1 -- 172.21.15.45:0/4018611130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82280095f0 con 0x7f8238112d70 2023-09-23T09:43:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.240+0000 7f8236d9d700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f82381ada60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8228005b40 tx=0x7f8228005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.241+0000 7f82277fe700 1 -- 172.21.15.45:0/4018611130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228023ad0 con 0x7f8238112d70 2023-09-23T09:43:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.241+0000 7f82277fe700 1 -- 172.21.15.45:0/4018611130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8228005580 con 0x7f8238112d70 2023-09-23T09:43:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.241+0000 7f82277fe700 1 -- 172.21.15.45:0/4018611130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822800fa90 con 0x7f8238112d70 2023-09-23T09:43:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.241+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823819f5e0 con 0x7f8238112d70 2023-09-23T09:43:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.241+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823819fa40 con 0x7f8238112d70 2023-09-23T09:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.244+0000 7f82277fe700 1 -- 172.21.15.45:0/4018611130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f8228029020 con 0x7f8238112d70 2023-09-23T09:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.244+0000 7f82277fe700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82200a1790 0x7f82200a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.244+0000 7f82277fe700 1 -- 172.21.15.45:0/4018611130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f82280d6820 con 0x7f8238112d70 2023-09-23T09:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.244+0000 7f823659c700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82200a1790 0x7f82200a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.245+0000 7f823659c700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82200a1790 0x7f82200a3c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c009190 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 Processor -- start 2023-09-23T09:43:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 -- start start 2023-09-23T09:43:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eec2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec2860 0x559c5eecd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eecd710 0x559c5eecfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5ee2b400 con 0x559c5eec1f40 2023-09-23T09:43:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5ee1cbc0 con 0x559c5eecd710 2023-09-23T09:43:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5ee1a4e0 con 0x559c5eec2860 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8235d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec2860 0x559c5eecd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8235d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec2860 0x559c5eecd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38782/0 (socket says 172.21.15.45:38782) 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.246+0000 7f8235d9b700 1 -- 172.21.15.45:0/68088975 learned_addr learned my addr 172.21.15.45:0/68088975 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8235d9b700 1 -- 172.21.15.45:0/68088975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eecd710 msgr2=0x559c5eecfb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8235d9b700 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eecd710 0x559c5eecfb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8235d9b700 1 -- 172.21.15.45:0/68088975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 msgr2=0x559c5eec2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f823659c700 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eec2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8235d9b700 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eec2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8235d9b700 1 -- 172.21.15.45:0/68088975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c5ee78870 con 0x559c5eec2860 2023-09-23T09:43:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f823659c700 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eec2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:43:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8235d9b700 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec2860 0x559c5eecd1d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f823000a4d0 tx=0x7f82300096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44b448749d31dc3f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8224ff9700 1 -- 172.21.15.45:0/68088975 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230010040 con 0x559c5eec2860 2023-09-23T09:43:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8224ff9700 1 -- 172.21.15.45:0/68088975 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8230014ae0 con 0x559c5eec2860 2023-09-23T09:43:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.247+0000 7f8224ff9700 1 -- 172.21.15.45:0/68088975 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82300139b0 con 0x559c5eec2860 2023-09-23T09:43:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 -- 172.21.15.45:0/68088975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec2860 msgr2=0x559c5eecd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec2860 0x559c5eecd1d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f823000a4d0 tx=0x7f82300096d0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 -- 172.21.15.45:0/68088975 shutdown_connections 2023-09-23T09:43:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eecd710 0x559c5eecfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eec2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/68088975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec2860 0x559c5eecd1d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 -- 172.21.15.45:0/68088975 >> 172.21.15.45:0/68088975 conn(0x559c5ee37200 msgr2=0x559c5eebe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 -- 172.21.15.45:0/68088975 shutdown_connections 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.248+0000 7f8248a3bb80 1 -- 172.21.15.45:0/68088975 wait complete. 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 -- start start 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 Processor -- start 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eecd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eeca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eecaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5ee15dd0 con 0x559c5eec1f40 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5edb26b0 con 0x559c5eec2860 2023-09-23T09:43:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5edb2350 con 0x559c5eecd710 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8248a3bb80 10 client.0 _ll_register_callbacks cb 0x559c5ec0f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f823659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eecd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8235d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eeca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f823659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eecd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36678/0 (socket says 172.21.15.45:36678) 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f823659c700 1 -- 172.21.15.45:0/819466705 learned_addr learned my addr 172.21.15.45:0/819466705 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8236d9d700 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eecaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8235d9b700 1 -- 172.21.15.45:0/819466705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 msgr2=0x559c5eecaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8235d9b700 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eecaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8235d9b700 1 -- 172.21.15.45:0/819466705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 msgr2=0x559c5eecd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.249+0000 7f8235d9b700 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eecd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f8235d9b700 1 -- 172.21.15.45:0/819466705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8230009d00 con 0x559c5eec2860 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f8235d9b700 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eeca390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f823000a4d0 tx=0x7f8230014f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5fa34eb0370ef4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f821dffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f821dffb700 1 -- 172.21.15.45:0/819466705 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230010070 con 0x559c5eec2860 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c5eeb6090 con 0x559c5eec2860 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f821dffb700 1 -- 172.21.15.45:0/819466705 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8230014830 con 0x559c5eec2860 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f821dffb700 1 -- 172.21.15.45:0/819466705 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82300135d0 con 0x559c5eec2860 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f8248a3bb80 10 client.26615 fetch_fsmap learned FSMap version 53 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x559c5ec40d80 con 0x559c5eec2860 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f821dffb700 1 -- 172.21.15.45:0/819466705 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8230032080 con 0x559c5eec2860 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.250+0000 7f821dffb700 1 -- 172.21.15.45:0/819466705 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8230008030 con 0x559c5eec2860 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f8248a3bb80 10 client.26615 fetch_fsmap finished waiting for FSMap version 53 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f8248a3bb80 10 client.26615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f8248a3bb80 20 client.26615 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5eedd070 0x559c5eedf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f8248a3bb80 4 client.26615 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:43:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c5ee0edb0 con 0x559c5eedd070 2023-09-23T09:43:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f823659c700 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5eedd070 0x559c5eedf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f821dffb700 10 client.26615 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.251+0000 7f823659c700 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5eedd070 0x559c5eedf490 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a8e00faf7b048c server_cookie=3ae94a5c687d93f7 in_seq=0 out_seq=0 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.253+0000 7f821dffb700 1 -- 172.21.15.45:0/819466705 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c5ee0edb0 con 0x559c5eedd070 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.253+0000 7f821dffb700 10 client.26615 handle_command_reply: tid=0 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.253+0000 7f8248a3bb80 1 client.26615 shutdown 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 msgr2=0x559c5eeca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eeca390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f823000a4d0 tx=0x7f8230014f00 comp rx=0 tx=0).stop 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 shutdown_connections 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5eedd070 0x559c5eedf490 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eecaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec1f40 0x559c5eecd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f821dffb700 0 client.26615 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/819466705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eeca390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 >> 172.21.15.45:0/819466705 conn(0x559c5ee37200 msgr2=0x559c5eebdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 shutdown_connections 2023-09-23T09:43:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 1 -- 172.21.15.45:0/819466705 wait complete. 2023-09-23T09:43:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.255+0000 7f8248a3bb80 20 client.26615 trim_cache size 0 max 16384 2023-09-23T09:43:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 Processor -- start 2023-09-23T09:43:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 -- start start 2023-09-23T09:43:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec1f40 0x559c5eebcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eed9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eecd710 0x559c5eed89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5ec968c0 con 0x559c5eecd710 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5eebd030 con 0x559c5eec2860 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5eebf100 con 0x559c5eec1f40 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8236d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eecd710 0x559c5eed89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f823659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec1f40 0x559c5eebcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8235d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eed9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f823659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec1f40 0x559c5eebcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38792/0 (socket says 172.21.15.45:38792) 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8236d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eecd710 0x559c5eed89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36690/0 (socket says 172.21.15.45:36690) 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f8235d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eed9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58038/0 (socket says 172.21.15.45:58038) 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.275+0000 7f823659c700 1 -- 172.21.15.45:0/270572887 learned_addr learned my addr 172.21.15.45:0/270572887 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f823659c700 1 -- 172.21.15.45:0/270572887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 msgr2=0x559c5eed9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f823659c700 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eed9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f823659c700 1 -- 172.21.15.45:0/270572887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eecd710 msgr2=0x559c5eed89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f823659c700 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eecd710 0x559c5eed89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f823659c700 1 -- 172.21.15.45:0/270572887 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c5ec29c00 con 0x559c5eec1f40 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f8236d9d700 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eecd710 0x559c5eed89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f8235d9b700 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eed9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f823659c700 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec1f40 0x559c5eebcaf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f822c00e4b0 tx=0x7f822c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13cd2d19b763ad7b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f82257fa700 1 -- 172.21.15.45:0/270572887 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c026070 con 0x559c5eec1f40 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f82257fa700 1 -- 172.21.15.45:0/270572887 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822c00eea0 con 0x559c5eec1f40 2023-09-23T09:43:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.276+0000 7f82257fa700 1 -- 172.21.15.45:0/270572887 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c015480 con 0x559c5eec1f40 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.277+0000 7f8248a3bb80 1 -- 172.21.15.45:0/270572887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec1f40 msgr2=0x559c5eebcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.277+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec1f40 0x559c5eebcaf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f822c00e4b0 tx=0x7f822c015000 comp rx=0 tx=0).stop 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.277+0000 7f8248a3bb80 1 -- 172.21.15.45:0/270572887 shutdown_connections 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.277+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eecd710 0x559c5eed89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.277+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eec1f40 0x559c5eebcaf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.277+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/270572887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec2860 0x559c5eed9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.277+0000 7f8248a3bb80 1 -- 172.21.15.45:0/270572887 >> 172.21.15.45:0/270572887 conn(0x559c5ee37200 msgr2=0x559c5eee3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.278+0000 7f8248a3bb80 1 -- 172.21.15.45:0/270572887 shutdown_connections 2023-09-23T09:43:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.278+0000 7f8248a3bb80 1 -- 172.21.15.45:0/270572887 wait complete. 2023-09-23T09:43:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 -- start start 2023-09-23T09:43:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 Processor -- start 2023-09-23T09:43:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec1f40 0x559c5eed3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec2860 0x559c5eed43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eed0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5eed0a30 con 0x559c5eec2860 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5eed0ba0 con 0x559c5eec1f40 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5eed0d60 con 0x559c5eecd710 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f8248a3bb80 10 client.0 _ll_register_callbacks cb 0x559c5ec0f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f823659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec1f40 0x559c5eed3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f823659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec1f40 0x559c5eed3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58042/0 (socket says 172.21.15.45:58042) 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.279+0000 7f823659c700 1 -- 172.21.15.45:0/3513296414 learned_addr learned my addr 172.21.15.45:0/3513296414 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f8235d9b700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec2860 0x559c5eed43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f8236d9d700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eed0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f823659c700 1 -- 172.21.15.45:0/3513296414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 msgr2=0x559c5eed0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f823659c700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eed0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f823659c700 1 -- 172.21.15.45:0/3513296414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec2860 msgr2=0x559c5eed43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f823659c700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec2860 0x559c5eed43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f823659c700 1 -- 172.21.15.45:0/3513296414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c5ec40d80 con 0x559c5eec1f40 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f8235d9b700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec2860 0x559c5eed43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f823659c700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec1f40 0x559c5eed3e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f822c009810 tx=0x7f822c015690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=294e77b617f16a47 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f821dffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f821dffb700 1 -- 172.21.15.45:0/3513296414 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c026050 con 0x559c5eec1f40 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f821dffb700 1 -- 172.21.15.45:0/3513296414 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f822c015af0 con 0x559c5eec1f40 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8230014ae0 con 0x559c5eec1f40 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f821dffb700 1 -- 172.21.15.45:0/3513296414 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f822c015cb0 con 0x559c5eec1f40 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f8248a3bb80 10 client.26621 fetch_fsmap learned FSMap version 53 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.280+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f823819fa40 con 0x559c5eec1f40 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f821dffb700 1 -- 172.21.15.45:0/3513296414 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822c03e020 con 0x559c5eec1f40 2023-09-23T09:43:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f821dffb700 1 -- 172.21.15.45:0/3513296414 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f822c027320 con 0x559c5eec1f40 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f8248a3bb80 10 client.26621 fetch_fsmap finished waiting for FSMap version 53 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f8248a3bb80 10 client.26621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f8248a3bb80 20 client.26621 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5ef31f60 0x559c5ef34380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f8248a3bb80 4 client.26621 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c5ee0ead0 con 0x559c5ef31f60 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f8235d9b700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5ef31f60 0x559c5ef34380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.281+0000 7f821dffb700 10 client.26621 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.282+0000 7f8235d9b700 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5ef31f60 0x559c5ef34380 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aed79bb4df18691 server_cookie=9310dc192382cc9f in_seq=0 out_seq=0 2023-09-23T09:43:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f821dffb700 1 -- 172.21.15.45:0/3513296414 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c5ee0ead0 con 0x559c5ef31f60 2023-09-23T09:43:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f821dffb700 10 client.26621 handle_command_reply: tid=0 2023-09-23T09:43:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 client.26621 shutdown 2023-09-23T09:43:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec1f40 msgr2=0x559c5eed3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec1f40 0x559c5eed3e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f822c009810 tx=0x7f822c015690 comp rx=0 tx=0).stop 2023-09-23T09:43:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 shutdown_connections 2023-09-23T09:43:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559c5ef31f60 0x559c5ef34380 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c5eecd710 0x559c5eed0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f821dffb700 0 client.26621 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c5eec1f40 0x559c5eed3e70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 --2- 172.21.15.45:0/3513296414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c5eec2860 0x559c5eed43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 >> 172.21.15.45:0/3513296414 conn(0x559c5ee37200 msgr2=0x559c5eed9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 shutdown_connections 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 1 -- 172.21.15.45:0/3513296414 wait complete. 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.301+0000 7f8248a3bb80 20 client.26621 trim_cache size 0 max 16384 2023-09-23T09:43:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:43:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:43:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:43:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:43:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82200a1790 msgr2=0x7f82200a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82200a1790 0x7f82200a3c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c009190 comp rx=0 tx=0).stop 2023-09-23T09:43:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 msgr2=0x7f82381ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f82381ada60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8228005b40 tx=0x7f8228005ee0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 shutdown_connections 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f82200a1790 0x7f82200a3c50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8238112d70 0x7f82381ada60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82381097c0 0x7f8238115b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 --2- 172.21.15.45:0/4018611130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f823810a0e0 0x7f8238116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 >> 172.21.15.45:0/4018611130 conn(0x7f823806cee0 msgr2=0x7f8238070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.302+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 shutdown_connections 2023-09-23T09:43:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:07.303+0000 7f823c851700 1 -- 172.21.15.45:0/4018611130 wait complete. 2023-09-23T09:43:08.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:07 smithi098 ceph-mon[102386]: pgmap v429: 97 pgs: 33 active+clean+snaptrim_wait, 50 active+clean, 14 active+clean+snaptrim; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 2.20k op/s 2023-09-23T09:43:08.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:07 smithi045 ceph-mon[98627]: pgmap v429: 97 pgs: 33 active+clean+snaptrim_wait, 50 active+clean, 14 active+clean+snaptrim; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 2.20k op/s 2023-09-23T09:43:08.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:07 smithi123 ceph-mon[103577]: pgmap v429: 97 pgs: 33 active+clean+snaptrim_wait, 50 active+clean, 14 active+clean+snaptrim; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 2.20k op/s 2023-09-23T09:43:09.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:43:09.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:08 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:09.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:08 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:09.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:08 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.557+0000 7fbfacaac700 1 -- 172.21.15.45:0/1673621783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa80ffec0 msgr2=0x7fbfa8105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.557+0000 7fbfacaac700 1 --2- 172.21.15.45:0/1673621783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa80ffec0 0x7fbfa8105a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009990 tx=0x7fbf98009ca0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.558+0000 7fbfacaac700 1 -- 172.21.15.45:0/1673621783 shutdown_connections 2023-09-23T09:43:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.558+0000 7fbfacaac700 1 --2- 172.21.15.45:0/1673621783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfa8106130 0x7fbfa810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.558+0000 7fbfacaac700 1 --2- 172.21.15.45:0/1673621783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfa80ff510 0x7fbfa80ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.558+0000 7fbfacaac700 1 --2- 172.21.15.45:0/1673621783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa80ffec0 0x7fbfa8105a50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.558+0000 7fbfacaac700 1 -- 172.21.15.45:0/1673621783 >> 172.21.15.45:0/1673621783 conn(0x7fbfa8077290 msgr2=0x7fbfa80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.558+0000 7fbfacaac700 1 -- 172.21.15.45:0/1673621783 shutdown_connections 2023-09-23T09:43:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.558+0000 7fbfacaac700 1 -- 172.21.15.45:0/1673621783 wait complete. 2023-09-23T09:43:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 Processor -- start 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 -- start start 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfa80ff510 0x7fbfa8196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa8106130 0x7fbfa8197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfa819b8e0 0x7fbfa819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa819c360 con 0x7fbfa80ff510 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa819c4d0 con 0x7fbfa819b8e0 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfacaac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa819c690 con 0x7fbfa8106130 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfa80ff510 0x7fbfa8196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa8106130 0x7fbfa8197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfa80ff510 0x7fbfa8196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36706/0 (socket says 172.21.15.45:36706) 2023-09-23T09:43:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa659c700 1 -- 172.21.15.45:0/2239200637 learned_addr learned my addr 172.21.15.45:0/2239200637 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa8106130 0x7fbfa8197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38814/0 (socket says 172.21.15.45:38814) 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa6d9d700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfa819b8e0 0x7fbfa819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa5d9b700 1 -- 172.21.15.45:0/2239200637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfa819b8e0 msgr2=0x7fbfa819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfa819b8e0 0x7fbfa819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa5d9b700 1 -- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfa80ff510 msgr2=0x7fbfa8196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfa80ff510 0x7fbfa8196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.559+0000 7fbfa5d9b700 1 -- 172.21.15.45:0/2239200637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c009520 con 0x7fbfa8106130 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.560+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa8106130 0x7fbfa8197480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8073510 tx=0x7fbf98009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.560+0000 7fbf977fe700 1 -- 172.21.15.45:0/2239200637 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9801d070 con 0x7fbfa8106130 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.560+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf980095f0 con 0x7fbfa8106130 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.560+0000 7fbf977fe700 1 -- 172.21.15.45:0/2239200637 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf98021410 con 0x7fbfa8106130 2023-09-23T09:43:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.560+0000 7fbf977fe700 1 -- 172.21.15.45:0/2239200637 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98017650 con 0x7fbfa8106130 2023-09-23T09:43:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.560+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa81a3e80 con 0x7fbfa8106130 2023-09-23T09:43:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.563+0000 7fbf977fe700 1 -- 172.21.15.45:0/2239200637 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fbf98021810 con 0x7fbfa8106130 2023-09-23T09:43:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.563+0000 7fbf977fe700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbf900a1710 0x7fbf900a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.563+0000 7fbf977fe700 1 -- 172.21.15.45:0/2239200637 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbf980ce630 con 0x7fbfa8106130 2023-09-23T09:43:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.564+0000 7fbfa659c700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbf900a1710 0x7fbf900a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.564+0000 7fbfa659c700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbf900a1710 0x7fbf900a3bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009ef0 tx=0x7fbf9c009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.565+0000 7fbfb8c96b80 1 Processor -- start 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfb8c96b80 1 -- start start 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ab87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8acbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ace7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8a21460 con 0x55a7f8ab8400 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8a12c20 con 0x55a7f8acc3e0 2023-09-23T09:43:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8a10540 con 0x55a7f8ab8d20 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ab87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ab87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36714/0 (socket says 172.21.15.45:36714) 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfa6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ace7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.570+0000 7fbfa659c700 1 -- 172.21.15.45:0/3917147849 learned_addr learned my addr 172.21.15.45:0/3917147849 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8acbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbfa6d9d700 1 -- 172.21.15.45:0/3917147849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 msgr2=0x55a7f8acbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbfa6d9d700 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8acbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbfa6d9d700 1 -- 172.21.15.45:0/3917147849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 msgr2=0x55a7f8ab87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbfa6d9d700 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ab87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbfa6d9d700 1 -- 172.21.15.45:0/3917147849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f8a6e8d0 con 0x55a7f8acc3e0 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbfa6d9d700 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ace7d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0011580 tx=0x7fbfa0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48aa193d0c16d3c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbf94ff9700 1 -- 172.21.15.45:0/3917147849 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000e040 con 0x55a7f8acc3e0 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbf94ff9700 1 -- 172.21.15.45:0/3917147849 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfa000f040 con 0x55a7f8acc3e0 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.571+0000 7fbf94ff9700 1 -- 172.21.15.45:0/3917147849 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0004430 con 0x55a7f8acc3e0 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/3917147849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8acc3e0 msgr2=0x55a7f8ace7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ace7d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0011580 tx=0x7fbfa0009920 comp rx=0 tx=0).stop 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/3917147849 shutdown_connections 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ace7d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ab87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/3917147849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8acbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/3917147849 >> 172.21.15.45:0/3917147849 conn(0x55a7f8a2d260 msgr2=0x55a7f8ab4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/3917147849 shutdown_connections 2023-09-23T09:43:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.572+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/3917147849 wait complete. 2023-09-23T09:43:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 -- start start 2023-09-23T09:43:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 Processor -- start 2023-09-23T09:43:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8a0be30 con 0x55a7f8acc3e0 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f89a8710 con 0x55a7f8ab8400 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f89a83b0 con 0x55a7f8ab8d20 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfb8c96b80 10 client.0 _ll_register_callbacks cb 0x55a7f8805800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58070/0 (socket says 172.21.15.45:58070) 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfa659c700 1 -- 172.21.15.45:0/515372692 learned_addr learned my addr 172.21.15.45:0/515372692 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.573+0000 7fbfa6d9d700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfa659c700 1 -- 172.21.15.45:0/515372692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 msgr2=0x55a7f8ac9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfa659c700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfa659c700 1 -- 172.21.15.45:0/515372692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8acc3e0 msgr2=0x55a7f8ac9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfa659c700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfa659c700 1 -- 172.21.15.45:0/515372692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa0011230 con 0x55a7f8ab8400 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfa659c700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac90e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00fb50 tx=0x7fbf9c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75fb2cd2a28c6d9c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbf8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbf8dffb700 1 -- 172.21.15.45:0/515372692 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c022070 con 0x55a7f8ab8400 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7f8aac250 con 0x55a7f8ab8400 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbf8dffb700 1 -- 172.21.15.45:0/515372692 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbf9c00fe60 con 0x55a7f8ab8400 2023-09-23T09:43:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbf8dffb700 1 -- 172.21.15.45:0/515372692 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9c01da30 con 0x55a7f8ab8400 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfb8c96b80 10 client.26633 fetch_fsmap learned FSMap version 54 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55a7f8836de0 con 0x55a7f8ab8400 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.574+0000 7fbf8dffb700 1 -- 172.21.15.45:0/515372692 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7f8aac250 con 0x55a7f8ab8400 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbf8dffb700 1 -- 172.21.15.45:0/515372692 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 54) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fbf9c01c410 con 0x55a7f8ab8400 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfb8c96b80 10 client.26633 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfb8c96b80 10 client.26633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfb8c96b80 20 client.26633 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8adbfd0 0x55a7f8ade3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfb8c96b80 4 client.26633 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7f8a04e10 con 0x55a7f8adbfd0 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8adbfd0 0x55a7f8ade3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbf8dffb700 10 client.26633 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.575+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8adbfd0 0x55a7f8ade3f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd818d3892e7b42 server_cookie=b4ace7ebb4ab94bd in_seq=0 out_seq=0 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.576+0000 7fbf8dffb700 1 -- 172.21.15.45:0/515372692 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7f8a04e10 con 0x55a7f8adbfd0 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.576+0000 7fbf8dffb700 10 client.26633 handle_command_reply: tid=0 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.576+0000 7fbfb8c96b80 1 client.26633 shutdown 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.576+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8400 msgr2=0x55a7f8ac90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.576+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac90e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00fb50 tx=0x7fbf9c0047c0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 shutdown_connections 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8adbfd0 0x55a7f8ade3f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac90e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/515372692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 >> 172.21.15.45:0/515372692 conn(0x55a7f8a2d260 msgr2=0x55a7f8ab4380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbf8dffb700 0 client.26633 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 shutdown_connections 2023-09-23T09:43:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/515372692 wait complete. 2023-09-23T09:43:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.577+0000 7fbfb8c96b80 20 client.26633 trim_cache size 0 max 16384 2023-09-23T09:43:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 Processor -- start 2023-09-23T09:43:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 -- start start 2023-09-23T09:43:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ad2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f888c920 con 0x55a7f8ab8400 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8ad95c0 con 0x55a7f8ab8d20 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.601+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8ad3490 con 0x55a7f8acc3e0 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ad2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ad2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36728/0 (socket says 172.21.15.45:36728) 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 -- 172.21.15.45:0/453451945 learned_addr learned my addr 172.21.15.45:0/453451945 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa6d9d700 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 -- 172.21.15.45:0/453451945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 msgr2=0x55a7f8ac6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 -- 172.21.15.45:0/453451945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 msgr2=0x55a7f8ac9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 -- 172.21.15.45:0/453451945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f8ab0480 con 0x55a7f8ab8400 2023-09-23T09:43:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbfa659c700 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ad2f20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c01c7f0 tx=0x7fbf9c009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833baba07815100d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbf957fa700 1 -- 172.21.15.45:0/453451945 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c022070 con 0x55a7f8ab8400 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbf957fa700 1 -- 172.21.15.45:0/453451945 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c00fe60 con 0x55a7f8ab8400 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.602+0000 7fbf957fa700 1 -- 172.21.15.45:0/453451945 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c01ccf0 con 0x55a7f8ab8400 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.603+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/453451945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 msgr2=0x55a7f8ad2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.603+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ad2f20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c01c7f0 tx=0x7fbf9c009c70 comp rx=0 tx=0).stop 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.603+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/453451945 shutdown_connections 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.603+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.603+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ad2f20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.603+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/453451945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.603+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/453451945 >> 172.21.15.45:0/453451945 conn(0x55a7f8a2d260 msgr2=0x55a7f8ac4440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/453451945 shutdown_connections 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/453451945 wait complete. 2023-09-23T09:43:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 -- start start 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 Processor -- start 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8ac5a70 con 0x55a7f8ab8400 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8ac5be0 con 0x55a7f8ab8d20 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f8ac5da0 con 0x55a7f8acc3e0 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfb8c96b80 10 client.0 _ll_register_callbacks cb 0x7fbf9c00f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfa5d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36734/0 (socket says 172.21.15.45:36734) 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.604+0000 7fbfa659c700 1 -- 172.21.15.45:0/2579646674 learned_addr learned my addr 172.21.15.45:0/2579646674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfa659c700 1 -- 172.21.15.45:0/2579646674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 msgr2=0x55a7f8ac5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfa659c700 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfa659c700 1 -- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 msgr2=0x55a7f8ac6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfa659c700 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfa659c700 1 -- 172.21.15.45:0/2579646674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7f8836de0 con 0x55a7f8ab8400 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfa659c700 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac6730 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009310 tx=0x7fbf9c009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907fb0a31c4f9ce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbf8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbf8dffb700 1 -- 172.21.15.45:0/2579646674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c022070 con 0x55a7f8ab8400 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7f8ad8930 con 0x55a7f8ab8400 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbf8dffb700 1 -- 172.21.15.45:0/2579646674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbf9c01cba0 con 0x55a7f8ab8400 2023-09-23T09:43:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbf8dffb700 1 -- 172.21.15.45:0/2579646674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9c01cd60 con 0x55a7f8ab8400 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfb8c96b80 10 client.16983 fetch_fsmap learned FSMap version 54 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.605+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55a7f881fc60 con 0x55a7f8ab8400 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbf8dffb700 1 -- 172.21.15.45:0/2579646674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7f8ad8930 con 0x55a7f8ab8400 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbf8dffb700 1 -- 172.21.15.45:0/2579646674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 54) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fbf9c003aa0 con 0x55a7f8ab8400 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbfb8c96b80 10 client.16983 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbfb8c96b80 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbfb8c96b80 20 client.16983 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8b30b30 0x55a7f8b32f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbfb8c96b80 4 client.16983 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fbfa000f040 con 0x55a7f8b30b30 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.606+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8b30b30 0x55a7f8b32f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbf8dffb700 10 client.16983 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfa5d9b700 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8b30b30 0x55a7f8b32f40 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb3197967ceaa235 server_cookie=decc6048fc5ca520 in_seq=0 out_seq=0 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbf8dffb700 1 -- 172.21.15.45:0/2579646674 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fbfa000f040 con 0x55a7f8b30b30 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbf8dffb700 10 client.16983 handle_command_reply: tid=0 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 client.16983 shutdown 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 msgr2=0x55a7f8ac6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac6730 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009310 tx=0x7fbf9c009d70 comp rx=0 tx=0).stop 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 shutdown_connections 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a7f8b30b30 0x55a7f8b32f40 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7f8acc3e0 0x55a7f8ac5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7f8ab8400 0x55a7f8ac6730 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 --2- 172.21.15.45:0/2579646674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a7f8ab8d20 0x55a7f8ac6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 >> 172.21.15.45:0/2579646674 conn(0x55a7f8a2d260 msgr2=0x55a7f8ac2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.607+0000 7fbf8dffb700 0 client.16983 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.610+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 shutdown_connections 2023-09-23T09:43:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.612+0000 7fbfb8c96b80 1 -- 172.21.15.45:0/2579646674 wait complete. 2023-09-23T09:43:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.613+0000 7fbfb8c96b80 20 client.16983 trim_cache size 0 max 16384 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:43:09.615 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:43:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.618+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbf900a1710 msgr2=0x7fbf900a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.618+0000 7fbfacaac700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbf900a1710 0x7fbf900a3bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009ef0 tx=0x7fbf9c009260 comp rx=0 tx=0).stop 2023-09-23T09:43:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.618+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa8106130 msgr2=0x7fbfa8197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.618+0000 7fbfacaac700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa8106130 0x7fbfa8197480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8073510 tx=0x7fbf98009580 comp rx=0 tx=0).stop 2023-09-23T09:43:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.625+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 shutdown_connections 2023-09-23T09:43:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.625+0000 7fbfacaac700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfa819b8e0 0x7fbfa819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.625+0000 7fbfacaac700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbf900a1710 0x7fbf900a3bd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.625+0000 7fbfacaac700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfa80ff510 0x7fbfa8196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.625+0000 7fbfacaac700 1 --2- 172.21.15.45:0/2239200637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa8106130 0x7fbfa8197480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.625+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 >> 172.21.15.45:0/2239200637 conn(0x7fbfa8077290 msgr2=0x7fbfa81057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:09.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.626+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 shutdown_connections 2023-09-23T09:43:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:09.630+0000 7fbfacaac700 1 -- 172.21.15.45:0/2239200637 wait complete. 2023-09-23T09:43:09.645 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:43:10.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:09 smithi098 ceph-mon[102386]: pgmap v430: 97 pgs: 33 active+clean+snaptrim_wait, 50 active+clean, 14 active+clean+snaptrim; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 2.19k op/s 2023-09-23T09:43:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:09 smithi045 ceph-mon[98627]: pgmap v430: 97 pgs: 33 active+clean+snaptrim_wait, 50 active+clean, 14 active+clean+snaptrim; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 2.19k op/s 2023-09-23T09:43:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:09 smithi123 ceph-mon[103577]: pgmap v430: 97 pgs: 33 active+clean+snaptrim_wait, 50 active+clean, 14 active+clean+snaptrim; 5.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 2.19k op/s 2023-09-23T09:43:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 805.0 s, 888.1 tps, lat 95.993 ms stddev 117.926 2023-09-23T09:43:12.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:11 smithi098 ceph-mon[102386]: pgmap v431: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-09-23T09:43:12.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:11 smithi045 ceph-mon[98627]: pgmap v431: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-09-23T09:43:12.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:11 smithi123 ceph-mon[103577]: pgmap v431: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-09-23T09:43:14.298 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:13 smithi098 ceph-mon[102386]: pgmap v432: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-09-23T09:43:14.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:13 smithi045 ceph-mon[98627]: pgmap v432: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-09-23T09:43:14.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:13 smithi123 ceph-mon[103577]: pgmap v432: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-09-23T09:43:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:15 smithi098 ceph-mon[102386]: pgmap v433: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-09-23T09:43:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:15 smithi098 ceph-mon[102386]: osdmap e103: 12 total, 12 up, 12 in 2023-09-23T09:43:16.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:15 smithi045 ceph-mon[98627]: pgmap v433: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-09-23T09:43:16.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:15 smithi045 ceph-mon[98627]: osdmap e103: 12 total, 12 up, 12 in 2023-09-23T09:43:16.349 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:15 smithi123 ceph-mon[103577]: pgmap v433: 97 pgs: 97 active+clean; 5.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-09-23T09:43:16.349 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:15 smithi123 ceph-mon[103577]: osdmap e103: 12 total, 12 up, 12 in 2023-09-23T09:43:16.882 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 810.0 s, 868.3 tps, lat 132.688 ms stddev 226.118 2023-09-23T09:43:18.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:17 smithi098 ceph-mon[102386]: pgmap v435: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:43:18.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:17 smithi045 ceph-mon[98627]: pgmap v435: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:43:18.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:17 smithi123 ceph-mon[103577]: pgmap v435: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:43:20.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:20 smithi098 ceph-mon[102386]: pgmap v436: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:43:20.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:20 smithi045 ceph-mon[98627]: pgmap v436: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:43:20.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:20 smithi123 ceph-mon[103577]: pgmap v436: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-09-23T09:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 -- 172.21.15.45:0/3336817983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 msgr2=0x7f5578101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 --2- 172.21.15.45:0/3336817983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578101410 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5568009990 tx=0x7f5568009ca0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 -- 172.21.15.45:0/3336817983 shutdown_connections 2023-09-23T09:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 --2- 172.21.15.45:0/3336817983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5578106150 0x7f557810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 --2- 172.21.15.45:0/3336817983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578101410 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 --2- 172.21.15.45:0/3336817983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55781019e0 0x7f5578105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 -- 172.21.15.45:0/3336817983 >> 172.21.15.45:0/3336817983 conn(0x7f55780fc870 msgr2=0x7f55780fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 -- 172.21.15.45:0/3336817983 shutdown_connections 2023-09-23T09:43:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 -- 172.21.15.45:0/3336817983 wait complete. 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.467+0000 7f558005f700 1 Processor -- start 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f558005f700 1 -- start start 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f558005f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f558005f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5578106150 0x7f5578073840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f558005f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55781a30f0 0x7f55781a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f558005f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781a5a20 con 0x7f5578101030 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f558005f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781a5b60 con 0x7f5578106150 2023-09-23T09:43:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f558005f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781a5cd0 con 0x7f55781a30f0 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578073300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36748/0 (socket says 172.21.15.45:36748) 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557d5fa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5578106150 0x7f5578073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 -- 172.21.15.45:0/2423380796 learned_addr learned my addr 172.21.15.45:0/2423380796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557e5fc700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55781a30f0 0x7f55781a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 -- 172.21.15.45:0/2423380796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55781a30f0 msgr2=0x7f55781a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55781a30f0 0x7f55781a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 -- 172.21.15.45:0/2423380796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5578106150 msgr2=0x7f5578073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5578106150 0x7f5578073840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 -- 172.21.15.45:0/2423380796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574009520 con 0x7f5578101030 2023-09-23T09:43:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5578106150 0x7f5578073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.468+0000 7f557ddfb700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578073300 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5568005910 tx=0x7f5568005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.469+0000 7f556effd700 1 -- 172.21.15.45:0/2423380796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556801d070 con 0x7f5578101030 2023-09-23T09:43:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.469+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55680095f0 con 0x7f5578101030 2023-09-23T09:43:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.469+0000 7f556effd700 1 -- 172.21.15.45:0/2423380796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5568021410 con 0x7f5578101030 2023-09-23T09:43:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.469+0000 7f556effd700 1 -- 172.21.15.45:0/2423380796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568017650 con 0x7f5578101030 2023-09-23T09:43:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.469+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55781a6190 con 0x7f5578101030 2023-09-23T09:43:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.472+0000 7f556effd700 1 -- 172.21.15.45:0/2423380796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f5568021580 con 0x7f5578101030 2023-09-23T09:43:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.473+0000 7f556effd700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55640a17d0 0x7f55640a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.474+0000 7f556effd700 1 -- 172.21.15.45:0/2423380796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f55680ce5e0 con 0x7f5578101030 2023-09-23T09:43:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.474+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55640a17d0 0x7f55640a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.474+0000 7f558c249b80 1 Processor -- start 2023-09-23T09:43:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.474+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55640a17d0 0x7f55640a3c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5574009fd0 tx=0x7f5574009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f558c249b80 1 -- start start 2023-09-23T09:43:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f558c249b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b114f40 0x559e1b115320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f558c249b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b1201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f558c249b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b122b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f558c249b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b07e400 con 0x559e1b120710 2023-09-23T09:43:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f558c249b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b06fbc0 con 0x559e1b114f40 2023-09-23T09:43:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f558c249b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b06d4e0 con 0x559e1b115860 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f557d5fa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b1201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f557d5fa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b1201d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38862/0 (socket says 172.21.15.45:38862) 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.475+0000 7f557d5fa700 1 -- 172.21.15.45:0/2946571130 learned_addr learned my addr 172.21.15.45:0/2946571130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f557e5fc700 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f557d5fa700 1 -- 172.21.15.45:0/2946571130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b114f40 msgr2=0x559e1b115320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b114f40 0x559e1b115320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f557d5fa700 1 -- 172.21.15.45:0/2946571130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 msgr2=0x559e1b122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f557d5fa700 1 -- 172.21.15.45:0/2946571130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e1b0cb870 con 0x559e1b115860 2023-09-23T09:43:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b1201d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f557400e6a0 tx=0x7f5574014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=229e204cec3d226b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f5563fff700 1 -- 172.21.15.45:0/2946571130 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574026630 con 0x559e1b115860 2023-09-23T09:43:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f5563fff700 1 -- 172.21.15.45:0/2946571130 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5574021070 con 0x559e1b115860 2023-09-23T09:43:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.476+0000 7f5563fff700 1 -- 172.21.15.45:0/2946571130 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557401d760 con 0x559e1b115860 2023-09-23T09:43:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 -- 172.21.15.45:0/2946571130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 msgr2=0x559e1b1201d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b1201d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f557400e6a0 tx=0x7f5574014000 comp rx=0 tx=0).stop 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 -- 172.21.15.45:0/2946571130 shutdown_connections 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b114f40 0x559e1b115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 --2- 172.21.15.45:0/2946571130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b1201d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 -- 172.21.15.45:0/2946571130 >> 172.21.15.45:0/2946571130 conn(0x559e1b08a200 msgr2=0x559e1b1113c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 -- 172.21.15.45:0/2946571130 shutdown_connections 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.477+0000 7f558c249b80 1 -- 172.21.15.45:0/2946571130 wait complete. 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 -- start start 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 Processor -- start 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b114f40 0x559e1b120180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b115860 0x559e1b11d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b11daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b068dd0 con 0x559e1b120710 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b0056b0 con 0x559e1b115860 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b005350 con 0x559e1b114f40 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f558c249b80 10 client.0 _ll_register_callbacks cb 0x559e1ae627a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f557e5fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b11daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f557e5fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b11daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36764/0 (socket says 172.21.15.45:36764) 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f557d5fa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b115860 0x559e1b11d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f557e5fc700 1 -- 172.21.15.45:0/1069833233 learned_addr learned my addr 172.21.15.45:0/1069833233 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.478+0000 7f557ddfb700 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b114f40 0x559e1b120180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f557d5fa700 1 -- 172.21.15.45:0/1069833233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b114f40 msgr2=0x559e1b120180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f557d5fa700 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b114f40 0x559e1b120180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f557d5fa700 1 -- 172.21.15.45:0/1069833233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 msgr2=0x559e1b11daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f557d5fa700 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b11daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f557d5fa700 1 -- 172.21.15.45:0/1069833233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5574017050 con 0x559e1b115860 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f557e5fc700 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b11daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f557d5fa700 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b115860 0x559e1b11d400 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f557400efd0 tx=0x7f557401ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adda75ad8b608c2c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f55617fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f55617fa700 1 -- 172.21.15.45:0/1069833233 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574005340 con 0x559e1b115860 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e1b109090 con 0x559e1b115860 2023-09-23T09:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f55617fa700 1 -- 172.21.15.45:0/1069833233 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f55740054a0 con 0x559e1b115860 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f55617fa700 1 -- 172.21.15.45:0/1069833233 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5574005660 con 0x559e1b115860 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f558c249b80 10 client.26648 fetch_fsmap learned FSMap version 54 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.479+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x559e1ae93d80 con 0x559e1b115860 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f55617fa700 1 -- 172.21.15.45:0/1069833233 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e1b109090 con 0x559e1b115860 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f55617fa700 1 -- 172.21.15.45:0/1069833233 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 54) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f557401c800 con 0x559e1b115860 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f558c249b80 10 client.26648 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f558c249b80 10 client.26648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f558c249b80 20 client.26648 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f558c249b80 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b1302c0 0x559e1b1326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f558c249b80 4 client.26648 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:43:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e1b061db0 con 0x559e1b1302c0 2023-09-23T09:43:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f557ddfb700 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b1302c0 0x559e1b1326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f55617fa700 10 client.26648 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.480+0000 7f557ddfb700 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b1302c0 0x559e1b1326e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5033adffa126b9d server_cookie=2c0d0766668e479d in_seq=0 out_seq=0 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.481+0000 7f55617fa700 1 -- 172.21.15.45:0/1069833233 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559e1b061db0 con 0x559e1b1302c0 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.481+0000 7f55617fa700 10 client.26648 handle_command_reply: tid=0 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.481+0000 7f558c249b80 1 client.26648 shutdown 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b115860 msgr2=0x559e1b11d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b115860 0x559e1b11d400 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f557400efd0 tx=0x7f557401ce20 comp rx=0 tx=0).stop 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 shutdown_connections 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b1302c0 0x559e1b1326e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b120710 0x559e1b11daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b114f40 0x559e1b120180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 --2- 172.21.15.45:0/1069833233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b115860 0x559e1b11d400 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 >> 172.21.15.45:0/1069833233 conn(0x559e1b08a200 msgr2=0x559e1b110f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.482+0000 7f55617fa700 0 client.26648 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.483+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 shutdown_connections 2023-09-23T09:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.483+0000 7f558c249b80 1 -- 172.21.15.45:0/1069833233 wait complete. 2023-09-23T09:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.483+0000 7f558c249b80 20 client.26648 trim_cache size 0 max 16384 2023-09-23T09:43:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 Processor -- start 2023-09-23T09:43:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 -- start start 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b117de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b127260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b11a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1aee98c0 con 0x559e1b114f40 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b112a30 con 0x559e1b120710 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f558c249b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b10dba0 con 0x559e1b115860 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.503+0000 7f557d5fa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b127260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.504+0000 7f557d5fa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b127260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38874/0 (socket says 172.21.15.45:38874) 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.504+0000 7f557d5fa700 1 -- 172.21.15.45:0/1917809992 learned_addr learned my addr 172.21.15.45:0/1917809992 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.504+0000 7f557e5fc700 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b11a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557ddfb700 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b117de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557d5fa700 1 -- 172.21.15.45:0/1917809992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 msgr2=0x559e1b11a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557d5fa700 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b11a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557d5fa700 1 -- 172.21.15.45:0/1917809992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 msgr2=0x559e1b117de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557d5fa700 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b117de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557d5fa700 1 -- 172.21.15.45:0/1917809992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e1ae7cc00 con 0x559e1b115860 2023-09-23T09:43:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557ddfb700 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b117de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:43:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f557d5fa700 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b127260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f557401cbe0 tx=0x7f5574009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb16115a5f66b8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.505+0000 7f556cff9700 1 -- 172.21.15.45:0/1917809992 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557400ed40 con 0x559e1b115860 2023-09-23T09:43:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.506+0000 7f556cff9700 1 -- 172.21.15.45:0/1917809992 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55740050c0 con 0x559e1b115860 2023-09-23T09:43:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.506+0000 7f556cff9700 1 -- 172.21.15.45:0/1917809992 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55740269f0 con 0x559e1b115860 2023-09-23T09:43:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 -- 172.21.15.45:0/1917809992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 msgr2=0x559e1b127260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b127260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f557401cbe0 tx=0x7f5574009350 comp rx=0 tx=0).stop 2023-09-23T09:43:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 -- 172.21.15.45:0/1917809992 shutdown_connections 2023-09-23T09:43:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b11a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b117de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 --2- 172.21.15.45:0/1917809992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b127260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 -- 172.21.15.45:0/1917809992 >> 172.21.15.45:0/1917809992 conn(0x559e1b08a200 msgr2=0x559e1b118770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 -- 172.21.15.45:0/1917809992 shutdown_connections 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 -- 172.21.15.45:0/1917809992 wait complete. 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 -- start start 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.507+0000 7f558c249b80 1 Processor -- start 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f558c249b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b1269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f558c249b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b126f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f558c249b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b17b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f558c249b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b17bcd0 con 0x559e1b114f40 2023-09-23T09:43:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f558c249b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b17be40 con 0x559e1b120710 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f558c249b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1b17c000 con 0x559e1b115860 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f558c249b80 10 client.0 _ll_register_callbacks cb 0x7f5574011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557ddfb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b1269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557d5fa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b126f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557e5fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b17b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557ddfb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b1269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36774/0 (socket says 172.21.15.45:36774) 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557ddfb700 1 -- 172.21.15.45:0/2916198773 learned_addr learned my addr 172.21.15.45:0/2916198773 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557d5fa700 1 -- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 msgr2=0x559e1b17b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b17b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557d5fa700 1 -- 172.21.15.45:0/2916198773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 msgr2=0x559e1b1269c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b1269c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557d5fa700 1 -- 172.21.15.45:0/2916198773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e1ae93d80 con 0x559e1b115860 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557e5fc700 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b17b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.508+0000 7f557ddfb700 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b1269c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f557d5fa700 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b126f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f557401cb60 tx=0x7f5574026660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=369a5b494555bb0e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f55617fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f55617fa700 1 -- 172.21.15.45:0/2916198773 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574005550 con 0x559e1b115860 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5574021070 con 0x559e1b115860 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f55617fa700 1 -- 172.21.15.45:0/2916198773 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f557401c3f0 con 0x559e1b115860 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f55617fa700 1 -- 172.21.15.45:0/2916198773 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55740056b0 con 0x559e1b115860 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 10 client.26656 fetch_fsmap learned FSMap version 54 2023-09-23T09:43:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x559e1aeba1f0 con 0x559e1b115860 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f55617fa700 1 -- 172.21.15.45:0/2916198773 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5574021070 con 0x559e1b115860 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f55617fa700 1 -- 172.21.15.45:0/2916198773 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 54) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f55740046d0 con 0x559e1b115860 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 10 client.26656 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 10 client.26656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 20 client.26656 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b184530 0x559e1b186920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 4 client.26656 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.509+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e1b061ad0 con 0x559e1b184530 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.510+0000 7f557ddfb700 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b184530 0x559e1b186920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.510+0000 7f55617fa700 10 client.26656 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.510+0000 7f557ddfb700 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b184530 0x559e1b186920 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de253986c3362a61 server_cookie=944a4c6162925c56 in_seq=0 out_seq=0 2023-09-23T09:43:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.527+0000 7f55617fa700 1 -- 172.21.15.45:0/2916198773 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e1b061ad0 con 0x559e1b184530 2023-09-23T09:43:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f55617fa700 10 client.26656 handle_command_reply: tid=0 2023-09-23T09:43:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 client.26656 shutdown 2023-09-23T09:43:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 msgr2=0x559e1b126f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b126f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f557401cb60 tx=0x7f5574026660 comp rx=0 tx=0).stop 2023-09-23T09:43:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 shutdown_connections 2023-09-23T09:43:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x559e1b184530 0x559e1b186920 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e1b120710 0x559e1b17b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f55617fa700 0 client.26656 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e1b114f40 0x559e1b1269c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 --2- 172.21.15.45:0/2916198773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e1b115860 0x559e1b126f00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.528+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 >> 172.21.15.45:0/2916198773 conn(0x559e1b08a200 msgr2=0x559e1b11ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.529+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 shutdown_connections 2023-09-23T09:43:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.530+0000 7f558c249b80 1 -- 172.21.15.45:0/2916198773 wait complete. 2023-09-23T09:43:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558c249b80 20 client.26656 trim_cache size 0 max 16384 2023-09-23T09:43:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:43:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:43:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:43:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:43:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55640a17d0 msgr2=0x7f55640a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55640a17d0 0x7f55640a3c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5574009fd0 tx=0x7f5574009260 comp rx=0 tx=0).stop 2023-09-23T09:43:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 msgr2=0x7f5578073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578073300 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5568005910 tx=0x7f5568005740 comp rx=0 tx=0).stop 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 shutdown_connections 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55781a30f0 0x7f55781a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f55640a17d0 0x7f55640a3c90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578101030 0x7f5578073300 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 --2- 172.21.15.45:0/2423380796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5578106150 0x7f5578073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.531+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 >> 172.21.15.45:0/2423380796 conn(0x7f55780fc870 msgr2=0x7f55780fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.532+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 shutdown_connections 2023-09-23T09:43:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:20.532+0000 7f558005f700 1 -- 172.21.15.45:0/2423380796 wait complete. 2023-09-23T09:43:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 815.0 s, 1318.6 tps, lat 75.338 ms stddev 83.235 2023-09-23T09:43:22.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:22 smithi098 ceph-mon[102386]: pgmap v437: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-23T09:43:22.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:22 smithi045 ceph-mon[98627]: pgmap v437: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-23T09:43:22.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:22 smithi123 ceph-mon[103577]: pgmap v437: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-23T09:43:24.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:24 smithi098 ceph-mon[102386]: pgmap v438: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-23T09:43:24.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:24 smithi045 ceph-mon[98627]: pgmap v438: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-23T09:43:24.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:24 smithi123 ceph-mon[103577]: pgmap v438: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-23T09:43:25.553 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:25 smithi098 ceph-mon[102386]: pgmap v439: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-23T09:43:25.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:25 smithi045 ceph-mon[98627]: pgmap v439: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-23T09:43:25.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:25 smithi123 ceph-mon[103577]: pgmap v439: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-23T09:43:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 820.0 s, 1310.2 tps, lat 77.196 ms stddev 99.258 2023-09-23T09:43:28.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:27 smithi045 ceph-mon[98627]: pgmap v440: 97 pgs: 97 active+clean; 5.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-09-23T09:43:28.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:27 smithi123 ceph-mon[103577]: pgmap v440: 97 pgs: 97 active+clean; 5.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-09-23T09:43:28.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:27 smithi098 ceph-mon[102386]: pgmap v440: 97 pgs: 97 active+clean; 5.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-09-23T09:43:29.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:28 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:29.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:28 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:29.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:28 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:43:30.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:29 smithi045 ceph-mon[98627]: pgmap v441: 97 pgs: 97 active+clean; 5.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-09-23T09:43:30.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:29 smithi123 ceph-mon[103577]: pgmap v441: 97 pgs: 97 active+clean; 5.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-09-23T09:43:30.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:29 smithi098 ceph-mon[102386]: pgmap v441: 97 pgs: 97 active+clean; 5.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-09-23T09:43:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 825.0 s, 1360.6 tps, lat 73.613 ms stddev 79.008 2023-09-23T09:43:32.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:31 smithi045 ceph-mon[98627]: pgmap v442: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2023-09-23T09:43:32.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:31 smithi123 ceph-mon[103577]: pgmap v442: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2023-09-23T09:43:32.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:31 smithi098 ceph-mon[102386]: pgmap v442: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2023-09-23T09:43:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.686+0000 7ffb05445700 1 -- 172.21.15.45:0/3781007027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 msgr2=0x7ffb00100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.686+0000 7ffb05445700 1 --2- 172.21.15.45:0/3781007027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb00100540 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00098f0 tx=0x7ffaf0009c00 comp rx=0 tx=0).stop 2023-09-23T09:43:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.686+0000 7ffb05445700 1 -- 172.21.15.45:0/3781007027 shutdown_connections 2023-09-23T09:43:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.686+0000 7ffb05445700 1 --2- 172.21.15.45:0/3781007027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb00105050 0x7ffb0010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.686+0000 7ffb05445700 1 --2- 172.21.15.45:0/3781007027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb00100540 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.686+0000 7ffb05445700 1 --2- 172.21.15.45:0/3781007027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb00100b10 0x7ffb00104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.686+0000 7ffb05445700 1 -- 172.21.15.45:0/3781007027 >> 172.21.15.45:0/3781007027 conn(0x7ffb00076840 msgr2=0x7ffb00076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.687+0000 7ffb05445700 1 -- 172.21.15.45:0/3781007027 shutdown_connections 2023-09-23T09:43:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.687+0000 7ffb05445700 1 -- 172.21.15.45:0/3781007027 wait complete. 2023-09-23T09:43:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 Processor -- start 2023-09-23T09:43:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 -- start start 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb000723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb00105050 0x7ffb00072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0006b1d0 0x7ffb0006b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0006be50 con 0x7ffb00105050 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0006bfc0 con 0x7ffb00100160 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffb05445700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0006ff60 con 0x7ffb0006b1d0 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffafe7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb00105050 0x7ffb00072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb000723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffaff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0006b1d0 0x7ffb0006b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb000723d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58138/0 (socket says 172.21.15.45:58138) 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.688+0000 7ffafeffd700 1 -- 172.21.15.45:0/696617420 learned_addr learned my addr 172.21.15.45:0/696617420 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffafeffd700 1 -- 172.21.15.45:0/696617420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0006b1d0 msgr2=0x7ffb0006b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffafeffd700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0006b1d0 0x7ffb0006b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffafeffd700 1 -- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb00105050 msgr2=0x7ffb00072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffafeffd700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb00105050 0x7ffb00072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffafeffd700 1 -- 172.21.15.45:0/696617420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40095f0 con 0x7ffb00100160 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0006b1d0 0x7ffb0006b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffafeffd700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb000723d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009230 tx=0x7ffaf000fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffae7fff700 1 -- 172.21.15.45:0/696617420 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf001c070 con 0x7ffb00100160 2023-09-23T09:43:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf0009520 con 0x7ffb00100160 2023-09-23T09:43:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffae7fff700 1 -- 172.21.15.45:0/696617420 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf00058e0 con 0x7ffb00100160 2023-09-23T09:43:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffae7fff700 1 -- 172.21.15.45:0/696617420 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00038e0 con 0x7ffb00100160 2023-09-23T09:43:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.689+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb00070440 con 0x7ffb00100160 2023-09-23T09:43:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.692+0000 7ffae7fff700 1 -- 172.21.15.45:0/696617420 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7ffaf0020030 con 0x7ffb00100160 2023-09-23T09:43:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.695+0000 7ffae7fff700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ffae80a1710 0x7ffae80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.695+0000 7ffae7fff700 1 -- 172.21.15.45:0/696617420 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ffaf00ce140 con 0x7ffb00100160 2023-09-23T09:43:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.695+0000 7ffb1162fb80 1 Processor -- start 2023-09-23T09:43:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.695+0000 7ffb1162fb80 1 -- start start 2023-09-23T09:43:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f8f40 0x562b8a5f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f9860 0x562b8a6063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a608d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a562400 con 0x562b8a606920 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a553bc0 con 0x562b8a5f8f40 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a5514e0 con 0x562b8a5f9860 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffaff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a608d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffaff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a608d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36798/0 (socket says 172.21.15.45:36798) 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3486893183 learned_addr learned my addr 172.21.15.45:0/3486893183 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffafe7fc700 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f9860 0x562b8a6063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffafeffd700 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f8f40 0x562b8a5f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3486893183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f9860 msgr2=0x562b8a6063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f9860 0x562b8a6063e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.696+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3486893183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f8f40 msgr2=0x562b8a5f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.697+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f8f40 0x562b8a5f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.697+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3486893183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b8a5af870 con 0x562b8a606920 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.697+0000 7ffafe7fc700 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f9860 0x562b8a6063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.697+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a608d10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8003eb0 tx=0x7ffaf8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeac878cc73e92e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.697+0000 7ffae57fa700 1 -- 172.21.15.45:0/3486893183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80092d0 con 0x562b8a606920 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.697+0000 7ffae57fa700 1 -- 172.21.15.45:0/3486893183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf800f040 con 0x562b8a606920 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.697+0000 7ffae57fa700 1 -- 172.21.15.45:0/3486893183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800e8b0 con 0x562b8a606920 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.698+0000 7ffafe7fc700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ffae80a1710 0x7ffae80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.698+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3486893183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 msgr2=0x562b8a608d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.698+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a608d10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8003eb0 tx=0x7ffaf8009920 comp rx=0 tx=0).stop 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.698+0000 7ffafe7fc700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ffae80a1710 0x7ffae80a3bd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009fd0 tx=0x7ffaf40047c0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3486893183 shutdown_connections 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a608d10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f8f40 0x562b8a5f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3486893183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f9860 0x562b8a6063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3486893183 >> 172.21.15.45:0/3486893183 conn(0x562b8a56e200 msgr2=0x562b8a5f53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3486893183 shutdown_connections 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3486893183 wait complete. 2023-09-23T09:43:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 -- start start 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.699+0000 7ffb1162fb80 1 Processor -- start 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a603b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a6040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a60fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a54cdd0 con 0x562b8a606920 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a4e96b0 con 0x562b8a5f9860 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a4e9350 con 0x562b8a5f8f40 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffb1162fb80 10 client.0 _ll_register_callbacks cb 0x562b8a3467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a60fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffafe7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a6040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a60fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36806/0 (socket says 172.21.15.45:36806) 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3674650983 learned_addr learned my addr 172.21.15.45:0/3674650983 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffafeffd700 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a603b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3674650983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 msgr2=0x562b8a603b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a603b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 msgr2=0x562b8a6040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a6040c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 -- 172.21.15.45:0/3674650983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf4009c80 con 0x562b8a606920 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffafe7fc700 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a6040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a60fd60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000f80 tx=0x7ffaf8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75d9a05c6e3158c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.700+0000 7ffae2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:43:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.701+0000 7ffae2ff5700 1 -- 172.21.15.45:0/3674650983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8007910 con 0x562b8a606920 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.701+0000 7ffae2ff5700 1 -- 172.21.15.45:0/3674650983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ffaf800e4f0 con 0x562b8a606920 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.701+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b8a5ed090 con 0x562b8a606920 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.701+0000 7ffae2ff5700 1 -- 172.21.15.45:0/3674650983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaf8007cc0 con 0x562b8a606920 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.701+0000 7ffb1162fb80 10 client.17013 fetch_fsmap learned FSMap version 55 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.701+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7ffaf8003b90 con 0x562b8a606920 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.701+0000 7ffae2ff5700 1 -- 172.21.15.45:0/3674650983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf8034080 con 0x562b8a606920 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffae2ff5700 1 -- 172.21.15.45:0/3674650983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffaf80155c0 con 0x562b8a606920 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffb1162fb80 10 client.17013 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffb1162fb80 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffb1162fb80 20 client.17013 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a614100 0x562b8a616520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffb1162fb80 4 client.17013 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:43:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b8a545db0 con 0x562b8a614100 2023-09-23T09:43:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffafeffd700 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a614100 0x562b8a616520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffae2ff5700 10 client.17013 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.702+0000 7ffafeffd700 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a614100 0x562b8a616520 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a0f7b67d4a592b server_cookie=ca8e215bc82ad735 in_seq=0 out_seq=0 2023-09-23T09:43:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffae2ff5700 1 -- 172.21.15.45:0/3674650983 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562b8a545db0 con 0x562b8a614100 2023-09-23T09:43:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffae2ff5700 10 client.17013 handle_command_reply: tid=0 2023-09-23T09:43:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 client.17013 shutdown 2023-09-23T09:43:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 msgr2=0x562b8a60fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a60fd60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000f80 tx=0x7ffaf8004d00 comp rx=0 tx=0).stop 2023-09-23T09:43:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 shutdown_connections 2023-09-23T09:43:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a614100 0x562b8a616520 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a606920 0x562b8a60fd60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a603b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3674650983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a6040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 >> 172.21.15.45:0/3674650983 conn(0x562b8a56e200 msgr2=0x562b8a5f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffae2ff5700 0 client.17013 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.703+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 shutdown_connections 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.704+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3674650983 wait complete. 2023-09-23T09:43:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.704+0000 7ffb1162fb80 20 client.17013 trim_cache size 0 max 16384 2023-09-23T09:43:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.728+0000 7ffb1162fb80 1 Processor -- start 2023-09-23T09:43:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffb1162fb80 1 -- start start 2023-09-23T09:43:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f8f40 0x562b8a60fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a60ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a606920 0x562b8a600e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a610b90 con 0x562b8a5f8f40 2023-09-23T09:43:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a3cd8c0 con 0x562b8a5f9860 2023-09-23T09:43:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a606710 con 0x562b8a606920 2023-09-23T09:43:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f8f40 0x562b8a60fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f8f40 0x562b8a60fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36810/0 (socket says 172.21.15.45:36810) 2023-09-23T09:43:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 -- 172.21.15.45:0/3500249606 learned_addr learned my addr 172.21.15.45:0/3500249606 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafe7fc700 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a60ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a606920 0x562b8a600e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 -- 172.21.15.45:0/3500249606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a606920 msgr2=0x562b8a600e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a606920 0x562b8a600e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 -- 172.21.15.45:0/3500249606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 msgr2=0x562b8a60ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a60ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 -- 172.21.15.45:0/3500249606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b8a61b150 con 0x562b8a5f8f40 2023-09-23T09:43:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a606920 0x562b8a600e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:43:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.729+0000 7ffafeffd700 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f8f40 0x562b8a60fb90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0097600 tx=0x7ffaf009ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a861e4c74460b06e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.730+0000 7ffae5ffb700 1 -- 172.21.15.45:0/3500249606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf009b800 con 0x562b8a5f8f40 2023-09-23T09:43:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.730+0000 7ffae5ffb700 1 -- 172.21.15.45:0/3500249606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf009b100 con 0x562b8a5f8f40 2023-09-23T09:43:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.730+0000 7ffae5ffb700 1 -- 172.21.15.45:0/3500249606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00cbad0 con 0x562b8a5f8f40 2023-09-23T09:43:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3500249606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f8f40 msgr2=0x562b8a60fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f8f40 0x562b8a60fb90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0097600 tx=0x7ffaf009ff90 comp rx=0 tx=0).stop 2023-09-23T09:43:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3500249606 shutdown_connections 2023-09-23T09:43:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a606920 0x562b8a600e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f8f40 0x562b8a60fb90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/3500249606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a5f9860 0x562b8a60ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3500249606 >> 172.21.15.45:0/3500249606 conn(0x562b8a56e200 msgr2=0x562b8a60f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.731+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3500249606 shutdown_connections 2023-09-23T09:43:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.733+0000 7ffb1162fb80 1 -- 172.21.15.45:0/3500249606 wait complete. 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.733+0000 7ffb1162fb80 1 -- start start 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.733+0000 7ffb1162fb80 1 Processor -- start 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a669210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f9860 0x562b8a669750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffb1162fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a606920 0x562b8a60a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a60a920 con 0x562b8a5f9860 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a60aa90 con 0x562b8a606920 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffb1162fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8a60ac50 con 0x562b8a5f8f40 2023-09-23T09:43:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffb1162fb80 10 client.0 _ll_register_callbacks cb 0x7ffaf4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a669210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a669210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38924/0 (socket says 172.21.15.45:38924) 2023-09-23T09:43:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.734+0000 7ffafeffd700 1 -- 172.21.15.45:0/4279955452 learned_addr learned my addr 172.21.15.45:0/4279955452 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:33.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.750+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a606920 0x562b8a60a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.756+0000 7ffaff7fe700 1 -- 172.21.15.45:0/4279955452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 msgr2=0x562b8a669210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.756+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a669210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.756+0000 7ffaff7fe700 1 -- 172.21.15.45:0/4279955452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f9860 msgr2=0x562b8a669750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.756+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f9860 0x562b8a669750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.756+0000 7ffaff7fe700 1 -- 172.21.15.45:0/4279955452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb00070440 con 0x562b8a606920 2023-09-23T09:43:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.771+0000 7ffaff7fe700 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a606920 0x562b8a60a340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8003b60 tx=0x7ffaf8015b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=393894217e28b42d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.777+0000 7ffae2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:43:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.777+0000 7ffae2ff5700 1 -- 172.21.15.45:0/4279955452 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8015650 con 0x562b8a606920 2023-09-23T09:43:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.777+0000 7ffae2ff5700 1 -- 172.21.15.45:0/4279955452 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ffaf800e4f0 con 0x562b8a606920 2023-09-23T09:43:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.779+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffaf800f040 con 0x562b8a606920 2023-09-23T09:43:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.787+0000 7ffae2ff5700 1 -- 172.21.15.45:0/4279955452 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaf800e6d0 con 0x562b8a606920 2023-09-23T09:43:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.788+0000 7ffb1162fb80 10 client.26672 fetch_fsmap learned FSMap version 55 2023-09-23T09:43:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.788+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x562b8a377d80 con 0x562b8a606920 2023-09-23T09:43:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffae2ff5700 1 -- 172.21.15.45:0/4279955452 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf800f040 con 0x562b8a606920 2023-09-23T09:43:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffae2ff5700 1 -- 172.21.15.45:0/4279955452 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffaf8028530 con 0x562b8a606920 2023-09-23T09:43:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffb1162fb80 10 client.26672 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:43:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffb1162fb80 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffb1162fb80 20 client.26672 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a65cad0 0x562b8a65eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffb1162fb80 4 client.26672 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:43:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.803+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b8a545ad0 con 0x562b8a65cad0 2023-09-23T09:43:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.818+0000 7ffafeffd700 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a65cad0 0x562b8a65eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.848+0000 7ffafeffd700 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a65cad0 0x562b8a65eef0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90123666f71cce19 server_cookie=d5efe2899847442e in_seq=0 out_seq=0 2023-09-23T09:43:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.849+0000 7ffae2ff5700 10 client.26672 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.924+0000 7ffae2ff5700 1 -- 172.21.15.45:0/4279955452 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b8a545ad0 con 0x562b8a65cad0 2023-09-23T09:43:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.924+0000 7ffae2ff5700 10 client.26672 handle_command_reply: tid=0 2023-09-23T09:43:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.924+0000 7ffb1162fb80 1 client.26672 shutdown 2023-09-23T09:43:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.924+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a606920 msgr2=0x562b8a60a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.924+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a606920 0x562b8a60a340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8003b60 tx=0x7ffaf8015b40 comp rx=0 tx=0).stop 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 shutdown_connections 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x562b8a65cad0 0x562b8a65eef0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b8a606920 0x562b8a60a340 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b8a5f8f40 0x562b8a669210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 --2- 172.21.15.45:0/4279955452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b8a5f9860 0x562b8a669750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 >> 172.21.15.45:0/4279955452 conn(0x562b8a56e200 msgr2=0x562b8a605420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffae2ff5700 0 client.26672 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 shutdown_connections 2023-09-23T09:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 1 -- 172.21.15.45:0/4279955452 wait complete. 2023-09-23T09:43:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.927+0000 7ffb1162fb80 20 client.26672 trim_cache size 0 max 16384 2023-09-23T09:43:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:43:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:43:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:43:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:43:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ffae80a1710 msgr2=0x7ffae80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ffae80a1710 0x7ffae80a3bd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009fd0 tx=0x7ffaf40047c0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 msgr2=0x7ffb000723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb000723d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009230 tx=0x7ffaf000fa80 comp rx=0 tx=0).stop 2023-09-23T09:43:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 shutdown_connections 2023-09-23T09:43:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0006b1d0 0x7ffb0006b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb00100160 0x7ffb000723d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ffae80a1710 0x7ffae80a3bd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 --2- 172.21.15.45:0/696617420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb00105050 0x7ffb00072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 >> 172.21.15.45:0/696617420 conn(0x7ffb00076840 msgr2=0x7ffb000fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 shutdown_connections 2023-09-23T09:43:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:33.928+0000 7ffb05445700 1 -- 172.21.15.45:0/696617420 wait complete. 2023-09-23T09:43:34.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:33 smithi098 ceph-mon[102386]: pgmap v443: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-09-23T09:43:34.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:33 smithi045 ceph-mon[98627]: pgmap v443: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-09-23T09:43:34.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:33 smithi123 ceph-mon[103577]: pgmap v443: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-09-23T09:43:36.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:35 smithi098 ceph-mon[102386]: pgmap v444: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2023-09-23T09:43:36.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:35 smithi045 ceph-mon[98627]: pgmap v444: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2023-09-23T09:43:36.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:35 smithi123 ceph-mon[103577]: pgmap v444: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2023-09-23T09:43:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 830.0 s, 1034.9 tps, lat 94.722 ms stddev 113.561 2023-09-23T09:43:38.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:37 smithi098 ceph-mon[102386]: pgmap v445: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-23T09:43:38.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:37 smithi045 ceph-mon[98627]: pgmap v445: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-23T09:43:38.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:37 smithi123 ceph-mon[103577]: pgmap v445: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-23T09:43:39.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:43:39.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 -- 172.21.15.45:0/2231045853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8105e30 msgr2=0x7fbef8106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 --2- 172.21.15.45:0/2231045853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8105e30 0x7fbef8106210 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbee80098f0 tx=0x7fbee8009c00 comp rx=0 tx=0).stop 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 -- 172.21.15.45:0/2231045853 shutdown_connections 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 --2- 172.21.15.45:0/2231045853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8101100 0x7fbef80ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 --2- 172.21.15.45:0/2231045853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8105e30 0x7fbef8106210 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 --2- 172.21.15.45:0/2231045853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8100740 0x7fbef8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 -- 172.21.15.45:0/2231045853 >> 172.21.15.45:0/2231045853 conn(0x7fbef8076e10 msgr2=0x7fbef80fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 -- 172.21.15.45:0/2231045853 shutdown_connections 2023-09-23T09:43:39.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.843+0000 7fbf00566700 1 -- 172.21.15.45:0/2231045853 wait complete. 2023-09-23T09:43:39.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 Processor -- start 2023-09-23T09:43:39.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 -- start start 2023-09-23T09:43:39.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8100740 0x7fbef8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8101100 0x7fbef8199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8105e30 0x7fbef819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef819e230 con 0x7fbef8101100 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef819e3a0 con 0x7fbef8105e30 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbf00566700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef819e560 con 0x7fbef8100740 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbefe302700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8100740 0x7fbef8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8101100 0x7fbef8199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbefeb03700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8105e30 0x7fbef819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbefe302700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8100740 0x7fbef8199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38936/0 (socket says 172.21.15.45:38936) 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8101100 0x7fbef8199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36832/0 (socket says 172.21.15.45:36832) 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbefe302700 1 -- 172.21.15.45:0/3246174170 learned_addr learned my addr 172.21.15.45:0/3246174170 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.844+0000 7fbefeb03700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8105e30 0x7fbef819dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58184/0 (socket says 172.21.15.45:58184) 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefe302700 1 -- 172.21.15.45:0/3246174170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8105e30 msgr2=0x7fbef819dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefe302700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8105e30 0x7fbef819dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefe302700 1 -- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8101100 msgr2=0x7fbef8199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefe302700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8101100 0x7fbef8199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefe302700 1 -- 172.21.15.45:0/3246174170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40095f0 con 0x7fbef8100740 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefeb03700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8105e30 0x7fbef819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefdb01700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8101100 0x7fbef8199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbefe302700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8100740 0x7fbef8199250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009810 tx=0x7fbee800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbeef7fe700 1 -- 172.21.15.45:0/3246174170 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee801c070 con 0x7fbef8100740 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee8009520 con 0x7fbef8100740 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbeef7fe700 1 -- 172.21.15.45:0/3246174170 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbee800bbe0 con 0x7fbef8100740 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbeef7fe700 1 -- 172.21.15.45:0/3246174170 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004930 con 0x7fbef8100740 2023-09-23T09:43:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.845+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef819ea70 con 0x7fbef8100740 2023-09-23T09:43:39.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.847+0000 7fbeef7fe700 1 -- 172.21.15.45:0/3246174170 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fbee8020070 con 0x7fbef8100740 2023-09-23T09:43:39.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.848+0000 7fbeef7fe700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbee40a1750 0x7fbee40a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.848+0000 7fbeef7fe700 1 -- 172.21.15.45:0/3246174170 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbee80ce100 con 0x7fbef8100740 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.848+0000 7fbefdb01700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbee40a1750 0x7fbee40a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 Processor -- start 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefdb01700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbee40a1750 0x7fbee40a3c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbef819a860 tx=0x7fbef4009310 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 -- start start 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c1d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c1dbd0 0x558849c1ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849b7b460 con 0x558849c1dbd0 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849b6cc20 con 0x558849c12400 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849b6a540 con 0x558849c12d20 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefe302700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c1d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefeb03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c1dbd0 0x558849c1ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c1d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38946/0 (socket says 172.21.15.45:38946) 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefe302700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58188/0 (socket says 172.21.15.45:58188) 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefeb03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c1dbd0 0x558849c1ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36842/0 (socket says 172.21.15.45:36842) 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.849+0000 7fbefdb01700 1 -- 172.21.15.45:0/834275813 learned_addr learned my addr 172.21.15.45:0/834275813 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefdb01700 1 -- 172.21.15.45:0/834275813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 msgr2=0x558849c127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefdb01700 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefdb01700 1 -- 172.21.15.45:0/834275813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c1dbd0 msgr2=0x558849c1ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefdb01700 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c1dbd0 0x558849c1ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefdb01700 1 -- 172.21.15.45:0/834275813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558849bc88d0 con 0x558849c12d20 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefe302700 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefeb03700 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c1dbd0 0x558849c1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbefdb01700 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c1d690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbef4004360 tx=0x7fbef4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc60dd38c5379005 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbeecff9700 1 -- 172.21.15.45:0/834275813 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4003d10 con 0x558849c12d20 2023-09-23T09:43:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbeecff9700 1 -- 172.21.15.45:0/834275813 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef4003e70 con 0x558849c12d20 2023-09-23T09:43:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.850+0000 7fbeecff9700 1 -- 172.21.15.45:0/834275813 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef401e8a0 con 0x558849c12d20 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 -- 172.21.15.45:0/834275813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 msgr2=0x558849c1d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c1d690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbef4004360 tx=0x7fbef4014000 comp rx=0 tx=0).stop 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 -- 172.21.15.45:0/834275813 shutdown_connections 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c1dbd0 0x558849c1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/834275813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c1d690 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 -- 172.21.15.45:0/834275813 >> 172.21.15.45:0/834275813 conn(0x558849b87260 msgr2=0x558849c0e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 -- 172.21.15.45:0/834275813 shutdown_connections 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 -- 172.21.15.45:0/834275813 wait complete. 2023-09-23T09:43:39.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.851+0000 7fbf0c750b80 1 -- start start 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 1 Processor -- start 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c1a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c1b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849b65e30 con 0x558849c12d20 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849b02710 con 0x558849c12400 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849b023b0 con 0x558849c1dbd0 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbf0c750b80 10 client.0 _ll_register_callbacks cb 0x55884995f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefeb03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c1b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36846/0 (socket says 172.21.15.45:36846) 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefdb01700 1 -- 172.21.15.45:0/4086476230 learned_addr learned my addr 172.21.15.45:0/4086476230 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefe302700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c1a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefeb03700 1 -- 172.21.15.45:0/4086476230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 msgr2=0x558849c1a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefeb03700 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c1a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefeb03700 1 -- 172.21.15.45:0/4086476230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 msgr2=0x558849c1ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefeb03700 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefeb03700 1 -- 172.21.15.45:0/4086476230 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef0003a60 con 0x558849c1dbd0 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.852+0000 7fbefdb01700 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:39.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbefeb03700 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c1b3b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbef0003db0 tx=0x7fbef000e7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a565705f37d5cf4e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbee1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbee1ffb700 1 -- 172.21.15.45:0/4086476230 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0004db0 con 0x558849c1dbd0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558849c06250 con 0x558849c1dbd0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbee1ffb700 1 -- 172.21.15.45:0/4086476230 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbef00090d0 con 0x558849c1dbd0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbee1ffb700 1 -- 172.21.15.45:0/4086476230 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef000eec0 con 0x558849c1dbd0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 10 client.26686 fetch_fsmap learned FSMap version 55 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fbef4017070 con 0x558849c1dbd0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbee1ffb700 1 -- 172.21.15.45:0/4086476230 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558849c06250 con 0x558849c1dbd0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbee1ffb700 1 -- 172.21.15.45:0/4086476230 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbef0024070 con 0x558849c1dbd0 2023-09-23T09:43:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 10 client.26686 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 20 client.26686 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c2d860 0x558849c2fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 4 client.26686 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.853+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558849b5ee10 con 0x558849c2d860 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.854+0000 7fbefe302700 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c2d860 0x558849c2fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.854+0000 7fbefe302700 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c2d860 0x558849c2fc80 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939c75dbbb8c088d server_cookie=ca13308b3b6688f2 in_seq=0 out_seq=0 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.854+0000 7fbee1ffb700 10 client.26686 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbee1ffb700 1 -- 172.21.15.45:0/4086476230 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558849b5ee10 con 0x558849c2d860 2023-09-23T09:43:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbee1ffb700 10 client.26686 handle_command_reply: tid=0 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 client.26686 shutdown 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 msgr2=0x558849c1b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c1b3b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbef0003db0 tx=0x7fbef000e7d0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 shutdown_connections 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c2d860 0x558849c2fc80 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c1b3b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbee1ffb700 0 client.26686 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c1a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/4086476230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 >> 172.21.15.45:0/4086476230 conn(0x558849b87260 msgr2=0x558849c0e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:39.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 shutdown_connections 2023-09-23T09:43:39.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 1 -- 172.21.15.45:0/4086476230 wait complete. 2023-09-23T09:43:39.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.855+0000 7fbf0c750b80 20 client.26686 trim_cache size 0 max 16384 2023-09-23T09:43:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.875+0000 7fbf0c750b80 1 Processor -- start 2023-09-23T09:43:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.875+0000 7fbf0c750b80 1 -- start start 2023-09-23T09:43:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588499e6920 con 0x558849c12d20 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849c0d580 con 0x558849c12400 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849c167a0 con 0x558849c1dbd0 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbefeb03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbefeb03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c195a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38958/0 (socket says 172.21.15.45:38958) 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.876+0000 7fbefeb03700 1 -- 172.21.15.45:0/2410029536 learned_addr learned my addr 172.21.15.45:0/2410029536 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbefeb03700 1 -- 172.21.15.45:0/2410029536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 msgr2=0x558849c152a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbefeb03700 1 --2- 172.21.15.45:0/2410029536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c152a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbefeb03700 1 -- 172.21.15.45:0/2410029536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 msgr2=0x558849c1b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbefeb03700 1 --2- 172.21.15.45:0/2410029536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbefeb03700 1 -- 172.21.15.45:0/2410029536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558849c0a480 con 0x558849c1dbd0 2023-09-23T09:43:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbefeb03700 1 --2- 172.21.15.45:0/2410029536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c195a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbef001e450 tx=0x7fbef000eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37d275aed54bd33 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:39.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbeed7fa700 1 -- 172.21.15.45:0/2410029536 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef001e840 con 0x558849c1dbd0 2023-09-23T09:43:39.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbeed7fa700 1 -- 172.21.15.45:0/2410029536 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef000f040 con 0x558849c1dbd0 2023-09-23T09:43:39.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.877+0000 7fbeed7fa700 1 -- 172.21.15.45:0/2410029536 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0007810 con 0x558849c1dbd0 2023-09-23T09:43:39.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.878+0000 7fbf0c750b80 1 -- 172.21.15.45:0/2410029536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 msgr2=0x558849c195a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.878+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/2410029536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c195a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbef001e450 tx=0x7fbef000eba0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- 172.21.15.45:0/2410029536 shutdown_connections 2023-09-23T09:43:39.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/2410029536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c1dbd0 0x558849c195a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/2410029536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c12400 0x558849c152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/2410029536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12d20 0x558849c1b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- 172.21.15.45:0/2410029536 >> 172.21.15.45:0/2410029536 conn(0x558849b87260 msgr2=0x558849c18e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:39.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- 172.21.15.45:0/2410029536 shutdown_connections 2023-09-23T09:43:39.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- 172.21.15.45:0/2410029536 wait complete. 2023-09-23T09:43:39.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- start start 2023-09-23T09:43:39.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 Processor -- start 2023-09-23T09:43:39.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12400 0x558849c24440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c22700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c1dbd0 0x558849c22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849c23180 con 0x558849c12400 2023-09-23T09:43:39.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849c232f0 con 0x558849c1dbd0 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558849c234b0 con 0x558849c12d20 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.880+0000 7fbf0c750b80 10 client.0 _ll_register_callbacks cb 0x7fbef00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12400 0x558849c24440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c22700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12400 0x558849c24440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36854/0 (socket says 172.21.15.45:36854) 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefdb01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c22700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38962/0 (socket says 172.21.15.45:38962) 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 -- 172.21.15.45:0/3730539639 learned_addr learned my addr 172.21.15.45:0/3730539639 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefeb03700 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c1dbd0 0x558849c22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 -- 172.21.15.45:0/3730539639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 msgr2=0x558849c22700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c22700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 -- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c1dbd0 msgr2=0x558849c22c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c1dbd0 0x558849c22c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 -- 172.21.15.45:0/3730539639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef4017070 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefdb01700 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c22700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbefe302700 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12400 0x558849c24440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbee8097560 tx=0x7fbee80a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df848fddf1f3a5ef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbee1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbee1ffb700 1 -- 172.21.15.45:0/3730539639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee80a3360 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbee1ffb700 1 -- 172.21.15.45:0/3730539639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbee80cba30 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.881+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558849c2a0e0 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbee1ffb700 1 -- 172.21.15.45:0/3730539639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbee80cbbf0 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 10 client.17031 fetch_fsmap learned FSMap version 55 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x558849990de0 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbee1ffb700 1 -- 172.21.15.45:0/3730539639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbee80b9020 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbee1ffb700 1 -- 172.21.15.45:0/3730539639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbee809b2c0 con 0x558849c12400 2023-09-23T09:43:39.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 10 client.17031 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 20 client.17031 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c82730 0x558849c84b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 4 client.17031 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fbef4003e70 con 0x558849c82730 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.882+0000 7fbefdb01700 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c82730 0x558849c84b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbee1ffb700 10 client.17031 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbefdb01700 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c82730 0x558849c84b50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1860f1eca7fad30 server_cookie=3e8546cb45c15cc1 in_seq=0 out_seq=0 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbee1ffb700 1 -- 172.21.15.45:0/3730539639 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fbef4003e70 con 0x558849c82730 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbee1ffb700 10 client.17031 handle_command_reply: tid=0 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 client.17031 shutdown 2023-09-23T09:43:39.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12400 msgr2=0x558849c24440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12400 0x558849c24440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbee8097560 tx=0x7fbee80a1040 comp rx=0 tx=0).stop 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 shutdown_connections 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558849c82730 0x558849c84b50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558849c1dbd0 0x558849c22c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbee1ffb700 0 client.17031 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558849c12400 0x558849c24440 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 --2- 172.21.15.45:0/3730539639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558849c12d20 0x558849c22700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.883+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 >> 172.21.15.45:0/3730539639 conn(0x558849b87260 msgr2=0x558849c1aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:39.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.884+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 shutdown_connections 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:43:39.891 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:43:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.884+0000 7fbf0c750b80 1 -- 172.21.15.45:0/3730539639 wait complete. 2023-09-23T09:43:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.884+0000 7fbf0c750b80 20 client.17031 trim_cache size 0 max 16384 2023-09-23T09:43:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbee40a1750 msgr2=0x7fbee40a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbee40a1750 0x7fbee40a3c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbef819a860 tx=0x7fbef4009310 comp rx=0 tx=0).stop 2023-09-23T09:43:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8100740 msgr2=0x7fbef8199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8100740 0x7fbef8199250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009810 tx=0x7fbee800fa00 comp rx=0 tx=0).stop 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 shutdown_connections 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef8105e30 0x7fbef819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef8100740 0x7fbef8199250 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fbee40a1750 0x7fbee40a3c10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 --2- 172.21.15.45:0/3246174170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef8101100 0x7fbef8199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 >> 172.21.15.45:0/3246174170 conn(0x7fbef8076e10 msgr2=0x7fbef810dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 shutdown_connections 2023-09-23T09:43:39.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:43:39.885+0000 7fbf00566700 1 -- 172.21.15.45:0/3246174170 wait complete. 2023-09-23T09:43:39.902 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:43:40.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:39 smithi045 ceph-mon[98627]: pgmap v446: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 857 op/s 2023-09-23T09:43:40.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:39 smithi098 ceph-mon[102386]: pgmap v446: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 857 op/s 2023-09-23T09:43:40.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:39 smithi123 ceph-mon[103577]: pgmap v446: 97 pgs: 97 active+clean; 5.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 857 op/s 2023-09-23T09:43:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 835.0 s, 1711.2 tps, lat 59.218 ms stddev 70.350 2023-09-23T09:43:42.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:41 smithi098 ceph-mon[102386]: pgmap v447: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 23 MiB/s wr, 1.60k op/s 2023-09-23T09:43:42.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:41 smithi045 ceph-mon[98627]: pgmap v447: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 23 MiB/s wr, 1.60k op/s 2023-09-23T09:43:42.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:41 smithi123 ceph-mon[103577]: pgmap v447: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 23 MiB/s wr, 1.60k op/s 2023-09-23T09:43:44.303 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:43 smithi098 ceph-mon[102386]: pgmap v448: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-09-23T09:43:44.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:43 smithi045 ceph-mon[98627]: pgmap v448: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-09-23T09:43:44.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:43 smithi123 ceph-mon[103577]: pgmap v448: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-09-23T09:43:46.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:45 smithi098 ceph-mon[102386]: pgmap v449: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-23T09:43:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:45 smithi045 ceph-mon[98627]: pgmap v449: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-23T09:43:46.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:45 smithi123 ceph-mon[103577]: pgmap v449: 97 pgs: 97 active+clean; 5.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-23T09:43:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 840.0 s, 1688.2 tps, lat 59.698 ms stddev 70.549 2023-09-23T09:43:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 -- 172.21.15.45:0/1436050220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb410f570 msgr2=0x7f0fb4107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/1436050220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb410f570 0x7f0fb4107e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00b0a0 tx=0x7f0fac00b3b0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 -- 172.21.15.45:0/1436050220 shutdown_connections 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/1436050220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb411a600 0x7f0fb411c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/1436050220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb410f570 0x7f0fb4107e20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/1436050220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb410f9e0 0x7f0fb4108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 -- 172.21.15.45:0/1436050220 >> 172.21.15.45:0/1436050220 conn(0x7f0fb406cee0 msgr2=0x7f0fb406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 -- 172.21.15.45:0/1436050220 shutdown_connections 2023-09-23T09:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.096+0000 7f0fb88a3700 1 -- 172.21.15.45:0/1436050220 wait complete. 2023-09-23T09:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 Processor -- start 2023-09-23T09:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 -- start start 2023-09-23T09:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb410f570 0x7f0fb41a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb410f9e0 0x7f0fb41a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb411a600 0x7f0fb41aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41ab020 con 0x7f0fb410f570 2023-09-23T09:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41ab190 con 0x7f0fb410f9e0 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.097+0000 7f0fb88a3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41ab350 con 0x7f0fb411a600 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb410f9e0 0x7f0fb41a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb411a600 0x7f0fb41aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb410f9e0 0x7f0fb41a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58220/0 (socket says 172.21.15.45:58220) 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/2992135662 learned_addr learned my addr 172.21.15.45:0/2992135662 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb259c700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb410f570 0x7f0fb41a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/2992135662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb411a600 msgr2=0x7f0fb41aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb411a600 0x7f0fb41aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb410f570 msgr2=0x7f0fb41a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb410f570 0x7f0fb41a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/2992135662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac009d00 con 0x7f0fb410f9e0 2023-09-23T09:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb259c700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb410f570 0x7f0fb41a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.098+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb410f9e0 0x7f0fb41a6370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400b500 tx=0x7f0fa400b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.099+0000 7f0fa37fe700 1 -- 172.21.15.45:0/2992135662 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4011830 con 0x7f0fb410f9e0 2023-09-23T09:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.099+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb41ab630 con 0x7f0fb410f9e0 2023-09-23T09:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.099+0000 7f0fa37fe700 1 -- 172.21.15.45:0/2992135662 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa4011e50 con 0x7f0fb410f9e0 2023-09-23T09:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.099+0000 7f0fa37fe700 1 -- 172.21.15.45:0/2992135662 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4010de0 con 0x7f0fb410f9e0 2023-09-23T09:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.099+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb41b2e70 con 0x7f0fb410f9e0 2023-09-23T09:43:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.103+0000 7f0fc4a8db80 1 Processor -- start 2023-09-23T09:43:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fc4a8db80 1 -- start start 2023-09-23T09:43:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fa37fe700 1 -- 172.21.15.45:0/2992135662 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f0fa4010480 con 0x7f0fb410f9e0 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89bf40 0x55dbee89c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fb2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89bf40 0x55dbee89c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fb2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89bf40 0x55dbee89c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:38982/0 (socket says 172.21.15.45:38982) 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee8a7710 0x55dbee8a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee805400 con 0x55dbee8a7710 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee7f6bc0 con 0x55dbee89c860 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.104+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee7f44e0 con 0x55dbee89bf40 2023-09-23T09:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.105+0000 7f0fb2d9d700 1 -- 172.21.15.45:0/236425041 learned_addr learned my addr 172.21.15.45:0/236425041 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.105+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee8a7710 0x55dbee8a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/236425041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89bf40 msgr2=0x55dbee89c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89bf40 0x55dbee89c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/236425041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 msgr2=0x55dbee8a71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/236425041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbee852870 con 0x55dbee8a7710 2023-09-23T09:43:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fa37fe700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0f9c0a1790 0x7f0f9c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fb2d9d700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0f9c0a1790 0x7f0f9c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.106+0000 7f0fa37fe700 1 -- 172.21.15.45:0/2992135662 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0fa40cccb0 con 0x7f0fb410f9e0 2023-09-23T09:43:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.107+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee8a7710 0x55dbee8a9b00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40ca470 tx=0x7f0fa40d0eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5730827b078c5d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.107+0000 7f0fa0ff9700 1 -- 172.21.15.45:0/236425041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4009330 con 0x55dbee8a7710 2023-09-23T09:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.107+0000 7f0fb2d9d700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0f9c0a1790 0x7f0f9c0a3c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009b20 tx=0x7f0fa8008040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.107+0000 7f0fa0ff9700 1 -- 172.21.15.45:0/236425041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa4095fd0 con 0x55dbee8a7710 2023-09-23T09:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.107+0000 7f0fa0ff9700 1 -- 172.21.15.45:0/236425041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40997d0 con 0x55dbee8a7710 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/236425041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee8a7710 msgr2=0x55dbee8a9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee8a7710 0x55dbee8a9b00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40ca470 tx=0x7f0fa40d0eb0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/236425041 shutdown_connections 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee8a7710 0x55dbee8a9b00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89bf40 0x55dbee89c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/236425041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8a71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/236425041 >> 172.21.15.45:0/236425041 conn(0x55dbee811200 msgr2=0x55dbee8983c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/236425041 shutdown_connections 2023-09-23T09:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.108+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/236425041 wait complete. 2023-09-23T09:43:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 -- start start 2023-09-23T09:43:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 Processor -- start 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89bf40 0x55dbee8a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89c860 0x55dbee8a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee7efdd0 con 0x55dbee89c860 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee78c6b0 con 0x55dbee89bf40 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee78c350 con 0x55dbee8a7710 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fc4a8db80 10 client.0 _ll_register_callbacks cb 0x55dbee5e97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fb1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89c860 0x55dbee8a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fb259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89bf40 0x55dbee8a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fb259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89bf40 0x55dbee8a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58232/0 (socket says 172.21.15.45:58232) 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fb1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89c860 0x55dbee8a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36884/0 (socket says 172.21.15.45:36884) 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fb259c700 1 -- 172.21.15.45:0/1029355575 learned_addr learned my addr 172.21.15.45:0/1029355575 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.109+0000 7f0fb2d9d700 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.110+0000 7f0fb2d9d700 1 -- 172.21.15.45:0/1029355575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89bf40 msgr2=0x55dbee8a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.110+0000 7f0fb2d9d700 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89bf40 0x55dbee8a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.110+0000 7f0fb2d9d700 1 -- 172.21.15.45:0/1029355575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89c860 msgr2=0x55dbee8a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.110+0000 7f0fb2d9d700 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89c860 0x55dbee8a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.110+0000 7f0fb2d9d700 1 -- 172.21.15.45:0/1029355575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb41b2e70 con 0x55dbee8a7710 2023-09-23T09:43:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0fb2d9d700 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8a4ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8011540 tx=0x7f0fa8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ff5e27810327df server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0f99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:43:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1029355575 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8015bb0 con 0x55dbee8a7710 2023-09-23T09:43:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1029355575 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0fa8015d10 con 0x55dbee8a7710 2023-09-23T09:43:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbee890090 con 0x55dbee8a7710 2023-09-23T09:43:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1029355575 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fa8021440 con 0x55dbee8a7710 2023-09-23T09:43:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0fc4a8db80 10 client.26704 fetch_fsmap learned FSMap version 55 2023-09-23T09:43:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.112+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55dbee61ad80 con 0x55dbee8a7710 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1029355575 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbee890090 con 0x55dbee8a7710 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1029355575 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0fa8014510 con 0x55dbee8a7710 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0fc4a8db80 10 client.26704 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0fc4a8db80 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0fc4a8db80 20 client.26704 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee8b7310 0x55dbee8b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0fc4a8db80 4 client.26704 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:43:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.113+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbee7e8db0 con 0x55dbee8b7310 2023-09-23T09:43:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.117+0000 7f0fb259c700 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee8b7310 0x55dbee8b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.147+0000 7f0fb259c700 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee8b7310 0x55dbee8b9730 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc72133558d1c30c server_cookie=5f79569eb06b66a in_seq=0 out_seq=0 2023-09-23T09:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.152+0000 7f0f99ffb700 10 client.26704 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.163+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1029355575 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dbee7e8db0 con 0x55dbee8b7310 2023-09-23T09:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.163+0000 7f0f99ffb700 10 client.26704 handle_command_reply: tid=0 2023-09-23T09:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.163+0000 7f0fc4a8db80 1 client.26704 shutdown 2023-09-23T09:43:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.169+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 msgr2=0x55dbee8a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.169+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8a4ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8011540 tx=0x7f0fa8004770 comp rx=0 tx=0).stop 2023-09-23T09:43:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.183+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 shutdown_connections 2023-09-23T09:43:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.183+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee8b7310 0x55dbee8b9730 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.183+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8a4ee0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.183+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89bf40 0x55dbee8a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.183+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1029355575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89c860 0x55dbee8a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.183+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 >> 172.21.15.45:0/1029355575 conn(0x55dbee811200 msgr2=0x55dbee897f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.184+0000 7f0f99ffb700 0 client.26704 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.187+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 shutdown_connections 2023-09-23T09:43:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.192+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1029355575 wait complete. 2023-09-23T09:43:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.194+0000 7f0fc4a8db80 20 client.26704 trim_cache size 0 max 16384 2023-09-23T09:43:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 Processor -- start 2023-09-23T09:43:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 -- start start 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89c860 0x55dbee89e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee8a7710 0x55dbee89ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee6708c0 con 0x55dbee89bf40 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee8b9d90 con 0x55dbee8a7710 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee89cea0 con 0x55dbee89c860 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fb259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fb259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36890/0 (socket says 172.21.15.45:36890) 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fb1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89c860 0x55dbee89e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.218+0000 7f0fb259c700 1 -- 172.21.15.45:0/1362176210 learned_addr learned my addr 172.21.15.45:0/1362176210 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/1362176210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee8a7710 msgr2=0x55dbee89ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee8a7710 0x55dbee89ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/1362176210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 msgr2=0x55dbee8a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/1362176210 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbee603c00 con 0x55dbee89c860 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fb259c700 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89c860 0x55dbee89e540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4096700 tx=0x7f0fa4004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bf8f2ccfa4c890 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fa17fa700 1 -- 172.21.15.45:0/1362176210 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40b1070 con 0x55dbee89c860 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fa17fa700 1 -- 172.21.15.45:0/1362176210 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa4009330 con 0x55dbee89c860 2023-09-23T09:43:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.219+0000 7f0fa17fa700 1 -- 172.21.15.45:0/1362176210 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4099a60 con 0x55dbee89c860 2023-09-23T09:43:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1362176210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89c860 msgr2=0x55dbee89e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89c860 0x55dbee89e540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4096700 tx=0x7f0fa4004860 comp rx=0 tx=0).stop 2023-09-23T09:43:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1362176210 shutdown_connections 2023-09-23T09:43:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee8a7710 0x55dbee89ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1362176210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee89c860 0x55dbee89e540 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1362176210 >> 172.21.15.45:0/1362176210 conn(0x55dbee811200 msgr2=0x55dbee89de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.220+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1362176210 shutdown_connections 2023-09-23T09:43:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.221+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1362176210 wait complete. 2023-09-23T09:43:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.221+0000 7f0fc4a8db80 1 -- start start 2023-09-23T09:43:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.221+0000 7f0fc4a8db80 1 Processor -- start 2023-09-23T09:43:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.221+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fc4a8db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee8accf0 con 0x55dbee89bf40 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee8ace60 con 0x55dbee89c860 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fc4a8db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbee8ad020 con 0x55dbee8a7710 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fc4a8db80 10 client.0 _ll_register_callbacks cb 0x7f0fa8011af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58246/0 (socket says 172.21.15.45:58246) 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8ac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39004/0 (socket says 172.21.15.45:39004) 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/1046696460 learned_addr learned my addr 172.21.15.45:0/1046696460 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:43:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/1046696460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 msgr2=0x55dbee8ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/1046696460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 msgr2=0x55dbee8adfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8adfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.222+0000 7f0fb1d9b700 1 -- 172.21.15.45:0/1046696460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb411d320 con 0x55dbee89c860 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0fb1d9b700 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8ac270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4096140 tx=0x7f0fa4010fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f766193cc27a4c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0f99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1046696460 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40b1030 con 0x55dbee89c860 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1046696460 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0fa40b2070 con 0x55dbee89c860 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbee7e8ad0 con 0x55dbee89c860 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1046696460 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fa40997d0 con 0x55dbee89c860 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0fc4a8db80 10 client.26696 fetch_fsmap learned FSMap version 55 2023-09-23T09:43:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55dbee852870 con 0x55dbee89c860 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.223+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1046696460 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbee7e8ad0 con 0x55dbee89c860 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1046696460 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0fa4099ef0 con 0x55dbee89c860 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0fc4a8db80 10 client.26696 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0fc4a8db80 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0fc4a8db80 20 client.26696 populate_metadata read hostname 'smithi045' 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee90c020 0x55dbee90e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0fc4a8db80 4 client.26696 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dbee63bf10 con 0x55dbee90c020 2023-09-23T09:43:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.224+0000 7f0fb259c700 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee90c020 0x55dbee90e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:43:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.225+0000 7f0f99ffb700 10 client.26696 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.225+0000 7f0fb259c700 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee90c020 0x55dbee90e440 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc27bd1d39459138 server_cookie=2ae4f55b8fbd6e3f in_seq=0 out_seq=0 2023-09-23T09:43:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0f99ffb700 1 -- 172.21.15.45:0/1046696460 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dbee63bf10 con 0x55dbee90c020 2023-09-23T09:43:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0f99ffb700 10 client.26696 handle_command_reply: tid=0 2023-09-23T09:43:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 client.26696 shutdown 2023-09-23T09:43:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 msgr2=0x55dbee8ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8ac270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4096140 tx=0x7f0fa4010fc0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 shutdown_connections 2023-09-23T09:43:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55dbee90c020 0x55dbee90e440 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dbee8a7710 0x55dbee8ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbee89bf40 0x55dbee8adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 --2- 172.21.15.45:0/1046696460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dbee89c860 0x55dbee8ac270 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 >> 172.21.15.45:0/1046696460 conn(0x55dbee811200 msgr2=0x55dbee89e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0f99ffb700 0 client.26696 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 shutdown_connections 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.252+0000 7f0fc4a8db80 1 -- 172.21.15.45:0/1046696460 wait complete. 2023-09-23T09:43:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.253+0000 7f0fc4a8db80 20 client.26696 trim_cache size 0 max 16384 2023-09-23T09:43:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:43:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:43:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:43:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:43:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.254+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0f9c0a1790 msgr2=0x7f0f9c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.254+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0f9c0a1790 0x7f0f9c0a3c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009b20 tx=0x7f0fa8008040 comp rx=0 tx=0).stop 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.254+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb410f9e0 msgr2=0x7f0fb41a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.254+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb410f9e0 0x7f0fb41a6370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400b500 tx=0x7f0fa400b810 comp rx=0 tx=0).stop 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 shutdown_connections 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fb411a600 0x7f0fb41aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0f9c0a1790 0x7f0f9c0a3c50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb410f570 0x7f0fb41a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 --2- 172.21.15.45:0/2992135662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fb410f9e0 0x7f0fb41a6370 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 >> 172.21.15.45:0/2992135662 conn(0x7f0fb406cee0 msgr2=0x7f0fb410d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 shutdown_connections 2023-09-23T09:43:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:43:47.255+0000 7f0fb88a3700 1 -- 172.21.15.45:0/2992135662 wait complete. 2023-09-23T09:43:48.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:47 smithi098 ceph-mon[102386]: pgmap v450: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 19 MiB/s wr, 1.33k op/s 2023-09-23T09:43:48.339 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:47 smithi045 ceph-mon[98627]: pgmap v450: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 19 MiB/s wr, 1.33k op/s 2023-09-23T09:43:48.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:47 smithi123 ceph-mon[103577]: pgmap v450: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 19 MiB/s wr, 1.33k op/s 2023-09-23T09:43:50.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:50 smithi098 ceph-mon[102386]: pgmap v451: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-09-23T09:43:50.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:50 smithi045 ceph-mon[98627]: pgmap v451: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-09-23T09:43:50.349 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:50 smithi123 ceph-mon[103577]: pgmap v451: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-09-23T09:43:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 845.0 s, 1411.6 tps, lat 70.517 ms stddev 75.909 2023-09-23T09:43:52.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:52 smithi098 ceph-mon[102386]: pgmap v452: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 20 MiB/s wr, 1.28k op/s 2023-09-23T09:43:52.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:52 smithi045 ceph-mon[98627]: pgmap v452: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 20 MiB/s wr, 1.28k op/s 2023-09-23T09:43:52.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:52 smithi123 ceph-mon[103577]: pgmap v452: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 20 MiB/s wr, 1.28k op/s 2023-09-23T09:43:54.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:54 smithi045 ceph-mon[98627]: pgmap v453: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.5 MiB/s wr, 501 op/s 2023-09-23T09:43:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:54 smithi123 ceph-mon[103577]: pgmap v453: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.5 MiB/s wr, 501 op/s 2023-09-23T09:43:54.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:54 smithi098 ceph-mon[102386]: pgmap v453: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.5 MiB/s wr, 501 op/s 2023-09-23T09:43:56.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:56 smithi045 ceph-mon[98627]: pgmap v454: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 557 op/s 2023-09-23T09:43:56.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:56 smithi123 ceph-mon[103577]: pgmap v454: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 557 op/s 2023-09-23T09:43:56.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:56 smithi098 ceph-mon[102386]: pgmap v454: 97 pgs: 97 active+clean; 5.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 557 op/s 2023-09-23T09:43:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 850.0 s, 1412.0 tps, lat 70.218 ms stddev 84.469 2023-09-23T09:43:58.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:43:58 smithi045 ceph-mon[98627]: pgmap v455: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 710 op/s 2023-09-23T09:43:58.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:43:58 smithi123 ceph-mon[103577]: pgmap v455: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 710 op/s 2023-09-23T09:43:58.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:43:58 smithi098 ceph-mon[102386]: pgmap v455: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 710 op/s 2023-09-23T09:44:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:00 smithi123 ceph-mon[103577]: pgmap v456: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.7 MiB/s wr, 551 op/s 2023-09-23T09:44:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.427+0000 7fde1a055700 1 -- 172.21.15.45:0/886924746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1410f570 msgr2=0x7fde14107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.427+0000 7fde1a055700 1 --2- 172.21.15.45:0/886924746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1410f570 0x7fde14107e20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fde0801a9a0 tx=0x7fde0801acb0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.432+0000 7fde1a055700 1 -- 172.21.15.45:0/886924746 shutdown_connections 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.432+0000 7fde1a055700 1 --2- 172.21.15.45:0/886924746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde1411a600 0x7fde1411c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.432+0000 7fde1a055700 1 --2- 172.21.15.45:0/886924746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1410f570 0x7fde14107e20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.432+0000 7fde1a055700 1 --2- 172.21.15.45:0/886924746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1410f9e0 0x7fde14108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.432+0000 7fde1a055700 1 -- 172.21.15.45:0/886924746 >> 172.21.15.45:0/886924746 conn(0x7fde1406cee0 msgr2=0x7fde1406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.432+0000 7fde1a055700 1 -- 172.21.15.45:0/886924746 shutdown_connections 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.432+0000 7fde1a055700 1 -- 172.21.15.45:0/886924746 wait complete. 2023-09-23T09:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 Processor -- start 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 -- start start 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1410f570 0x7fde14119890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde1410f9e0 0x7fde14119dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1411a600 0x7fde14115730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde14115ed0 con 0x7fde1411a600 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde14116040 con 0x7fde1410f9e0 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.433+0000 7fde1a055700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde14116200 con 0x7fde1410f570 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.434+0000 7fde13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1411a600 0x7fde14115730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.434+0000 7fde13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1411a600 0x7fde14115730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36908/0 (socket says 172.21.15.45:36908) 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.434+0000 7fde13fff700 1 -- 172.21.15.45:0/1425134818 learned_addr learned my addr 172.21.15.45:0/1425134818 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.435+0000 7fde13fff700 1 -- 172.21.15.45:0/1425134818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1410f570 msgr2=0x7fde14119890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.435+0000 7fde13fff700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1410f570 0x7fde14119890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.435+0000 7fde13fff700 1 -- 172.21.15.45:0/1425134818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde1410f9e0 msgr2=0x7fde14119dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.435+0000 7fde13fff700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde1410f9e0 0x7fde14119dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.435+0000 7fde13fff700 1 -- 172.21.15.45:0/1425134818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0801a600 con 0x7fde1411a600 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.435+0000 7fde13fff700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1411a600 0x7fde14115730 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fde0400b480 tx=0x7fde0400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.436+0000 7fde10ff9700 1 -- 172.21.15.45:0/1425134818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde040118f0 con 0x7fde1411a600 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.436+0000 7fde10ff9700 1 -- 172.21.15.45:0/1425134818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde04010380 con 0x7fde1411a600 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.436+0000 7fde10ff9700 1 -- 172.21.15.45:0/1425134818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0400f540 con 0x7fde1411a600 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.437+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde141164e0 con 0x7fde1411a600 2023-09-23T09:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.437+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1407b300 con 0x7fde1411a600 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 Processor -- start 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 -- start start 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769983b400 con 0x5576998d1f40 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769982cbc0 con 0x5576998dd710 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.442+0000 7fde2623fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769982a4e0 con 0x5576998d2860 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.443+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.443+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58264/0 (socket says 172.21.15.45:58264) 2023-09-23T09:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.443+0000 7fde12ffd700 1 -- 172.21.15.45:0/2580593586 learned_addr learned my addr 172.21.15.45:0/2580593586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.443+0000 7fde12ffd700 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.450+0000 7fde12ffd700 1 -- 172.21.15.45:0/2580593586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 msgr2=0x5576998dd1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.450+0000 7fde12ffd700 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998dd1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.450+0000 7fde12ffd700 1 -- 172.21.15.45:0/2580593586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 msgr2=0x5576998d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.450+0000 7fde12ffd700 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998d2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.450+0000 7fde12ffd700 1 -- 172.21.15.45:0/2580593586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557699888870 con 0x5576998dd710 2023-09-23T09:44:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.452+0000 7fde10ff9700 1 -- 172.21.15.45:0/1425134818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fde040107f0 con 0x7fde1411a600 2023-09-23T09:44:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.453+0000 7fde10ff9700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fddfc0a1920 0x7fddfc0a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.453+0000 7fde10ff9700 1 -- 172.21.15.45:0/1425134818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fde040cc740 con 0x7fde1411a600 2023-09-23T09:44:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.473+0000 7fde137fe700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fddfc0a1920 0x7fddfc0a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.474+0000 7fde137fe700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fddfc0a1920 0x7fddfc0a3de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde0801a8f0 tx=0x7fde0801aef0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.476+0000 7fde12ffd700 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dfb00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d780 tx=0x7fde0c00b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a9291ee4f0ba7c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.480+0000 7fddf9ffb700 1 -- 172.21.15.45:0/2580593586 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00d030 con 0x5576998dd710 2023-09-23T09:44:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.480+0000 7fddf9ffb700 1 -- 172.21.15.45:0/2580593586 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0c00d190 con 0x5576998dd710 2023-09-23T09:44:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.480+0000 7fddf9ffb700 1 -- 172.21.15.45:0/2580593586 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00c690 con 0x5576998dd710 2023-09-23T09:44:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.483+0000 7fde2623fb80 1 -- 172.21.15.45:0/2580593586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 msgr2=0x5576998dfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.483+0000 7fde2623fb80 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dfb00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d780 tx=0x7fde0c00b4a0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.486+0000 7fde2623fb80 1 -- 172.21.15.45:0/2580593586 shutdown_connections 2023-09-23T09:44:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.486+0000 7fde2623fb80 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dfb00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.486+0000 7fde2623fb80 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.486+0000 7fde2623fb80 1 --2- 172.21.15.45:0/2580593586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998dd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.486+0000 7fde2623fb80 1 -- 172.21.15.45:0/2580593586 >> 172.21.15.45:0/2580593586 conn(0x557699847200 msgr2=0x5576998ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.491+0000 7fde2623fb80 1 -- 172.21.15.45:0/2580593586 shutdown_connections 2023-09-23T09:44:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.491+0000 7fde2623fb80 1 -- 172.21.15.45:0/2580593586 wait complete. 2023-09-23T09:44:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 -- start start 2023-09-23T09:44:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 Processor -- start 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998da9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998daef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557699825dd0 con 0x5576998d1f40 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576997c26b0 con 0x5576998dd710 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576997c2350 con 0x5576998d2860 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.492+0000 7fde2623fb80 10 client.0 _ll_register_callbacks cb 0x55769961f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.493+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998da9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.493+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998da9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39026/0 (socket says 172.21.15.45:39026) 2023-09-23T09:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.493+0000 7fde12ffd700 1 -- 172.21.15.45:0/1583018054 learned_addr learned my addr 172.21.15.45:0/1583018054 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.493+0000 7fde137fe700 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.493+0000 7fde13fff700 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.494+0000 7fde137fe700 1 -- 172.21.15.45:0/1583018054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 msgr2=0x5576998da9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.494+0000 7fde137fe700 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998da9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.494+0000 7fde137fe700 1 -- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 msgr2=0x5576998daef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.494+0000 7fde137fe700 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998daef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.494+0000 7fde137fe700 1 -- 172.21.15.45:0/1583018054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c015040 con 0x5576998d1f40 2023-09-23T09:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.494+0000 7fde137fe700 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998da470 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde08006010 tx=0x7fde0801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c081a7d3fe84c5f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.496+0000 7fddf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.496+0000 7fddf2ffd700 1 -- 172.21.15.45:0/1583018054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08003bf0 con 0x5576998d1f40 2023-09-23T09:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.496+0000 7fddf2ffd700 1 -- 172.21.15.45:0/1583018054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fde08013030 con 0x5576998d1f40 2023-09-23T09:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.496+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576998c6090 con 0x5576998d1f40 2023-09-23T09:44:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.496+0000 7fddf2ffd700 1 -- 172.21.15.45:0/1583018054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde08003d50 con 0x5576998d1f40 2023-09-23T09:44:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.496+0000 7fde2623fb80 10 client.17055 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.496+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557699650d80 con 0x5576998d1f40 2023-09-23T09:44:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fddf2ffd700 1 -- 172.21.15.45:0/1583018054 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576998c6090 con 0x5576998d1f40 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fddf2ffd700 1 -- 172.21.15.45:0/1583018054 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fde08004ab0 con 0x5576998d1f40 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fde2623fb80 10 client.17055 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fde2623fb80 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fde2623fb80 20 client.17055 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fde2623fb80 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5576998ed180 0x5576998ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fde2623fb80 4 client.17055 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.497+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55769981edb0 con 0x5576998ed180 2023-09-23T09:44:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.498+0000 7fde12ffd700 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5576998ed180 0x5576998ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.502+0000 7fddf2ffd700 10 client.17055 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.502+0000 7fde12ffd700 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5576998ed180 0x5576998ef5a0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63fd9b41adf7649 server_cookie=e676e6a9f66b69ad in_seq=0 out_seq=0 2023-09-23T09:44:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.502+0000 7fddf2ffd700 1 -- 172.21.15.45:0/1583018054 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55769981edb0 con 0x5576998ed180 2023-09-23T09:44:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.502+0000 7fddf2ffd700 10 client.17055 handle_command_reply: tid=0 2023-09-23T09:44:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.502+0000 7fde2623fb80 1 client.17055 shutdown 2023-09-23T09:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 msgr2=0x5576998da470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998da470 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde08006010 tx=0x7fde0801acb0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 shutdown_connections 2023-09-23T09:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5576998ed180 0x5576998ef5a0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998daef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x5576998da470 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 --2- 172.21.15.45:0/1583018054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x5576998da9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 >> 172.21.15.45:0/1583018054 conn(0x557699847200 msgr2=0x5576998cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fddf2ffd700 0 client.17055 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 shutdown_connections 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 1 -- 172.21.15.45:0/1583018054 wait complete. 2023-09-23T09:44:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.506+0000 7fde2623fb80 20 client.17055 trim_cache size 0 max 16384 2023-09-23T09:44:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.528+0000 7fde2623fb80 1 Processor -- start 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.528+0000 7fde2623fb80 1 -- start start 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d1f40 0x5576998d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d2860 0x5576998e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde2623fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576996a68c0 con 0x5576998d2860 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde2623fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576998ca490 con 0x5576998dd710 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde2623fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576998e4740 con 0x5576998d1f40 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d2860 0x5576998e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d2860 0x5576998e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36930/0 (socket says 172.21.15.45:36930) 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 -- 172.21.15.45:0/380472445 learned_addr learned my addr 172.21.15.45:0/380472445 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde137fe700 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d1f40 0x5576998d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde13fff700 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 -- 172.21.15.45:0/380472445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d1f40 msgr2=0x5576998d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d1f40 0x5576998d4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 -- 172.21.15.45:0/380472445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 msgr2=0x5576998dae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 -- 172.21.15.45:0/380472445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576998f41b0 con 0x5576998d2860 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fde12ffd700 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d2860 0x5576998e4200 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00dd70 tx=0x7fde0c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf0cbca2be49ae2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.529+0000 7fddfa7fc700 1 -- 172.21.15.45:0/380472445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c020050 con 0x5576998d2860 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fddfa7fc700 1 -- 172.21.15.45:0/380472445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0c021040 con 0x5576998d2860 2023-09-23T09:44:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fddfa7fc700 1 -- 172.21.15.45:0/380472445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c022d90 con 0x5576998d2860 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 -- 172.21.15.45:0/380472445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d2860 msgr2=0x5576998e4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d2860 0x5576998e4200 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00dd70 tx=0x7fde0c009040 comp rx=0 tx=0).stop 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 -- 172.21.15.45:0/380472445 shutdown_connections 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x5576998dae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d1f40 0x5576998d4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 --2- 172.21.15.45:0/380472445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d2860 0x5576998e4200 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 -- 172.21.15.45:0/380472445 >> 172.21.15.45:0/380472445 conn(0x557699847200 msgr2=0x5576998da700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 -- 172.21.15.45:0/380472445 shutdown_connections 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.530+0000 7fde2623fb80 1 -- 172.21.15.45:0/380472445 wait complete. 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 -- start start 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 Processor -- start 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x557699934f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x557699935460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x55769993ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557699935d20 con 0x5576998d1f40 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576999404e0 con 0x5576998dd710 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557699940620 con 0x5576998d2860 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde2623fb80 10 client.0 _ll_register_callbacks cb 0x7fde080041e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x557699934f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x557699935460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde12ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x557699935460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39042/0 (socket says 172.21.15.45:39042) 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde13fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x55769993ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde12ffd700 1 -- 172.21.15.45:0/3987716349 learned_addr learned my addr 172.21.15.45:0/3987716349 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde12ffd700 1 -- 172.21.15.45:0/3987716349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 msgr2=0x55769993ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde12ffd700 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x55769993ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde12ffd700 1 -- 172.21.15.45:0/3987716349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 msgr2=0x557699934f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.531+0000 7fde12ffd700 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x557699934f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fde12ffd700 1 -- 172.21.15.45:0/3987716349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c011040 con 0x5576998d2860 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fde137fe700 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x557699934f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fde12ffd700 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x557699935460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fde0c000f80 tx=0x7fde0c00b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913ad0ab13caf7f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fddf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fddf2ffd700 1 -- 172.21.15.45:0/3987716349 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c022530 con 0x5576998d2860 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde0c00d190 con 0x5576998d2860 2023-09-23T09:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fddf2ffd700 1 -- 172.21.15.45:0/3987716349 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fde0c022690 con 0x5576998d2860 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fddf2ffd700 1 -- 172.21.15.45:0/3987716349 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde0c020020 con 0x5576998d2860 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fde2623fb80 10 client.26719 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557699639c00 con 0x5576998d2860 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.532+0000 7fddf2ffd700 1 -- 172.21.15.45:0/3987716349 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0c00d190 con 0x5576998d2860 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fddf2ffd700 1 -- 172.21.15.45:0/3987716349 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fde0c032070 con 0x5576998d2860 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde2623fb80 10 client.26719 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde2623fb80 10 client.26719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde2623fb80 20 client.26719 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde2623fb80 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557699941950 0x557699943d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde2623fb80 4 client.26719 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55769981ead0 con 0x557699941950 2023-09-23T09:44:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde137fe700 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557699941950 0x557699943d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fde137fe700 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557699941950 0x557699943d70 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6d818db6f4e3e6 server_cookie=635fd6f9c4b77ecc in_seq=0 out_seq=0 2023-09-23T09:44:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.533+0000 7fddf2ffd700 10 client.26719 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:00.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:00 smithi098 ceph-mon[102386]: pgmap v456: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.7 MiB/s wr, 551 op/s 2023-09-23T09:44:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fddf2ffd700 1 -- 172.21.15.45:0/3987716349 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55769981ead0 con 0x557699941950 2023-09-23T09:44:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fddf2ffd700 10 client.26719 handle_command_reply: tid=0 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 client.26719 shutdown 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 msgr2=0x557699935460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x557699935460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fde0c000f80 tx=0x7fde0c00b9a0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 shutdown_connections 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557699941950 0x557699943d70 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576998dd710 0x55769993ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fddf2ffd700 0 client.26719 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576998d1f40 0x557699934f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 --2- 172.21.15.45:0/3987716349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576998d2860 0x557699935460 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 >> 172.21.15.45:0/3987716349 conn(0x557699847200 msgr2=0x5576998dbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 shutdown_connections 2023-09-23T09:44:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 1 -- 172.21.15.45:0/3987716349 wait complete. 2023-09-23T09:44:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.556+0000 7fde2623fb80 20 client.26719 trim_cache size 0 max 16384 2023-09-23T09:44:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:44:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:44:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:44:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fddfc0a1920 msgr2=0x7fddfc0a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fddfc0a1920 0x7fddfc0a3de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde0801a8f0 tx=0x7fde0801aef0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1411a600 msgr2=0x7fde14115730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1411a600 0x7fde14115730 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fde0400b480 tx=0x7fde0400b840 comp rx=0 tx=0).stop 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 shutdown_connections 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fddfc0a1920 0x7fddfc0a3de0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde1411a600 0x7fde14115730 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1410f570 0x7fde14119890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 --2- 172.21.15.45:0/1425134818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde1410f9e0 0x7fde14119dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 >> 172.21.15.45:0/1425134818 conn(0x7fde1406cee0 msgr2=0x7fde140717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 shutdown_connections 2023-09-23T09:44:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:00.557+0000 7fde1a055700 1 -- 172.21.15.45:0/1425134818 wait complete. 2023-09-23T09:44:00.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:00 smithi045 ceph-mon[98627]: pgmap v456: 97 pgs: 97 active+clean; 5.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.7 MiB/s wr, 551 op/s 2023-09-23T09:44:01.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:01 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:44:01.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:01 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:44:01.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:01 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 855.0 s, 1037.3 tps, lat 78.183 ms stddev 89.466 2023-09-23T09:44:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:02 smithi098 ceph-mon[102386]: pgmap v457: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 784 op/s 2023-09-23T09:44:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:02 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:44:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:02 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:44:02.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:02 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:44:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:02 smithi045 ceph-mon[98627]: pgmap v457: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 784 op/s 2023-09-23T09:44:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:02 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:44:02.592 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:02 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:44:02.592 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:02 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:44:02.599 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:02 smithi123 ceph-mon[103577]: pgmap v457: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 784 op/s 2023-09-23T09:44:02.599 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:02 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:44:02.599 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:02 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:44:02.599 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:02 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:44:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:03 smithi098 ceph-mon[102386]: pgmap v458: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 514 op/s 2023-09-23T09:44:03.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:03 smithi045 ceph-mon[98627]: pgmap v458: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 514 op/s 2023-09-23T09:44:03.604 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:03 smithi123 ceph-mon[103577]: pgmap v458: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 514 op/s 2023-09-23T09:44:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:05 smithi098 ceph-mon[102386]: pgmap v459: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 820 op/s 2023-09-23T09:44:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:05 smithi098 ceph-mon[102386]: osdmap e104: 12 total, 12 up, 12 in 2023-09-23T09:44:05.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:05 smithi045 ceph-mon[98627]: pgmap v459: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 820 op/s 2023-09-23T09:44:05.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:05 smithi045 ceph-mon[98627]: osdmap e104: 12 total, 12 up, 12 in 2023-09-23T09:44:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:05 smithi123 ceph-mon[103577]: pgmap v459: 97 pgs: 97 active+clean; 5.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 820 op/s 2023-09-23T09:44:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:05 smithi123 ceph-mon[103577]: osdmap e104: 12 total, 12 up, 12 in 2023-09-23T09:44:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 860.0 s, 786.6 tps, lat 151.329 ms stddev 273.197 2023-09-23T09:44:08.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:07 smithi098 ceph-mon[102386]: pgmap v461: 97 pgs: 34 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 50 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2023-09-23T09:44:08.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:07 smithi045 ceph-mon[98627]: pgmap v461: 97 pgs: 34 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 50 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2023-09-23T09:44:08.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:07 smithi123 ceph-mon[103577]: pgmap v461: 97 pgs: 34 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 50 active+clean; 5.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2023-09-23T09:44:09.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 -- 172.21.15.45:0/3027748060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1010b490 msgr2=0x7f5c10107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 --2- 172.21.15.45:0/3027748060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1010b490 0x7f5c10107e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009990 tx=0x7f5c0c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 -- 172.21.15.45:0/3027748060 shutdown_connections 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 --2- 172.21.15.45:0/3027748060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c10115e20 0x7f5c10118230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 --2- 172.21.15.45:0/3027748060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1010b490 0x7f5c10107e20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 --2- 172.21.15.45:0/3027748060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10108360 0x7f5c101087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 -- 172.21.15.45:0/3027748060 >> 172.21.15.45:0/3027748060 conn(0x7f5c1006d0e0 msgr2=0x7f5c1006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 -- 172.21.15.45:0/3027748060 shutdown_connections 2023-09-23T09:44:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.119+0000 7f5c1740d700 1 -- 172.21.15.45:0/3027748060 wait complete. 2023-09-23T09:44:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.120+0000 7f5c1740d700 1 Processor -- start 2023-09-23T09:44:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.120+0000 7f5c1740d700 1 -- start start 2023-09-23T09:44:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c1740d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c10108360 0x7f5c101a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c1740d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1010b490 0x7f5c101a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c1740d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10115e20 0x7f5c101a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c1740d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101a1eb0 con 0x7f5c10115e20 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c1740d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101a2020 con 0x7f5c10108360 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c1740d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101a21e0 con 0x7f5c1010b490 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10115e20 0x7f5c101a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10115e20 0x7f5c101a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36950/0 (socket says 172.21.15.45:36950) 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 -- 172.21.15.45:0/4059766623 learned_addr learned my addr 172.21.15.45:0/4059766623 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 -- 172.21.15.45:0/4059766623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1010b490 msgr2=0x7f5c101a7d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1010b490 0x7f5c101a7d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 -- 172.21.15.45:0/4059766623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c10108360 msgr2=0x7f5c101a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c10108360 0x7f5c101a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.121+0000 7f5c159aa700 1 -- 172.21.15.45:0/4059766623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0095f0 con 0x7f5c10115e20 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.122+0000 7f5c159aa700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10115e20 0x7f5c101a1840 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000b410 tx=0x7f5c0000b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.122+0000 7f5c067fc700 1 -- 172.21.15.45:0/4059766623 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00011830 con 0x7f5c10115e20 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.122+0000 7f5c067fc700 1 -- 172.21.15.45:0/4059766623 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c00011e50 con 0x7f5c10115e20 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.122+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c101a24c0 con 0x7f5c10115e20 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.122+0000 7f5c067fc700 1 -- 172.21.15.45:0/4059766623 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000f2e0 con 0x7f5c10115e20 2023-09-23T09:44:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.122+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c101ae7c0 con 0x7f5c10115e20 2023-09-23T09:44:10.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.125+0000 7f5c067fc700 1 -- 172.21.15.45:0/4059766623 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f5c000105a0 con 0x7f5c10115e20 2023-09-23T09:44:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.126+0000 7f5c067fc700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5bfc0a1750 0x7f5bfc0a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.126+0000 7f5c067fc700 1 -- 172.21.15.45:0/4059766623 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5c000ccca0 con 0x7f5c10115e20 2023-09-23T09:44:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.126+0000 7f5c151a9700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5bfc0a1750 0x7f5bfc0a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.127+0000 7f5c235f7b80 1 Processor -- start 2023-09-23T09:44:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.128+0000 7f5c235f7b80 1 -- start start 2023-09-23T09:44:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.128+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b824ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.128+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.128+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b8327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.128+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b785460 con 0x55f73b8303e0 2023-09-23T09:44:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.128+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b776c20 con 0x55f73b825530 2023-09-23T09:44:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.128+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b774540 con 0x55f73b824c10 2023-09-23T09:44:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b8327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b8327d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36958/0 (socket says 172.21.15.45:36958) 2023-09-23T09:44:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 -- 172.21.15.45:0/2478716350 learned_addr learned my addr 172.21.15.45:0/2478716350 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c149a8700 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c151a9700 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b824ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 -- 172.21.15.45:0/2478716350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 msgr2=0x55f73b824ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b824ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 -- 172.21.15.45:0/2478716350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 msgr2=0x55f73b82fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.129+0000 7f5c159aa700 1 -- 172.21.15.45:0/2478716350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f73b7d28d0 con 0x55f73b8303e0 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.130+0000 7f5c159aa700 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b8327d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5c000999b0 tx=0x7f5c00099690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5fc6a60a586981 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.130+0000 7f5c149a8700 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.130+0000 7f5bfb7fe700 1 -- 172.21.15.45:0/2478716350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0009de00 con 0x55f73b8303e0 2023-09-23T09:44:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.130+0000 7f5c151a9700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5bfc0a1750 0x7f5bfc0a3c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c003730 tx=0x7f5c0c00b810 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.130+0000 7f5bfb7fe700 1 -- 172.21.15.45:0/2478716350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c000a2040 con 0x55f73b8303e0 2023-09-23T09:44:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.130+0000 7f5bfb7fe700 1 -- 172.21.15.45:0/2478716350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000991a0 con 0x55f73b8303e0 2023-09-23T09:44:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.131+0000 7f5c235f7b80 1 -- 172.21.15.45:0/2478716350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 msgr2=0x55f73b8327d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.131+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b8327d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5c000999b0 tx=0x7f5c00099690 comp rx=0 tx=0).stop 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.131+0000 7f5c235f7b80 1 -- 172.21.15.45:0/2478716350 shutdown_connections 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.131+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b8327d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.131+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b824ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.131+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/2478716350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.131+0000 7f5c235f7b80 1 -- 172.21.15.45:0/2478716350 >> 172.21.15.45:0/2478716350 conn(0x55f73b791260 msgr2=0x55f73b818790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.132+0000 7f5c235f7b80 1 -- 172.21.15.45:0/2478716350 shutdown_connections 2023-09-23T09:44:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.132+0000 7f5c235f7b80 1 -- 172.21.15.45:0/2478716350 wait complete. 2023-09-23T09:44:10.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.132+0000 7f5c235f7b80 1 -- start start 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.132+0000 7f5c235f7b80 1 Processor -- start 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b82d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b825530 0x55f73b82d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b82dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b76fe30 con 0x55f73b825530 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b70c710 con 0x55f73b8303e0 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b70c3b0 con 0x55f73b824c10 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c235f7b80 10 client.0 _ll_register_callbacks cb 0x55f73b569800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b82dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b82dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58312/0 (socket says 172.21.15.45:58312) 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c159aa700 1 -- 172.21.15.45:0/1585304259 learned_addr learned my addr 172.21.15.45:0/1585304259 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.133+0000 7f5c149a8700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b825530 0x55f73b82d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c151a9700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b82d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c159aa700 1 -- 172.21.15.45:0/1585304259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 msgr2=0x55f73b82d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c159aa700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b82d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c159aa700 1 -- 172.21.15.45:0/1585304259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b825530 msgr2=0x55f73b82d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c159aa700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b825530 0x55f73b82d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c159aa700 1 -- 172.21.15.45:0/1585304259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c101ae7c0 con 0x55f73b8303e0 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c149a8700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b825530 0x55f73b82d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c151a9700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b82d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:44:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5c159aa700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b82dc80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c00096720 tx=0x7f5c000109a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a0d5f895fd5574 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.134+0000 7f5bf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:44:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.135+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1585304259 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000991a0 con 0x55f73b8303e0 2023-09-23T09:44:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.135+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1585304259 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5c00099300 con 0x55f73b8303e0 2023-09-23T09:44:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.135+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f73b810250 con 0x55f73b8303e0 2023-09-23T09:44:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.135+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1585304259 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c00099f10 con 0x55f73b8303e0 2023-09-23T09:44:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 10 client.26711 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55f73b59ade0 con 0x55f73b8303e0 2023-09-23T09:44:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1585304259 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f73b810250 con 0x55f73b8303e0 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1585304259 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5c00095aa0 con 0x55f73b8303e0 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 10 client.26711 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 10 client.26711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 20 client.26711 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b81dc50 0x55f73b820070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 4 client.26711 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f73b768e10 con 0x55f73b81dc50 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.136+0000 7f5c151a9700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b81dc50 0x55f73b820070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.137+0000 7f5c151a9700 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b81dc50 0x55f73b820070 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be743eaa89ae6efb server_cookie=19e9060f3b760b36 in_seq=0 out_seq=0 2023-09-23T09:44:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.137+0000 7f5bf8ff9700 10 client.26711 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1585304259 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f73b768e10 con 0x55f73b81dc50 2023-09-23T09:44:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5bf8ff9700 10 client.26711 handle_command_reply: tid=0 2023-09-23T09:44:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 client.26711 shutdown 2023-09-23T09:44:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 msgr2=0x55f73b82dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b82dc80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c00096720 tx=0x7f5c000109a0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 shutdown_connections 2023-09-23T09:44:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b81dc50 0x55f73b820070 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b82dc80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b82d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1585304259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b825530 0x55f73b82d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5bf8ff9700 0 client.26711 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 >> 172.21.15.45:0/1585304259 conn(0x55f73b791260 msgr2=0x55f73b818320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 shutdown_connections 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.140+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1585304259 wait complete. 2023-09-23T09:44:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.141+0000 7f5c235f7b80 20 client.26711 trim_cache size 0 max 16384 2023-09-23T09:44:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 Processor -- start 2023-09-23T09:44:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 -- start start 2023-09-23T09:44:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b827c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b82a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b5f0920 con 0x55f73b8303e0 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b8175b0 con 0x55f73b825530 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b825940 con 0x55f73b824c10 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c151a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b827c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b82a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c159aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b82a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36974/0 (socket says 172.21.15.45:36974) 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c159aa700 1 -- 172.21.15.45:0/1761750258 learned_addr learned my addr 172.21.15.45:0/1761750258 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c151a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b827c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39076/0 (socket says 172.21.15.45:39076) 2023-09-23T09:44:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.163+0000 7f5c149a8700 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5c159aa700 1 -- 172.21.15.45:0/1761750258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 msgr2=0x55f73b827c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5c159aa700 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b827c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5c159aa700 1 -- 172.21.15.45:0/1761750258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 msgr2=0x55f73b82d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5c159aa700 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5c159aa700 1 -- 172.21.15.45:0/1761750258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f73b82cd70 con 0x55f73b8303e0 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5c159aa700 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b82a4d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5c000961b0 tx=0x7f5c00004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dc9ce4760f0219 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5bfbfff700 1 -- 172.21.15.45:0/1761750258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0004fe80 con 0x55f73b8303e0 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5bfbfff700 1 -- 172.21.15.45:0/1761750258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c00009180 con 0x55f73b8303e0 2023-09-23T09:44:10.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.164+0000 7f5bfbfff700 1 -- 172.21.15.45:0/1761750258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000a1d00 con 0x55f73b8303e0 2023-09-23T09:44:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1761750258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 msgr2=0x55f73b82a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b82a4d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5c000961b0 tx=0x7f5c00004860 comp rx=0 tx=0).stop 2023-09-23T09:44:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1761750258 shutdown_connections 2023-09-23T09:44:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b8303e0 0x55f73b82a4d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b824c10 0x55f73b827c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1761750258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b825530 0x55f73b82d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1761750258 >> 172.21.15.45:0/1761750258 conn(0x55f73b791260 msgr2=0x55f73b82d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1761750258 shutdown_connections 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1761750258 wait complete. 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 -- start start 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.165+0000 7f5c235f7b80 1 Processor -- start 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b824c10 0x55f73b835100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b825530 0x55f73b835640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c235f7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b89b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b835de0 con 0x55f73b824c10 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b89b5a0 con 0x55f73b8303e0 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c235f7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73b89b760 con 0x55f73b825530 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c235f7b80 10 client.0 _ll_register_callbacks cb 0x7f5c0c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b824c10 0x55f73b835100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b824c10 0x55f73b835100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36976/0 (socket says 172.21.15.45:36976) 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c149a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b825530 0x55f73b835640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 -- 172.21.15.45:0/1219056872 learned_addr learned my addr 172.21.15.45:0/1219056872 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:10.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c159aa700 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b89b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 -- 172.21.15.45:0/1219056872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b825530 msgr2=0x55f73b835640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b825530 0x55f73b835640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 -- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 msgr2=0x55f73b89b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b89b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 -- 172.21.15.45:0/1219056872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f73b59ade0 con 0x55f73b824c10 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5c151a9700 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b824c10 0x55f73b835100 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c006010 tx=0x7f5c0c029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6b79e52c7580df server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.166+0000 7f5bf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1219056872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c004c00 con 0x55f73b824c10 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c000a2040 con 0x55f73b824c10 2023-09-23T09:44:10.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1219056872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5c0c004d60 con 0x55f73b824c10 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1219056872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c0c0317a0 con 0x55f73b824c10 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 10 client.17076 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55f73b583c60 con 0x55f73b824c10 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1219056872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c000a2040 con 0x55f73b824c10 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1219056872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5c0c0055f0 con 0x55f73b824c10 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 10 client.17076 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 20 client.17076 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b89cf30 0x55f73b89f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 4 client.17076 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.167+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f73b768b30 con 0x55f73b89cf30 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.168+0000 7f5c149a8700 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b89cf30 0x55f73b89f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c149a8700 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b89cf30 0x55f73b89f350 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a55b9347a9b191 server_cookie=2f9cd63e84f89db4 in_seq=0 out_seq=0 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5bf8ff9700 10 client.17076 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5bf8ff9700 1 -- 172.21.15.45:0/1219056872 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f73b768b30 con 0x55f73b89cf30 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5bf8ff9700 10 client.17076 handle_command_reply: tid=0 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 client.17076 shutdown 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b824c10 msgr2=0x55f73b835100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b824c10 0x55f73b835100 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c006010 tx=0x7f5c0c029000 comp rx=0 tx=0).stop 2023-09-23T09:44:10.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 shutdown_connections 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55f73b89cf30 0x55f73b89f350 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f73b8303e0 0x55f73b89b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5bf8ff9700 0 client.17076 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f73b824c10 0x55f73b835100 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 --2- 172.21.15.45:0/1219056872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73b825530 0x55f73b835640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 >> 172.21.15.45:0/1219056872 conn(0x55f73b791260 msgr2=0x55f73b82c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 shutdown_connections 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 1 -- 172.21.15.45:0/1219056872 wait complete. 2023-09-23T09:44:10.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.169+0000 7f5c235f7b80 20 client.17076 trim_cache size 0 max 16384 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:44:10.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:44:10.179 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:44:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5bfc0a1750 msgr2=0x7f5bfc0a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5bfc0a1750 0x7f5bfc0a3c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c003730 tx=0x7f5c0c00b810 comp rx=0 tx=0).stop 2023-09-23T09:44:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10115e20 msgr2=0x7f5c101a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10115e20 0x7f5c101a1840 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000b410 tx=0x7f5c0000b7d0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 shutdown_connections 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f5bfc0a1750 0x7f5bfc0a3c10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c10115e20 0x7f5c101a1840 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c10108360 0x7f5c101a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 --2- 172.21.15.45:0/4059766623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1010b490 0x7f5c101a7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 >> 172.21.15.45:0/4059766623 conn(0x7f5c1006d0e0 msgr2=0x7f5c10116200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 shutdown_connections 2023-09-23T09:44:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:10.170+0000 7f5c1740d700 1 -- 172.21.15.45:0/4059766623 wait complete. 2023-09-23T09:44:10.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:09 smithi045 ceph-mon[98627]: pgmap v462: 97 pgs: 34 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 50 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2023-09-23T09:44:10.186 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:44:10.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:09 smithi098 ceph-mon[102386]: pgmap v462: 97 pgs: 34 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 50 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2023-09-23T09:44:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:09 smithi123 ceph-mon[103577]: pgmap v462: 97 pgs: 34 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 50 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2023-09-23T09:44:11.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 865.0 s, 1267.6 tps, lat 79.355 ms stddev 96.290 2023-09-23T09:44:12.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:11 smithi098 ceph-mon[102386]: pgmap v463: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.98k op/s 2023-09-23T09:44:12.335 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:11 smithi045 ceph-mon[98627]: pgmap v463: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.98k op/s 2023-09-23T09:44:12.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:11 smithi123 ceph-mon[103577]: pgmap v463: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.98k op/s 2023-09-23T09:44:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/4276958646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100b10 msgr2=0x7f8fd4104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/4276958646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100b10 0x7f8fd4104920 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00099c0 tx=0x7f8fd0009cd0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/4276958646 shutdown_connections 2023-09-23T09:44:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/4276958646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd4105050 0x7f8fd410cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/4276958646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd4100160 0x7f8fd4100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/4276958646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100b10 0x7f8fd4104920 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/4276958646 >> 172.21.15.45:0/4276958646 conn(0x7f8fd4076840 msgr2=0x7f8fd4076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/4276958646 shutdown_connections 2023-09-23T09:44:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.761+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/4276958646 wait complete. 2023-09-23T09:44:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 Processor -- start 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 -- start start 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100160 0x7f8fd4195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd4105050 0x7f8fd41961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd419aab0 0x7f8fd419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd419b560 con 0x7f8fd4100160 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd419b6d0 con 0x7f8fd419aab0 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.762+0000 7f8fdbe2c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd419b890 con 0x7f8fd4105050 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100160 0x7f8fd4195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100160 0x7f8fd4195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36990/0 (socket says 172.21.15.45:36990) 2023-09-23T09:44:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/985366639 learned_addr learned my addr 172.21.15.45:0/985366639 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd419aab0 0x7f8fd419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/985366639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd4105050 msgr2=0x7f8fd41961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd4105050 0x7f8fd41961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/985366639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd419aab0 msgr2=0x7f8fd419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd419aab0 0x7f8fd419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/985366639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00095f0 con 0x7f8fd4100160 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100160 0x7f8fd4195c40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d950 tx=0x7f8fc400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fcaffd700 1 -- 172.21.15.45:0/985366639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4009b40 con 0x7f8fd4100160 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fcaffd700 1 -- 172.21.15.45:0/985366639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc4009ca0 con 0x7f8fd4100160 2023-09-23T09:44:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd419bb10 con 0x7f8fd4100160 2023-09-23T09:44:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.763+0000 7f8fcaffd700 1 -- 172.21.15.45:0/985366639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400f620 con 0x7f8fd4100160 2023-09-23T09:44:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.764+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd419c020 con 0x7f8fd4100160 2023-09-23T09:44:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.767+0000 7f8fcaffd700 1 -- 172.21.15.45:0/985366639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f8fc4010460 con 0x7f8fd4100160 2023-09-23T09:44:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 Processor -- start 2023-09-23T09:44:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 -- start start 2023-09-23T09:44:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0ef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0ef860 0x55a97f0fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f058400 con 0x55a97f0ef860 2023-09-23T09:44:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f049bc0 con 0x55a97f0fa710 2023-09-23T09:44:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.769+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f0474e0 con 0x55a97f0eef40 2023-09-23T09:44:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0ef860 0x55a97f0fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0ef860 0x55a97f0fa1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37000/0 (socket says 172.21.15.45:37000) 2023-09-23T09:44:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1833132493 learned_addr learned my addr 172.21.15.45:0/1833132493 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd93c7700 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1833132493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 msgr2=0x55a97f0ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1833132493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 msgr2=0x55a97f0fcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0fcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1833132493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a97f0a5870 con 0x55a97f0ef860 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.770+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0ef860 0x55a97f0fa1d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40ca9f0 tx=0x7f8fc40d56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986bf7234ed0590 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.771+0000 7f8fcaffd700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8fc00a1710 0x7f8fc00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.771+0000 7f8fcaffd700 1 -- 172.21.15.45:0/985366639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8fc40cd3a0 con 0x7f8fd4100160 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.771+0000 7f8fbffff700 1 -- 172.21.15.45:0/1833132493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4009e10 con 0x55a97f0ef860 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.771+0000 7f8fbffff700 1 -- 172.21.15.45:0/1833132493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc40965e0 con 0x55a97f0ef860 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.771+0000 7f8fbffff700 1 -- 172.21.15.45:0/1833132493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4099c00 con 0x55a97f0ef860 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.772+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8fc00a1710 0x7f8fc00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.772+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1833132493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0ef860 msgr2=0x55a97f0fa1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.772+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0ef860 0x55a97f0fa1d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40ca9f0 tx=0x7f8fc40d56b0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8fc00a1710 0x7f8fc00a3bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0076f0 tx=0x7f8fcc00b040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1833132493 shutdown_connections 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0fcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1833132493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0ef860 0x55a97f0fa1d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1833132493 >> 172.21.15.45:0/1833132493 conn(0x55a97f064200 msgr2=0x55a97f0eb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1833132493 shutdown_connections 2023-09-23T09:44:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.773+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1833132493 wait complete. 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 -- start start 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 Processor -- start 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0eef40 0x55a97f0fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f042dd0 con 0x55a97f0fa710 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97efdf6b0 con 0x55a97f0eef40 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97efdf350 con 0x55a97f0ef860 2023-09-23T09:44:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.774+0000 7f8fe8016b80 10 client.0 _ll_register_callbacks cb 0x55a97ee3c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0eef40 0x55a97f0fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0eef40 0x55a97f0fa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58354/0 (socket says 172.21.15.45:58354) 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1048484950 learned_addr learned my addr 172.21.15.45:0/1048484950 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd93c7700 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1048484950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 msgr2=0x55a97f0f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1048484950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 msgr2=0x55a97f0f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 -- 172.21.15.45:0/1048484950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc009050 con 0x55a97f0eef40 2023-09-23T09:44:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.775+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0eef40 0x55a97f0fa150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40a2100 tx=0x7f8fc40d5f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae53686f12924da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.776+0000 7f8fbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.776+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/1048484950 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4009e10 con 0x55a97f0eef40 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.776+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/1048484950 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8fc409a9a0 con 0x55a97f0eef40 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.776+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a97f0e3090 con 0x55a97f0eef40 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.776+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/1048484950 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc4099c00 con 0x55a97f0eef40 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.776+0000 7f8fe8016b80 10 client.26729 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.776+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f8fd419c020 con 0x55a97f0eef40 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/1048484950 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a97f0e3090 con 0x55a97f0eef40 2023-09-23T09:44:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/1048484950 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8fc40cad00 con 0x55a97f0eef40 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fe8016b80 10 client.26729 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fe8016b80 10 client.26729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fe8016b80 20 client.26729 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f109dc0 0x55a97f10c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fe8016b80 4 client.26729 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a97f03bdb0 con 0x55a97f109dc0 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.777+0000 7f8fd93c7700 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f109dc0 0x55a97f10c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.778+0000 7f8fbd7fa700 10 client.26729 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.778+0000 7f8fd93c7700 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f109dc0 0x55a97f10c1e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7339fbbf7a97824b server_cookie=d742b9025df92f84 in_seq=0 out_seq=0 2023-09-23T09:44:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/1048484950 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a97f03bdb0 con 0x55a97f109dc0 2023-09-23T09:44:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fbd7fa700 10 client.26729 handle_command_reply: tid=0 2023-09-23T09:44:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 client.26729 shutdown 2023-09-23T09:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0eef40 msgr2=0x55a97f0fa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0eef40 0x55a97f0fa150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40a2100 tx=0x7f8fc40d5f90 comp rx=0 tx=0).stop 2023-09-23T09:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 shutdown_connections 2023-09-23T09:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f109dc0 0x55a97f10c1e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0eef40 0x55a97f0fa150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1048484950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 >> 172.21.15.45:0/1048484950 conn(0x55a97f064200 msgr2=0x55a97f0eaf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.781+0000 7f8fbd7fa700 0 client.26729 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.782+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 shutdown_connections 2023-09-23T09:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.782+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1048484950 wait complete. 2023-09-23T09:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.782+0000 7f8fe8016b80 20 client.26729 trim_cache size 0 max 16384 2023-09-23T09:44:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 Processor -- start 2023-09-23T09:44:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 -- start start 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0f7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0ef860 0x55a97f0f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97eec38c0 con 0x55a97f0fa710 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f0ea1a0 con 0x55a97f0ef860 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f0f8950 con 0x55a97f0eef40 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.806+0000 7f8fda3c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fd93c7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0ef860 0x55a97f0f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37016/0 (socket says 172.21.15.45:37016) 2023-09-23T09:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fd93c7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0ef860 0x55a97f0f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58362/0 (socket says 172.21.15.45:58362) 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 -- 172.21.15.45:0/1419369456 learned_addr learned my addr 172.21.15.45:0/1419369456 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 -- 172.21.15.45:0/1419369456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 msgr2=0x55a97f0f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 -- 172.21.15.45:0/1419369456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0ef860 msgr2=0x55a97f0f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0ef860 0x55a97f0f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 -- 172.21.15.45:0/1419369456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a97ee6dd80 con 0x55a97f0fa710 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f9cf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc008f10 tx=0x7f8fcc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46580bef63cb0a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fc8ff9700 1 -- 172.21.15.45:0/1419369456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc00e050 con 0x55a97f0fa710 2023-09-23T09:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fc8ff9700 1 -- 172.21.15.45:0/1419369456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fcc00f040 con 0x55a97f0fa710 2023-09-23T09:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.807+0000 7f8fc8ff9700 1 -- 172.21.15.45:0/1419369456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc022600 con 0x55a97f0fa710 2023-09-23T09:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.808+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1419369456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 msgr2=0x55a97f0f9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.808+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f9cf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc008f10 tx=0x7f8fcc004ad0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.808+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1419369456 shutdown_connections 2023-09-23T09:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.808+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0fa710 0x55a97f0f9cf0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.808+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0eef40 0x55a97f0f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.808+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/1419369456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0ef860 0x55a97f0f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.808+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1419369456 >> 172.21.15.45:0/1419369456 conn(0x55a97f064200 msgr2=0x55a97f10bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.809+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1419369456 shutdown_connections 2023-09-23T09:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.809+0000 7f8fe8016b80 1 -- 172.21.15.45:0/1419369456 wait complete. 2023-09-23T09:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.810+0000 7f8fe8016b80 1 -- start start 2023-09-23T09:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.810+0000 7f8fe8016b80 1 Processor -- start 2023-09-23T09:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.811+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0eef40 0x55a97f0ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fe8016b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f15cc40 con 0x55a97f0eef40 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f15cdb0 con 0x55a97f0fa710 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fe8016b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97f15cf70 con 0x55a97f0ef860 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fe8016b80 10 client.0 _ll_register_callbacks cb 0x7f8fd0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd9bc8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0eef40 0x55a97f0ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39126/0 (socket says 172.21.15.45:39126) 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 -- 172.21.15.45:0/3347101992 learned_addr learned my addr 172.21.15.45:0/3347101992 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fda3c9700 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 -- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 msgr2=0x55a97f0ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 -- 172.21.15.45:0/3347101992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0eef40 msgr2=0x55a97f0ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0eef40 0x55a97f0ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.812+0000 7f8fd93c7700 1 -- 172.21.15.45:0/3347101992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd419c020 con 0x55a97f0ef860 2023-09-23T09:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.813+0000 7f8fd93c7700 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0ff780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009990 tx=0x7f8fd0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c534def312c3e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.813+0000 7f8fbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.813+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/3347101992 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd001c040 con 0x55a97f0ef860 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.813+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/3347101992 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8fd0028ce0 con 0x55a97f0ef860 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.813+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fc40965e0 con 0x55a97f0ef860 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.813+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/3347101992 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd000b930 con 0x55a97f0ef860 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 10 client.26746 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f8fcc008c80 con 0x55a97f0ef860 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/3347101992 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc40965e0 con 0x55a97f0ef860 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/3347101992 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8fd00054c0 con 0x55a97f0ef860 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 10 client.26746 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 10 client.26746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 20 client.26746 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f15e670 0x55a97f160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 4 client.26746 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a97f03bad0 con 0x55a97f15e670 2023-09-23T09:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.814+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f15e670 0x55a97f160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.815+0000 7f8fbd7fa700 10 client.26746 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.815+0000 7f8fd9bc8700 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f15e670 0x55a97f160a90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73cb7fb31b2d41c3 server_cookie=2e3503527c092ef in_seq=0 out_seq=0 2023-09-23T09:44:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fbd7fa700 1 -- 172.21.15.45:0/3347101992 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a97f03bad0 con 0x55a97f15e670 2023-09-23T09:44:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fbd7fa700 10 client.26746 handle_command_reply: tid=0 2023-09-23T09:44:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 client.26746 shutdown 2023-09-23T09:44:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 msgr2=0x55a97f0ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0ff780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009990 tx=0x7f8fd0000bc0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 shutdown_connections 2023-09-23T09:44:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a97f15e670 0x55a97f160a90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a97f0fa710 0x55a97f0ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a97f0eef40 0x55a97f0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fbd7fa700 0 client.26746 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 --2- 172.21.15.45:0/3347101992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a97f0ef860 0x55a97f0ff780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 >> 172.21.15.45:0/3347101992 conn(0x55a97f064200 msgr2=0x55a97f0f2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 shutdown_connections 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 1 -- 172.21.15.45:0/3347101992 wait complete. 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.852+0000 7f8fe8016b80 20 client.26746 trim_cache size 0 max 16384 2023-09-23T09:44:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:44:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:44:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:44:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:44:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8fc00a1710 msgr2=0x7f8fc00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8fc00a1710 0x7f8fc00a3bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0076f0 tx=0x7f8fcc00b040 comp rx=0 tx=0).stop 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100160 msgr2=0x7f8fd4195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100160 0x7f8fd4195c40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d950 tx=0x7f8fc400dd10 comp rx=0 tx=0).stop 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 shutdown_connections 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd419aab0 0x7f8fd419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8fc00a1710 0x7f8fc00a3bd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100160 0x7f8fd4195c40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 --2- 172.21.15.45:0/985366639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd4105050 0x7f8fd41961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 >> 172.21.15.45:0/985366639 conn(0x7f8fd4076840 msgr2=0x7f8fd40fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 shutdown_connections 2023-09-23T09:44:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:13.853+0000 7f8fdbe2c700 1 -- 172.21.15.45:0/985366639 wait complete. 2023-09-23T09:44:14.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:13 smithi098 ceph-mon[102386]: pgmap v464: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.98k op/s 2023-09-23T09:44:14.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:13 smithi045 ceph-mon[98627]: pgmap v464: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.98k op/s 2023-09-23T09:44:14.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:13 smithi123 ceph-mon[103577]: pgmap v464: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.98k op/s 2023-09-23T09:44:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:16 smithi098 ceph-mon[102386]: pgmap v465: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-09-23T09:44:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:16 smithi098 ceph-mon[102386]: osdmap e105: 12 total, 12 up, 12 in 2023-09-23T09:44:16.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:16 smithi045 ceph-mon[98627]: pgmap v465: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-09-23T09:44:16.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:16 smithi045 ceph-mon[98627]: osdmap e105: 12 total, 12 up, 12 in 2023-09-23T09:44:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:16 smithi123 ceph-mon[103577]: pgmap v465: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-09-23T09:44:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:16 smithi123 ceph-mon[103577]: osdmap e105: 12 total, 12 up, 12 in 2023-09-23T09:44:16.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 870.0 s, 1427.6 tps, lat 70.832 ms stddev 98.355 2023-09-23T09:44:18.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:18 smithi045 ceph-mon[98627]: pgmap v467: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 658 op/s 2023-09-23T09:44:18.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:18 smithi123 ceph-mon[103577]: pgmap v467: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 658 op/s 2023-09-23T09:44:18.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:18 smithi098 ceph-mon[102386]: pgmap v467: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 658 op/s 2023-09-23T09:44:20.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:20 smithi045 ceph-mon[98627]: pgmap v468: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 693 op/s 2023-09-23T09:44:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:20 smithi123 ceph-mon[103577]: pgmap v468: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 693 op/s 2023-09-23T09:44:20.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:20 smithi098 ceph-mon[102386]: pgmap v468: 97 pgs: 97 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 693 op/s 2023-09-23T09:44:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 875.0 s, 1301.7 tps, lat 75.269 ms stddev 97.103 2023-09-23T09:44:22.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:22 smithi098 ceph-mon[102386]: pgmap v469: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-09-23T09:44:22.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:22 smithi045 ceph-mon[98627]: pgmap v469: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-09-23T09:44:22.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:22 smithi123 ceph-mon[103577]: pgmap v469: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-09-23T09:44:23.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:23 smithi098 ceph-mon[102386]: pgmap v470: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-09-23T09:44:23.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:23 smithi045 ceph-mon[98627]: pgmap v470: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-09-23T09:44:23.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:23 smithi123 ceph-mon[103577]: pgmap v470: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-09-23T09:44:25.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:25 smithi098 ceph-mon[102386]: pgmap v471: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 20 MiB/s wr, 1.70k op/s 2023-09-23T09:44:25.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:25 smithi045 ceph-mon[98627]: pgmap v471: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 20 MiB/s wr, 1.70k op/s 2023-09-23T09:44:25.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:25 smithi123 ceph-mon[103577]: pgmap v471: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 20 MiB/s wr, 1.70k op/s 2023-09-23T09:44:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 880.0 s, 795.4 tps, lat 127.432 ms stddev 141.002 2023-09-23T09:44:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 -- 172.21.15.45:0/3755173544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520100160 msgr2=0x7fa520100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 --2- 172.21.15.45:0/3755173544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520100160 0x7fa520100540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa5100099c0 tx=0x7fa510009cd0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 -- 172.21.15.45:0/3755173544 shutdown_connections 2023-09-23T09:44:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 --2- 172.21.15.45:0/3755173544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520105050 0x7fa52010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 --2- 172.21.15.45:0/3755173544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520100160 0x7fa520100540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 --2- 172.21.15.45:0/3755173544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa520100b10 0x7fa520104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 -- 172.21.15.45:0/3755173544 >> 172.21.15.45:0/3755173544 conn(0x7fa520076840 msgr2=0x7fa520076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.102+0000 7fa524df2700 1 -- 172.21.15.45:0/3755173544 shutdown_connections 2023-09-23T09:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.106+0000 7fa524df2700 1 -- 172.21.15.45:0/3755173544 wait complete. 2023-09-23T09:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.106+0000 7fa524df2700 1 Processor -- start 2023-09-23T09:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.107+0000 7fa524df2700 1 -- start start 2023-09-23T09:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.107+0000 7fa524df2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520100160 0x7fa52010b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.107+0000 7fa524df2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa520105050 0x7fa52010be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.107+0000 7fa524df2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52019dba0 0x7fa52019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.107+0000 7fa524df2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52019e620 con 0x7fa520100160 2023-09-23T09:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.107+0000 7fa524df2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52019e760 con 0x7fa520105050 2023-09-23T09:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.107+0000 7fa524df2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52019e8d0 con 0x7fa52019dba0 2023-09-23T09:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520100160 0x7fa52010b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520100160 0x7fa52010b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37052/0 (socket says 172.21.15.45:37052) 2023-09-23T09:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 -- 172.21.15.45:0/3536772506 learned_addr learned my addr 172.21.15.45:0/3536772506 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa520105050 0x7fa52010be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 -- 172.21.15.45:0/3536772506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52019dba0 msgr2=0x7fa52019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52019dba0 0x7fa52019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 -- 172.21.15.45:0/3536772506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa520105050 msgr2=0x7fa52010be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa520105050 0x7fa52010be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.108+0000 7fa51e59c700 1 -- 172.21.15.45:0/3536772506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100095f0 con 0x7fa520100160 2023-09-23T09:44:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.109+0000 7fa51e59c700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520100160 0x7fa52010b8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b580 tx=0x7fa51000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.119+0000 7fa50f7fe700 1 -- 172.21.15.45:0/3536772506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51001d070 con 0x7fa520100160 2023-09-23T09:44:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.119+0000 7fa50f7fe700 1 -- 172.21.15.45:0/3536772506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa51000fbf0 con 0x7fa520100160 2023-09-23T09:44:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.119+0000 7fa50f7fe700 1 -- 172.21.15.45:0/3536772506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100173f0 con 0x7fa520100160 2023-09-23T09:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.121+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52019aad0 con 0x7fa520100160 2023-09-23T09:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.121+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52019af00 con 0x7fa520100160 2023-09-23T09:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.124+0000 7fa50f7fe700 1 -- 172.21.15.45:0/3536772506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fa510022470 con 0x7fa520100160 2023-09-23T09:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.125+0000 7fa50f7fe700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa5080a18e0 0x7fa5080a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.125+0000 7fa50f7fe700 1 -- 172.21.15.45:0/3536772506 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa5100ce210 con 0x7fa520100160 2023-09-23T09:44:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.126+0000 7fa530fdcb80 1 Processor -- start 2023-09-23T09:44:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.126+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa5080a18e0 0x7fa5080a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.127+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa5080a18e0 0x7fa5080a3da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa520196650 tx=0x7fa514009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.128+0000 7fa530fdcb80 1 -- start start 2023-09-23T09:44:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.128+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.128+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.128+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5db2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.128+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5d0e400 con 0x55a8c5db0710 2023-09-23T09:44:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.128+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5cffbc0 con 0x55a8c5da4f40 2023-09-23T09:44:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.128+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5cfd4e0 con 0x55a8c5da5860 2023-09-23T09:44:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.129+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.129+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58410/0 (socket says 172.21.15.45:58410) 2023-09-23T09:44:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.129+0000 7fa51e59c700 1 -- 172.21.15.45:0/797720693 learned_addr learned my addr 172.21.15.45:0/797720693 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.130+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.130+0000 7fa51dd9b700 1 -- 172.21.15.45:0/797720693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 msgr2=0x55a8c5da5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.130+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.130+0000 7fa51dd9b700 1 -- 172.21.15.45:0/797720693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 msgr2=0x55a8c5db2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.130+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5db2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.130+0000 7fa51dd9b700 1 -- 172.21.15.45:0/797720693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8c5d5b870 con 0x55a8c5da5860 2023-09-23T09:44:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.131+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db01d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa520073d50 tx=0x7fa514013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5433f54f6d8df473 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.131+0000 7fa50cff9700 1 -- 172.21.15.45:0/797720693 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514010ee0 con 0x55a8c5da5860 2023-09-23T09:44:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.131+0000 7fa50cff9700 1 -- 172.21.15.45:0/797720693 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa514004830 con 0x55a8c5da5860 2023-09-23T09:44:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.131+0000 7fa50cff9700 1 -- 172.21.15.45:0/797720693 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5140054e0 con 0x55a8c5da5860 2023-09-23T09:44:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.132+0000 7fa530fdcb80 1 -- 172.21.15.45:0/797720693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 msgr2=0x55a8c5db01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.132+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db01d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa520073d50 tx=0x7fa514013000 comp rx=0 tx=0).stop 2023-09-23T09:44:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.145+0000 7fa530fdcb80 1 -- 172.21.15.45:0/797720693 shutdown_connections 2023-09-23T09:44:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.145+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5db2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.145+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.145+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/797720693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db01d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.145+0000 7fa530fdcb80 1 -- 172.21.15.45:0/797720693 >> 172.21.15.45:0/797720693 conn(0x55a8c5d1a200 msgr2=0x55a8c5da13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.157+0000 7fa530fdcb80 1 -- 172.21.15.45:0/797720693 shutdown_connections 2023-09-23T09:44:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.157+0000 7fa530fdcb80 1 -- 172.21.15.45:0/797720693 wait complete. 2023-09-23T09:44:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 -- start start 2023-09-23T09:44:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 Processor -- start 2023-09-23T09:44:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da4f40 0x55a8c5db0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da5860 0x55a8c5dad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dadab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5cf8dd0 con 0x55a8c5db0710 2023-09-23T09:44:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5c956b0 con 0x55a8c5da5860 2023-09-23T09:44:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5c95350 con 0x55a8c5da4f40 2023-09-23T09:44:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.158+0000 7fa530fdcb80 10 client.0 _ll_register_callbacks cb 0x55a8c5af27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dadab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dadab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37066/0 (socket says 172.21.15.45:37066) 2023-09-23T09:44:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 -- 172.21.15.45:0/229266424 learned_addr learned my addr 172.21.15.45:0/229266424 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 -- 172.21.15.45:0/229266424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da4f40 msgr2=0x55a8c5db0120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:44:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da4f40 0x55a8c5db0120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 -- 172.21.15.45:0/229266424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da5860 msgr2=0x55a8c5dad410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da5860 0x55a8c5dad410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.160+0000 7fa51ed9d700 1 -- 172.21.15.45:0/229266424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa514010830 con 0x55a8c5db0710 2023-09-23T09:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.161+0000 7fa51ed9d700 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dadab0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa51800eb00 tx=0x7fa51800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74b327dd6969029 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.166+0000 7fa505ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:44:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.170+0000 7fa505ffb700 1 -- 172.21.15.45:0/229266424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518018070 con 0x55a8c5db0710 2023-09-23T09:44:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.170+0000 7fa505ffb700 1 -- 172.21.15.45:0/229266424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa5180092e0 con 0x55a8c5db0710 2023-09-23T09:44:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.170+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8c5d99090 con 0x55a8c5db0710 2023-09-23T09:44:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.171+0000 7fa505ffb700 1 -- 172.21.15.45:0/229266424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa518010040 con 0x55a8c5db0710 2023-09-23T09:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.171+0000 7fa530fdcb80 10 client.17100 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.171+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55a8c5b23d80 con 0x55a8c5db0710 2023-09-23T09:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.171+0000 7fa505ffb700 1 -- 172.21.15.45:0/229266424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8c5d99090 con 0x55a8c5db0710 2023-09-23T09:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa505ffb700 1 -- 172.21.15.45:0/229266424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa51801b020 con 0x55a8c5db0710 2023-09-23T09:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa530fdcb80 10 client.17100 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa530fdcb80 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa530fdcb80 20 client.17100 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5dc02d0 0x55a8c5dc26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa530fdcb80 4 client.17100 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8c5cf1db0 con 0x55a8c5dc02d0 2023-09-23T09:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.172+0000 7fa51e59c700 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5dc02d0 0x55a8c5dc26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.173+0000 7fa505ffb700 10 client.17100 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.173+0000 7fa51e59c700 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5dc02d0 0x55a8c5dc26f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67b7a1daf32132b server_cookie=950010601a684021 in_seq=0 out_seq=0 2023-09-23T09:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa505ffb700 1 -- 172.21.15.45:0/229266424 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8c5cf1db0 con 0x55a8c5dc02d0 2023-09-23T09:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa505ffb700 10 client.17100 handle_command_reply: tid=0 2023-09-23T09:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 client.17100 shutdown 2023-09-23T09:44:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 msgr2=0x55a8c5dadab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dadab0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa51800eb00 tx=0x7fa51800eec0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 shutdown_connections 2023-09-23T09:44:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5dc02d0 0x55a8c5dc26f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dadab0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da4f40 0x55a8c5db0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/229266424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da5860 0x55a8c5dad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 >> 172.21.15.45:0/229266424 conn(0x55a8c5d1a200 msgr2=0x55a8c5da0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.174+0000 7fa505ffb700 0 client.17100 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.175+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 shutdown_connections 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.175+0000 7fa530fdcb80 1 -- 172.21.15.45:0/229266424 wait complete. 2023-09-23T09:44:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.175+0000 7fa530fdcb80 20 client.17100 trim_cache size 0 max 16384 2023-09-23T09:44:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 Processor -- start 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 -- start start 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dbb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5b798c0 con 0x55a8c5db0710 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5d98fd0 con 0x55a8c5da4f40 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.200+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5db7850 con 0x55a8c5da5860 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dbb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58424/0 (socket says 172.21.15.45:58424) 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dbb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37076/0 (socket says 172.21.15.45:37076) 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51e59c700 1 -- 172.21.15.45:0/3524860689 learned_addr learned my addr 172.21.15.45:0/3524860689 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 -- 172.21.15.45:0/3524860689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 msgr2=0x55a8c5db7290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db7290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 -- 172.21.15.45:0/3524860689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 msgr2=0x55a8c5da7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 -- 172.21.15.45:0/3524860689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8c5b0cc00 con 0x55a8c5db0710 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51e59c700 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa51ed9d700 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dbb9e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa51800eb00 tx=0x7fa518008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f179da39a7e84af3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa50d7fa700 1 -- 172.21.15.45:0/3524860689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518018070 con 0x55a8c5db0710 2023-09-23T09:44:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa50d7fa700 1 -- 172.21.15.45:0/3524860689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa518006000 con 0x55a8c5db0710 2023-09-23T09:44:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.201+0000 7fa50d7fa700 1 -- 172.21.15.45:0/3524860689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518010040 con 0x55a8c5db0710 2023-09-23T09:44:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.202+0000 7fa530fdcb80 1 -- 172.21.15.45:0/3524860689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 msgr2=0x55a8c5dbb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.202+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dbb9e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa51800eb00 tx=0x7fa518008f90 comp rx=0 tx=0).stop 2023-09-23T09:44:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.202+0000 7fa530fdcb80 1 -- 172.21.15.45:0/3524860689 shutdown_connections 2023-09-23T09:44:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.202+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5db0710 0x55a8c5dbb9e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.202+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5da4f40 0x55a8c5da7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.202+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/3524860689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5da5860 0x55a8c5db7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.202+0000 7fa530fdcb80 1 -- 172.21.15.45:0/3524860689 >> 172.21.15.45:0/3524860689 conn(0x55a8c5d1a200 msgr2=0x55a8c5dc22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.209+0000 7fa530fdcb80 1 -- 172.21.15.45:0/3524860689 shutdown_connections 2023-09-23T09:44:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.209+0000 7fa530fdcb80 1 -- 172.21.15.45:0/3524860689 wait complete. 2023-09-23T09:44:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 -- start start 2023-09-23T09:44:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 Processor -- start 2023-09-23T09:44:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5da4f40 0x55a8c5db5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5db0710 0x55a8c5db57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5db5d40 0x55a8c5e13100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5e136e0 con 0x55a8c5da4f40 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5e13850 con 0x55a8c5db5d40 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5e13a10 con 0x55a8c5db0710 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa530fdcb80 10 client.0 _ll_register_callbacks cb 0x7fa51800e460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5da4f40 0x55a8c5db5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa51e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5da4f40 0x55a8c5db5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37078/0 (socket says 172.21.15.45:37078) 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa51e59c700 1 -- 172.21.15.45:0/391701651 learned_addr learned my addr 172.21.15.45:0/391701651 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.210+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5db0710 0x55a8c5db57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa51e59c700 1 -- 172.21.15.45:0/391701651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5db0710 msgr2=0x55a8c5db57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa51e59c700 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5db0710 0x55a8c5db57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa51e59c700 1 -- 172.21.15.45:0/391701651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5db5d40 msgr2=0x55a8c5e13100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa51e59c700 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5db5d40 0x55a8c5e13100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa51e59c700 1 -- 172.21.15.45:0/391701651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52019af00 con 0x55a8c5da4f40 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa51e59c700 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5da4f40 0x55a8c5db5270 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa510097670 tx=0x7fa510097590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e7714851e345e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa505ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa505ffb700 1 -- 172.21.15.45:0/391701651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51009f2c0 con 0x55a8c5da4f40 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa505ffb700 1 -- 172.21.15.45:0/391701651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa5100a3470 con 0x55a8c5da4f40 2023-09-23T09:44:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.211+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa514004830 con 0x55a8c5da4f40 2023-09-23T09:44:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.212+0000 7fa505ffb700 1 -- 172.21.15.45:0/391701651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5100b9350 con 0x55a8c5da4f40 2023-09-23T09:44:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.212+0000 7fa530fdcb80 10 client.17106 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.212+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55a8c5b23d80 con 0x55a8c5da4f40 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.212+0000 7fa505ffb700 1 -- 172.21.15.45:0/391701651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa514004830 con 0x55a8c5da4f40 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.212+0000 7fa505ffb700 1 -- 172.21.15.45:0/391701651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa5100b9070 con 0x55a8c5da4f40 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.213+0000 7fa530fdcb80 10 client.17106 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.213+0000 7fa530fdcb80 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.213+0000 7fa530fdcb80 20 client.17106 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.213+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5e14e30 0x55a8c5e17250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.213+0000 7fa530fdcb80 4 client.17106 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.213+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8c5cf1ad0 con 0x55a8c5e14e30 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.213+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5e14e30 0x55a8c5e17250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.214+0000 7fa505ffb700 10 client.17106 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.214+0000 7fa51dd9b700 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5e14e30 0x55a8c5e17250 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3e58b4676bb7ed server_cookie=fc719ee71d2dc47 in_seq=0 out_seq=0 2023-09-23T09:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.279+0000 7fa505ffb700 1 -- 172.21.15.45:0/391701651 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8c5cf1ad0 con 0x55a8c5e14e30 2023-09-23T09:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.279+0000 7fa505ffb700 10 client.17106 handle_command_reply: tid=0 2023-09-23T09:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.279+0000 7fa530fdcb80 1 client.17106 shutdown 2023-09-23T09:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.279+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5da4f40 msgr2=0x55a8c5db5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.279+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5da4f40 0x55a8c5db5270 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa510097670 tx=0x7fa510097590 comp rx=0 tx=0).stop 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 shutdown_connections 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55a8c5e14e30 0x55a8c5e17250 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8c5db5d40 0x55a8c5e13100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8c5da4f40 0x55a8c5db5270 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa505ffb700 0 client.17106 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 --2- 172.21.15.45:0/391701651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8c5db0710 0x55a8c5db57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 >> 172.21.15.45:0/391701651 conn(0x55a8c5d1a200 msgr2=0x55a8c5dbb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 shutdown_connections 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 1 -- 172.21.15.45:0/391701651 wait complete. 2023-09-23T09:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.280+0000 7fa530fdcb80 20 client.17106 trim_cache size 0 max 16384 2023-09-23T09:44:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:44:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:44:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:44:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:44:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.281+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa5080a18e0 msgr2=0x7fa5080a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.281+0000 7fa524df2700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa5080a18e0 0x7fa5080a3da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa520196650 tx=0x7fa514009260 comp rx=0 tx=0).stop 2023-09-23T09:44:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.281+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520100160 msgr2=0x7fa52010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.281+0000 7fa524df2700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520100160 0x7fa52010b8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b580 tx=0x7fa51000bfd0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 shutdown_connections 2023-09-23T09:44:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52019dba0 0x7fa52019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa5080a18e0 0x7fa5080a3da0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520100160 0x7fa52010b8e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 --2- 172.21.15.45:0/3536772506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa520105050 0x7fa52010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 >> 172.21.15.45:0/3536772506 conn(0x7fa520076840 msgr2=0x7fa5200fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 shutdown_connections 2023-09-23T09:44:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:27.283+0000 7fa524df2700 1 -- 172.21.15.45:0/3536772506 wait complete. 2023-09-23T09:44:28.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:27 smithi098 ceph-mon[102386]: pgmap v472: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-09-23T09:44:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:27 smithi045 ceph-mon[98627]: pgmap v472: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-09-23T09:44:28.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:27 smithi123 ceph-mon[103577]: pgmap v472: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-09-23T09:44:30.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:29 smithi098 ceph-mon[102386]: pgmap v473: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-09-23T09:44:30.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:29 smithi045 ceph-mon[98627]: pgmap v473: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-09-23T09:44:30.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:29 smithi123 ceph-mon[103577]: pgmap v473: 97 pgs: 97 active+clean; 5.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-09-23T09:44:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 885.0 s, 1114.6 tps, lat 89.631 ms stddev 95.935 2023-09-23T09:44:32.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:31 smithi098 ceph-mon[102386]: pgmap v474: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2023-09-23T09:44:32.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:31 smithi045 ceph-mon[98627]: pgmap v474: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2023-09-23T09:44:32.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:31 smithi123 ceph-mon[103577]: pgmap v474: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2023-09-23T09:44:34.300 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:33 smithi098 ceph-mon[102386]: pgmap v475: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 741 op/s 2023-09-23T09:44:34.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:33 smithi045 ceph-mon[98627]: pgmap v475: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 741 op/s 2023-09-23T09:44:34.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:33 smithi123 ceph-mon[103577]: pgmap v475: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 741 op/s 2023-09-23T09:44:36.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:35 smithi098 ceph-mon[102386]: pgmap v476: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 850 op/s 2023-09-23T09:44:36.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:35 smithi123 ceph-mon[103577]: pgmap v476: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 850 op/s 2023-09-23T09:44:36.340 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:35 smithi045 ceph-mon[98627]: pgmap v476: 97 pgs: 97 active+clean; 5.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 850 op/s 2023-09-23T09:44:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 890.0 s, 1081.0 tps, lat 92.028 ms stddev 108.954 2023-09-23T09:44:38.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:37 smithi098 ceph-mon[102386]: pgmap v477: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.19k op/s 2023-09-23T09:44:38.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:37 smithi045 ceph-mon[98627]: pgmap v477: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.19k op/s 2023-09-23T09:44:38.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:37 smithi123 ceph-mon[103577]: pgmap v477: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.19k op/s 2023-09-23T09:44:40.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:44:40.255 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:39 smithi045 ceph-mon[98627]: pgmap v478: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-09-23T09:44:40.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:39 smithi098 ceph-mon[102386]: pgmap v478: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-09-23T09:44:40.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:39 smithi123 ceph-mon[103577]: pgmap v478: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-09-23T09:44:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.414+0000 7f080eecb700 1 -- 172.21.15.45:0/4255788622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0808102910 msgr2=0x7f080810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.414+0000 7f080eecb700 1 --2- 172.21.15.45:0/4255788622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0808102910 0x7f080810adf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009990 tx=0x7f07f4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.414+0000 7f080eecb700 1 -- 172.21.15.45:0/4255788622 shutdown_connections 2023-09-23T09:44:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.414+0000 7f080eecb700 1 --2- 172.21.15.45:0/4255788622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f080810b3c0 0x7f080810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.414+0000 7f080eecb700 1 --2- 172.21.15.45:0/4255788622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0808101ff0 0x7f08081023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.414+0000 7f080eecb700 1 --2- 172.21.15.45:0/4255788622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0808102910 0x7f080810adf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.414+0000 7f080eecb700 1 -- 172.21.15.45:0/4255788622 >> 172.21.15.45:0/4255788622 conn(0x7f08080fb770 msgr2=0x7f08080fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.415+0000 7f080eecb700 1 -- 172.21.15.45:0/4255788622 shutdown_connections 2023-09-23T09:44:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.415+0000 7f080eecb700 1 -- 172.21.15.45:0/4255788622 wait complete. 2023-09-23T09:44:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.415+0000 7f080eecb700 1 Processor -- start 2023-09-23T09:44:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.415+0000 7f080eecb700 1 -- start start 2023-09-23T09:44:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080eecb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0808101ff0 0x7f0808197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080eecb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0808102910 0x7f08081975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080eecb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f080810b3c0 0x7f080819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080eecb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080819c050 con 0x7f0808101ff0 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080eecb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080819c1c0 con 0x7f0808102910 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080eecb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080819c380 con 0x7f080810b3c0 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0808101ff0 0x7f0808197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0808101ff0 0x7f0808197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37094/0 (socket says 172.21.15.45:37094) 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 -- 172.21.15.45:0/1258826238 learned_addr learned my addr 172.21.15.45:0/1258826238 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f0807fff700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0808102910 0x7f08081975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080d468700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f080810b3c0 0x7f080819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 -- 172.21.15.45:0/1258826238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f080810b3c0 msgr2=0x7f080819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f080810b3c0 0x7f080819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 -- 172.21.15.45:0/1258826238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0808102910 msgr2=0x7f08081975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0808102910 0x7f08081975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 -- 172.21.15.45:0/1258826238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f40095f0 con 0x7f0808101ff0 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.416+0000 7f080cc67700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0808101ff0 0x7f0808197070 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00b550 tx=0x7f07fc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.417+0000 7f0805ffb700 1 -- 172.21.15.45:0/1258826238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc011830 con 0x7f0808101ff0 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.417+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080819c660 con 0x7f0808101ff0 2023-09-23T09:44:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.417+0000 7f0805ffb700 1 -- 172.21.15.45:0/1258826238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07fc011e50 con 0x7f0808101ff0 2023-09-23T09:44:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.417+0000 7f0805ffb700 1 -- 172.21.15.45:0/1258826238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc010e40 con 0x7f0808101ff0 2023-09-23T09:44:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.417+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08081a3ea0 con 0x7f0808101ff0 2023-09-23T09:44:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.420+0000 7f0805ffb700 1 -- 172.21.15.45:0/1258826238 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f07fc010380 con 0x7f0808101ff0 2023-09-23T09:44:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.421+0000 7f0805ffb700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07f80a1710 0x7f07f80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.421+0000 7f0805ffb700 1 -- 172.21.15.45:0/1258826238 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f07fc0ccc70 con 0x7f0808101ff0 2023-09-23T09:44:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.421+0000 7f0807fff700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07f80a1710 0x7f07f80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.421+0000 7f0807fff700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07f80a1710 0x7f07f80a3bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f07f400b5c0 tx=0x7f07f4005ab0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 Processor -- start 2023-09-23T09:44:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 -- start start 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba538f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538fd20 0x55eba539a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba539abd0 0x55eba539cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba52f8460 con 0x55eba539abd0 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba52e9c20 con 0x55eba538fd20 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.424+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba52e7540 con 0x55eba538f400 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba538f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f0807fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538fd20 0x55eba539a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba538f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39208/0 (socket says 172.21.15.45:39208) 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 -- 172.21.15.45:0/1239939007 learned_addr learned my addr 172.21.15.45:0/1239939007 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080d468700 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba539abd0 0x55eba539cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 -- 172.21.15.45:0/1239939007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538fd20 msgr2=0x55eba539a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538fd20 0x55eba539a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 -- 172.21.15.45:0/1239939007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba539abd0 msgr2=0x55eba539cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba539abd0 0x55eba539cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 -- 172.21.15.45:0/1239939007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba53458d0 con 0x55eba538f400 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f0807fff700 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538fd20 0x55eba539a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f080cc67700 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba538f7e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f07fc096720 tx=0x7f07fc099ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a64cd351663cb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f07f2ffd700 1 -- 172.21.15.45:0/1239939007 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc099cb0 con 0x55eba538f400 2023-09-23T09:44:40.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f07f2ffd700 1 -- 172.21.15.45:0/1239939007 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07fc09de00 con 0x55eba538f400 2023-09-23T09:44:40.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.425+0000 7f07f2ffd700 1 -- 172.21.15.45:0/1239939007 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0a1bb0 con 0x55eba538f400 2023-09-23T09:44:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 -- 172.21.15.45:0/1239939007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 msgr2=0x55eba538f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba538f7e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f07fc096720 tx=0x7f07fc099ba0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 -- 172.21.15.45:0/1239939007 shutdown_connections 2023-09-23T09:44:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba539abd0 0x55eba539cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba538f7e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/1239939007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538fd20 0x55eba539a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 -- 172.21.15.45:0/1239939007 >> 172.21.15.45:0/1239939007 conn(0x55eba5304260 msgr2=0x55eba538b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 -- 172.21.15.45:0/1239939007 shutdown_connections 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 -- 172.21.15.45:0/1239939007 wait complete. 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.426+0000 7f081b0b5b80 1 -- start start 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 1 Processor -- start 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538f400 0x55eba53979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba5397ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba539abd0 0x55eba5398430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba52e2e30 con 0x55eba538fd20 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba527f710 con 0x55eba538f400 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba527f3b0 con 0x55eba539abd0 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.427+0000 7f081b0b5b80 10 client.0 _ll_register_callbacks cb 0x55eba50dc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba539abd0 0x55eba5398430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba539abd0 0x55eba5398430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39216/0 (socket says 172.21.15.45:39216) 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 -- 172.21.15.45:0/2428415836 learned_addr learned my addr 172.21.15.45:0/2428415836 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f0807fff700 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba5397ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 -- 172.21.15.45:0/2428415836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538f400 msgr2=0x55eba53979b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538f400 0x55eba53979b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 -- 172.21.15.45:0/2428415836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 msgr2=0x55eba5397ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba5397ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f080d468700 1 -- 172.21.15.45:0/2428415836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080810e520 con 0x55eba539abd0 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.428+0000 7f0807fff700 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba5397ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f080d468700 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba539abd0 0x55eba5398430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f080000ab40 tx=0x7f080000af00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5da29e2547f08fb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f07e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f07e7fff700 1 -- 172.21.15.45:0/2428415836 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800010040 con 0x55eba539abd0 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f07e7fff700 1 -- 172.21.15.45:0/2428415836 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0800006e40 con 0x55eba539abd0 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eba5383250 con 0x55eba539abd0 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f07e7fff700 1 -- 172.21.15.45:0/2428415836 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0800011c20 con 0x55eba539abd0 2023-09-23T09:44:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f081b0b5b80 10 client.26767 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f08081a3ea0 con 0x55eba539abd0 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.429+0000 7f07e7fff700 1 -- 172.21.15.45:0/2428415836 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eba5383250 con 0x55eba539abd0 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.430+0000 7f07e7fff700 1 -- 172.21.15.45:0/2428415836 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0800008030 con 0x55eba539abd0 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.430+0000 7f081b0b5b80 10 client.26767 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.430+0000 7f081b0b5b80 10 client.26767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.430+0000 7f081b0b5b80 20 client.26767 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.430+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba53bfac0 0x55eba53c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.430+0000 7f081b0b5b80 4 client.26767 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:44:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.430+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eba52dbe10 con 0x55eba53bfac0 2023-09-23T09:44:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.431+0000 7f080cc67700 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba53bfac0 0x55eba53c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.431+0000 7f07e7fff700 10 client.26767 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.431+0000 7f080cc67700 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba53bfac0 0x55eba53c1f80 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e493b9ae2abb72e5 server_cookie=5af686fcf257e009 in_seq=0 out_seq=0 2023-09-23T09:44:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f07e7fff700 1 -- 172.21.15.45:0/2428415836 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eba52dbe10 con 0x55eba53bfac0 2023-09-23T09:44:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f07e7fff700 10 client.26767 handle_command_reply: tid=0 2023-09-23T09:44:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 client.26767 shutdown 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba539abd0 msgr2=0x55eba5398430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba539abd0 0x55eba5398430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f080000ab40 tx=0x7f080000af00 comp rx=0 tx=0).stop 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 shutdown_connections 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba53bfac0 0x55eba53c1f80 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba539abd0 0x55eba5398430 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba538f400 0x55eba53979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f07e7fff700 0 client.26767 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2428415836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba5397ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 >> 172.21.15.45:0/2428415836 conn(0x55eba5304260 msgr2=0x55eba538b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 shutdown_connections 2023-09-23T09:44:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.432+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2428415836 wait complete. 2023-09-23T09:44:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.433+0000 7f081b0b5b80 20 client.26767 trim_cache size 0 max 16384 2023-09-23T09:44:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.454+0000 7fdbcefc0700 1 -- 172.21.15.45:0/3318100066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc810a0e0 msgr2=0x7fdbc8112830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.454+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/3318100066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc810a0e0 0x7fdbc8112830 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009990 tx=0x7fdbb8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 -- 172.21.15.45:0/3318100066 shutdown_connections 2023-09-23T09:44:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/3318100066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc8112d70 0x7fdbc8115160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/3318100066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbc81097c0 0x7fdbc8109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/3318100066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc810a0e0 0x7fdbc8112830 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 -- 172.21.15.45:0/3318100066 >> 172.21.15.45:0/3318100066 conn(0x7fdbc806cee0 msgr2=0x7fdbc806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 -- 172.21.15.45:0/3318100066 shutdown_connections 2023-09-23T09:44:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 -- 172.21.15.45:0/3318100066 wait complete. 2023-09-23T09:44:40.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 Processor -- start 2023-09-23T09:44:40.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 -- start start 2023-09-23T09:44:40.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538f400 0x55eba5398260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538fd20 0x55eba5396640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba5396b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba5163920 con 0x55eba538f400 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba539aa10 con 0x55eba539abd0 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba53b8110 con 0x55eba538fd20 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538f400 0x55eba5398260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538f400 0x55eba5398260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37120/0 (socket says 172.21.15.45:37120) 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 -- 172.21.15.45:0/2667550155 learned_addr learned my addr 172.21.15.45:0/2667550155 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080d468700 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba5396b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f0807fff700 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538fd20 0x55eba5396640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.455+0000 7fdbcefc0700 1 Processor -- start 2023-09-23T09:44:40.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 -- 172.21.15.45:0/2667550155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538fd20 msgr2=0x55eba5396640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538fd20 0x55eba5396640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 -- 172.21.15.45:0/2667550155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 msgr2=0x55eba5396b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba5396b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f080cc67700 1 -- 172.21.15.45:0/2667550155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba5387480 con 0x55eba538f400 2023-09-23T09:44:40.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.455+0000 7f0807fff700 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538fd20 0x55eba5396640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:44:40.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.456+0000 7f080d468700 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba5396b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcefc0700 1 -- start start 2023-09-23T09:44:40.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.456+0000 7f080cc67700 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538f400 0x55eba5398260 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f07fc099c40 tx=0x7f07fc099b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d7527ab775de3b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcefc0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc81097c0 0x7fdbc81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcefc0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbc810a0e0 0x7fdbc81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcefc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc8112d70 0x7fdbc81ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcefc0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81abae0 con 0x7fdbc8112d70 2023-09-23T09:44:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcefc0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81abc50 con 0x7fdbc810a0e0 2023-09-23T09:44:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcefc0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81abe10 con 0x7fdbc81097c0 2023-09-23T09:44:40.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.456+0000 7f07f37fe700 1 -- 172.21.15.45:0/2667550155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0a2040 con 0x55eba538f400 2023-09-23T09:44:40.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.456+0000 7f07f37fe700 1 -- 172.21.15.45:0/2667550155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07fc099cb0 con 0x55eba538f400 2023-09-23T09:44:40.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.456+0000 7f07f37fe700 1 -- 172.21.15.45:0/2667550155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc099400 con 0x55eba538f400 2023-09-23T09:44:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc8112d70 0x7fdbc81ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc8112d70 0x7fdbc81ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37128/0 (socket says 172.21.15.45:37128) 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbccd5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc81097c0 0x7fdbc81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 -- 172.21.15.45:0/1912518225 learned_addr learned my addr 172.21.15.45:0/1912518225 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbc810a0e0 0x7fdbc81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 -- 172.21.15.45:0/1912518225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc81097c0 msgr2=0x7fdbc81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc81097c0 0x7fdbc81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 -- 172.21.15.45:0/1912518225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbc810a0e0 msgr2=0x7fdbc81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbc810a0e0 0x7fdbc81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbcd55d700 1 -- 172.21.15.45:0/1912518225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc009520 con 0x7fdbc8112d70 2023-09-23T09:44:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.456+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbc810a0e0 0x7fdbc81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.457+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc8112d70 0x7fdbc81ab4e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000ef30 tx=0x7fdbc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.457+0000 7fdbc5ffb700 1 -- 172.21.15.45:0/1912518225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0018070 con 0x7fdbc8112d70 2023-09-23T09:44:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.457+0000 7fdbc5ffb700 1 -- 172.21.15.45:0/1912518225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc000f040 con 0x7fdbc8112d70 2023-09-23T09:44:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.457+0000 7fdbc5ffb700 1 -- 172.21.15.45:0/1912518225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000cd00 con 0x7fdbc8112d70 2023-09-23T09:44:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.457+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb80095f0 con 0x7fdbc8112d70 2023-09-23T09:44:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.457+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc81ac3c0 con 0x7fdbc8112d70 2023-09-23T09:44:40.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2667550155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538f400 msgr2=0x55eba5398260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538f400 0x55eba5398260 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f07fc099c40 tx=0x7f07fc099b30 comp rx=0 tx=0).stop 2023-09-23T09:44:40.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2667550155 shutdown_connections 2023-09-23T09:44:40.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba5396b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538f400 0x55eba5398260 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/2667550155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538fd20 0x55eba5396640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2667550155 >> 172.21.15.45:0/2667550155 conn(0x55eba5304260 msgr2=0x55eba5395f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2667550155 shutdown_connections 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.457+0000 7f081b0b5b80 1 -- 172.21.15.45:0/2667550155 wait complete. 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 -- start start 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 Processor -- start 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba53bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba53bc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba53baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba53bd040 con 0x55eba538fd20 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba53bd1b0 con 0x55eba539abd0 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba53bb460 con 0x55eba538f400 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f081b0b5b80 10 client.0 _ll_register_callbacks cb 0x7f080000a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f080cc67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba53bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f080d468700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba53baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f080cc67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba53bc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39238/0 (socket says 172.21.15.45:39238) 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f080d468700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba53baf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58484/0 (socket says 172.21.15.45:58484) 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.458+0000 7f080cc67700 1 -- 172.21.15.45:0/3783246247 learned_addr learned my addr 172.21.15.45:0/3783246247 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.459+0000 7f0807fff700 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba53bc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.459+0000 7f080cc67700 1 -- 172.21.15.45:0/3783246247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 msgr2=0x55eba53baf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.459+0000 7f080cc67700 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba53baf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.459+0000 7f080cc67700 1 -- 172.21.15.45:0/3783246247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 msgr2=0x55eba53bc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.459+0000 7f080cc67700 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba53bc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.459+0000 7f080cc67700 1 -- 172.21.15.45:0/3783246247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08081a3ea0 con 0x55eba538f400 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f080cc67700 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba53bc3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0ca980 tx=0x7f07fc0a2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e7820ae07aa622 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f07e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f07e7fff700 1 -- 172.21.15.45:0/3783246247 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0b8040 con 0x55eba538f400 2023-09-23T09:44:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f07e7fff700 1 -- 172.21.15.45:0/3783246247 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f07fc065470 con 0x55eba538f400 2023-09-23T09:44:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07fc09de00 con 0x55eba538f400 2023-09-23T09:44:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f07e7fff700 1 -- 172.21.15.45:0/3783246247 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07fc098150 con 0x55eba538f400 2023-09-23T09:44:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f081b0b5b80 10 client.26779 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.460+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55eba53458d0 con 0x55eba538f400 2023-09-23T09:44:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.461+0000 7f07e7fff700 1 -- 172.21.15.45:0/3783246247 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07fc09de00 con 0x55eba538f400 2023-09-23T09:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.461+0000 7fdbc5ffb700 1 -- 172.21.15.45:0/1912518225 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fdbc0003a00 con 0x7fdbc8112d70 2023-09-23T09:44:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f07e7fff700 1 -- 172.21.15.45:0/3783246247 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07fc07e070 con 0x55eba538f400 2023-09-23T09:44:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f081b0b5b80 10 client.26779 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f081b0b5b80 10 client.26779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f081b0b5b80 20 client.26779 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba54188e0 0x55eba541acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f081b0b5b80 4 client.26779 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:44:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eba52dbb30 con 0x55eba54188e0 2023-09-23T09:44:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.462+0000 7fdbc5ffb700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fdbb00a1810 0x7fdbb00a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.462+0000 7fdbc5ffb700 1 -- 172.21.15.45:0/1912518225 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdbc00cce30 con 0x7fdbc8112d70 2023-09-23T09:44:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.462+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fdbb00a1810 0x7fdbb00a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.462+0000 7f0807fff700 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba54188e0 0x55eba541acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.463+0000 7f0807fff700 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba54188e0 0x55eba541acd0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b212dce92081da33 server_cookie=62b02eae2ed235c5 in_seq=0 out_seq=0 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.463+0000 7f07e7fff700 10 client.26779 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.463+0000 7f07e7fff700 1 -- 172.21.15.45:0/3783246247 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eba52dbb30 con 0x55eba54188e0 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.463+0000 7f07e7fff700 10 client.26779 handle_command_reply: tid=0 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.463+0000 7f081b0b5b80 1 client.26779 shutdown 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.463+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 msgr2=0x55eba53bc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.463+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba53bc3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0ca980 tx=0x7f07fc0a2040 comp rx=0 tx=0).stop 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 shutdown_connections 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x55eba54188e0 0x55eba541acd0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eba539abd0 0x55eba53baf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eba538f400 0x55eba53bc3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 --2- 172.21.15.45:0/3783246247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eba538fd20 0x55eba53bc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 >> 172.21.15.45:0/3783246247 conn(0x55eba5304260 msgr2=0x55eba53967c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f07e7fff700 0 client.26779 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 shutdown_connections 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 1 -- 172.21.15.45:0/3783246247 wait complete. 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f081b0b5b80 20 client.26779 trim_cache size 0 max 16384 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:44:40.476 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:44:40.477 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:44:40.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07f80a1710 msgr2=0x7f07f80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.464+0000 7f080eecb700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07f80a1710 0x7f07f80a3bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f07f400b5c0 tx=0x7f07f4005ab0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0808101ff0 msgr2=0x7f0808197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0808101ff0 0x7f0808197070 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00b550 tx=0x7f07fc00b860 comp rx=0 tx=0).stop 2023-09-23T09:44:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.463+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fdbb00a1810 0x7fdbb00a3cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009df0 tx=0x7fdbbc009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 shutdown_connections 2023-09-23T09:44:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f080810b3c0 0x7f080819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f07f80a1710 0x7f07f80a3bd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0808101ff0 0x7f0808197070 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 --2- 172.21.15.45:0/1258826238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0808102910 0x7f08081975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 >> 172.21.15.45:0/1258826238 conn(0x7f08080fb770 msgr2=0x7f08080fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 shutdown_connections 2023-09-23T09:44:40.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:44:40.465+0000 7f080eecb700 1 -- 172.21.15.45:0/1258826238 wait complete. 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 Processor -- start 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 -- start start 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891eff40 0x557f891f0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891ffb20 0x557f89201f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f89159400 con 0x557f891ffb20 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8914abc0 con 0x557f891f0860 2023-09-23T09:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.465+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f891484e0 con 0x557f891eff40 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbc7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbc7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58492/0 (socket says 172.21.15.45:58492) 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbc7fff700 1 -- 172.21.15.45:0/4249385562 learned_addr learned my addr 172.21.15.45:0/4249385562 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891ffb20 0x557f89201f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891eff40 0x557f891f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbcd55d700 1 -- 172.21.15.45:0/4249385562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891eff40 msgr2=0x557f891f0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891eff40 0x557f891f0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbcd55d700 1 -- 172.21.15.45:0/4249385562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 msgr2=0x557f891ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbcd55d700 1 -- 172.21.15.45:0/4249385562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f891a6870 con 0x557f891ffb20 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.466+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891ffb20 0x557f89201f10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ca4e0 tx=0x7fdbc00998d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4127425aec4ec716 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.467+0000 7fdbaeffd700 1 -- 172.21.15.45:0/4249385562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000ce80 con 0x557f891ffb20 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.467+0000 7fdbaeffd700 1 -- 172.21.15.45:0/4249385562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc0004ad0 con 0x557f891ffb20 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.467+0000 7fdbaeffd700 1 -- 172.21.15.45:0/4249385562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00a2090 con 0x557f891ffb20 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.467+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/4249385562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891ffb20 msgr2=0x557f89201f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.467+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891ffb20 0x557f89201f10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ca4e0 tx=0x7fdbc00998d0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/4249385562 shutdown_connections 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891ffb20 0x557f89201f10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891eff40 0x557f891f0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/4249385562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/4249385562 >> 172.21.15.45:0/4249385562 conn(0x557f89165200 msgr2=0x557f891ec3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/4249385562 shutdown_connections 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/4249385562 wait complete. 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 -- start start 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.468+0000 7fdbdb1aab80 1 Processor -- start 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891fc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f89143dd0 con 0x557f891eff40 2023-09-23T09:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f890e06b0 con 0x557f891f0860 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f890e0350 con 0x557f891ffb20 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbdb1aab80 10 client.0 _ll_register_callbacks cb 0x557f88f3d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891fc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58500/0 (socket says 172.21.15.45:58500) 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 -- 172.21.15.45:0/3567163123 learned_addr learned my addr 172.21.15.45:0/3567163123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 -- 172.21.15.45:0/3567163123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 msgr2=0x557f891fcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 -- 172.21.15.45:0/3567163123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 msgr2=0x557f891ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891ff580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 -- 172.21.15.45:0/3567163123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc81ac3c0 con 0x557f891f0860 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891fc800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80052a0 tx=0x7fdbb80052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c7d2b06a939397 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.469+0000 7fdba7fff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdba7fff700 1 -- 172.21.15.45:0/3567163123 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb801d070 con 0x557f891f0860 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdba7fff700 1 -- 172.21.15.45:0/3567163123 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdbb8004910 con 0x557f891f0860 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f891e4090 con 0x557f891f0860 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdba7fff700 1 -- 172.21.15.45:0/3567163123 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbb8004ad0 con 0x557f891f0860 2023-09-23T09:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 10 client.26768 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557f88f6ed80 con 0x557f891f0860 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdba7fff700 1 -- 172.21.15.45:0/3567163123 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f891e4090 con 0x557f891f0860 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdba7fff700 1 -- 172.21.15.45:0/3567163123 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdbb800fcf0 con 0x557f891f0860 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 10 client.26768 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 10 client.26768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 20 client.26768 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f8920f5d0 0x557f892119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 4 client.26768 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.470+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f8913cdb0 con 0x557f8920f5d0 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.471+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f8920f5d0 0x557f892119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.471+0000 7fdba7fff700 10 client.26768 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.471+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f8920f5d0 0x557f892119f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669aa857ccdb2148 server_cookie=e17369c13a7e0e2e in_seq=0 out_seq=0 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdba7fff700 1 -- 172.21.15.45:0/3567163123 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f8913cdb0 con 0x557f8920f5d0 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdba7fff700 10 client.26768 handle_command_reply: tid=0 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 client.26768 shutdown 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 msgr2=0x557f891fc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891fc800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80052a0 tx=0x7fdbb80052d0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 shutdown_connections 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f8920f5d0 0x557f892119f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdba7fff700 0 client.26768 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/3567163123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891fc800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 >> 172.21.15.45:0/3567163123 conn(0x557f89165200 msgr2=0x557f891ebfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 shutdown_connections 2023-09-23T09:44:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/3567163123 wait complete. 2023-09-23T09:44:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.472+0000 7fdbdb1aab80 20 client.26768 trim_cache size 0 max 16384 2023-09-23T09:44:40.494 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 Processor -- start 2023-09-23T09:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 -- start start 2023-09-23T09:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f88fc48c0 con 0x557f891eff40 2023-09-23T09:44:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f891fada0 con 0x557f891f0860 2023-09-23T09:44:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.495+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f891eb170 con 0x557f891ffb20 2023-09-23T09:44:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbccd5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58508/0 (socket says 172.21.15.45:58508) 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbcd55d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 -- 172.21.15.45:0/640339752 learned_addr learned my addr 172.21.15.45:0/640339752 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbccd5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891fcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37156/0 (socket says 172.21.15.45:37156) 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 -- 172.21.15.45:0/640339752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 msgr2=0x557f891fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 -- 172.21.15.45:0/640339752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 msgr2=0x557f891fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 -- 172.21.15.45:0/640339752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f88f57c00 con 0x557f891f0860 2023-09-23T09:44:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8030040 tx=0x7fdbb8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571a5e14abc55104 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.496+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:44:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.497+0000 7fdbaf7fe700 1 -- 172.21.15.45:0/640339752 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb801d040 con 0x557f891f0860 2023-09-23T09:44:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.497+0000 7fdbaf7fe700 1 -- 172.21.15.45:0/640339752 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb801e040 con 0x557f891f0860 2023-09-23T09:44:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.497+0000 7fdbaf7fe700 1 -- 172.21.15.45:0/640339752 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800f460 con 0x557f891f0860 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.497+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/640339752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 msgr2=0x557f891ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.497+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8030040 tx=0x7fdbb8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/640339752 shutdown_connections 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891ffb20 0x557f891fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f891fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/640339752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891f0860 0x557f891ff120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/640339752 >> 172.21.15.45:0/640339752 conn(0x557f89165200 msgr2=0x557f891f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/640339752 shutdown_connections 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/640339752 wait complete. 2023-09-23T09:44:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- start start 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 Processor -- start 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f89204650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891f0860 0x557f89204b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891ffb20 0x557f892623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f89205330 con 0x557f891eff40 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f892054a0 con 0x557f891ffb20 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f89262900 con 0x557f891f0860 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbdb1aab80 10 client.0 _ll_register_callbacks cb 0x7fdbb800bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbccd5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f89204650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbccd5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f89204650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37162/0 (socket says 172.21.15.45:37162) 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbccd5c700 1 -- 172.21.15.45:0/1817245902 learned_addr learned my addr 172.21.15.45:0/1817245902 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.498+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891f0860 0x557f89204b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbcd55d700 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891ffb20 0x557f892623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbccd5c700 1 -- 172.21.15.45:0/1817245902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891f0860 msgr2=0x557f89204b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891f0860 0x557f89204b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbccd5c700 1 -- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891ffb20 msgr2=0x557f892623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891ffb20 0x557f892623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbccd5c700 1 -- 172.21.15.45:0/1817245902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc017070 con 0x557f891eff40 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbccd5c700 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f89204650 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x557f89212260 tx=0x7fdbbc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e836e26735ce1c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdba7fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdba7fff700 1 -- 172.21.15.45:0/1817245902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc023040 con 0x557f891eff40 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbc0004ad0 con 0x557f891eff40 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdba7fff700 1 -- 172.21.15.45:0/1817245902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdbbc029560 con 0x557f891eff40 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdba7fff700 1 -- 172.21.15.45:0/1817245902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbbc01d9f0 con 0x557f891eff40 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbdb1aab80 10 client.17139 fetch_fsmap learned FSMap version 55 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.499+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557f88f6ed80 con 0x557f891eff40 2023-09-23T09:44:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdba7fff700 1 -- 172.21.15.45:0/1817245902 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc0004ad0 con 0x557f891eff40 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdba7fff700 1 -- 172.21.15.45:0/1817245902 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdbbc032b90 con 0x557f891eff40 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdbdb1aab80 10 client.17139 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdbdb1aab80 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdbdb1aab80 20 client.17139 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f89264180 0x557f892665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdbdb1aab80 4 client.17139 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f8913cad0 con 0x557f89264180 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.500+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f89264180 0x557f892665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.501+0000 7fdbc7fff700 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f89264180 0x557f892665a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5586a5d4a2397a server_cookie=5d259a082ab2461a in_seq=0 out_seq=0 2023-09-23T09:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.501+0000 7fdba7fff700 10 client.17139 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.515+0000 7fdba7fff700 1 -- 172.21.15.45:0/1817245902 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f8913cad0 con 0x557f89264180 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.515+0000 7fdba7fff700 10 client.17139 handle_command_reply: tid=0 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.515+0000 7fdbdb1aab80 1 client.17139 shutdown 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 msgr2=0x557f89204650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f89204650 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x557f89212260 tx=0x7fdbbc0047c0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 shutdown_connections 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x557f89264180 0x557f892665a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f891ffb20 0x557f892623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdba7fff700 0 client.17139 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f891eff40 0x557f89204650 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 --2- 172.21.15.45:0/1817245902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f891f0860 0x557f89204b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 >> 172.21.15.45:0/1817245902 conn(0x557f89165200 msgr2=0x557f891f1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 shutdown_connections 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 1 -- 172.21.15.45:0/1817245902 wait complete. 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbdb1aab80 20 client.17139 trim_cache size 0 max 16384 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:44:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:44:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:44:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fdbb00a1810 msgr2=0x7fdbb00a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fdbb00a1810 0x7fdbb00a3cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009df0 tx=0x7fdbbc009260 comp rx=0 tx=0).stop 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc8112d70 msgr2=0x7fdbc81ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc8112d70 0x7fdbc81ab4e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000ef30 tx=0x7fdbc000c3a0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 shutdown_connections 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fdbb00a1810 0x7fdbb00a3cd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbc8112d70 0x7fdbc81ab4e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.516+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc81097c0 0x7fdbc81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.517+0000 7fdbcefc0700 1 --2- 172.21.15.45:0/1912518225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbc810a0e0 0x7fdbc81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.517+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 >> 172.21.15.45:0/1912518225 conn(0x7fdbc806cee0 msgr2=0x7fdbc806f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.517+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 shutdown_connections 2023-09-23T09:44:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:40.517+0000 7fdbcefc0700 1 -- 172.21.15.45:0/1912518225 wait complete. 2023-09-23T09:44:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 895.0 s, 965.9 tps, lat 103.612 ms stddev 110.806 2023-09-23T09:44:42.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:41 smithi098 ceph-mon[102386]: pgmap v479: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.54k op/s 2023-09-23T09:44:42.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:41 smithi045 ceph-mon[98627]: pgmap v479: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.54k op/s 2023-09-23T09:44:42.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:41 smithi123 ceph-mon[103577]: pgmap v479: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.54k op/s 2023-09-23T09:44:44.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:43 smithi098 ceph-mon[102386]: pgmap v480: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-09-23T09:44:44.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:43 smithi045 ceph-mon[98627]: pgmap v480: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-09-23T09:44:44.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:43 smithi123 ceph-mon[103577]: pgmap v480: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-09-23T09:44:46.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:45 smithi098 ceph-mon[102386]: pgmap v481: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-09-23T09:44:46.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:45 smithi045 ceph-mon[98627]: pgmap v481: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-09-23T09:44:46.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:45 smithi123 ceph-mon[103577]: pgmap v481: 97 pgs: 97 active+clean; 5.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-09-23T09:44:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 900.0 s, 1166.1 tps, lat 84.077 ms stddev 101.219 2023-09-23T09:44:47.092 INFO:teuthology.orchestra.run.smithi045.stdout:transaction type: 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:scaling factor: 32 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:query mode: simple 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:number of clients: 100 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:number of threads: 4 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:duration: 900 s 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:number of transactions actually processed: 1699729 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:latency average = 52.952 ms 2023-09-23T09:44:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:latency stddev = 153.674 ms 2023-09-23T09:44:47.094 INFO:teuthology.orchestra.run.smithi045.stdout:tps = 1888.147606 (including connections establishing) 2023-09-23T09:44:47.094 INFO:teuthology.orchestra.run.smithi045.stdout:tps = 1888.152313 (excluding connections establishing) 2023-09-23T09:44:47.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-09-23T09:44:48.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:48 smithi098 ceph-mon[102386]: pgmap v482: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 24 MiB/s wr, 1.67k op/s 2023-09-23T09:44:48.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:48 smithi045 ceph-mon[98627]: pgmap v482: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 24 MiB/s wr, 1.67k op/s 2023-09-23T09:44:48.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:48 smithi123 ceph-mon[103577]: pgmap v482: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 24 MiB/s wr, 1.67k op/s 2023-09-23T09:44:50.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:50 smithi098 ceph-mon[102386]: pgmap v483: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-09-23T09:44:50.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:50 smithi045 ceph-mon[98627]: pgmap v483: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-09-23T09:44:50.590 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:50 smithi123 ceph-mon[103577]: pgmap v483: 97 pgs: 97 active+clean; 5.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-09-23T09:44:52.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:52 smithi098 ceph-mon[102386]: pgmap v484: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-09-23T09:44:52.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:52 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:44:52.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:52 smithi045 ceph-mon[98627]: pgmap v484: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-09-23T09:44:52.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:52 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:44:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:52 smithi123 ceph-mon[103577]: pgmap v484: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-09-23T09:44:52.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:52 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:44:53.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-23T09:44:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 -- 172.21.15.45:0/1445459330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c18100160 msgr2=0x7f2c18100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/1445459330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c18100160 0x7f2c18100540 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2c080099c0 tx=0x7f2c08009cd0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 -- 172.21.15.45:0/1445459330 shutdown_connections 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/1445459330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18105050 0x7f2c1810cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/1445459330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c18100160 0x7f2c18100540 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/1445459330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c18100b10 0x7f2c18104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 -- 172.21.15.45:0/1445459330 >> 172.21.15.45:0/1445459330 conn(0x7f2c18076840 msgr2=0x7f2c18076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 -- 172.21.15.45:0/1445459330 shutdown_connections 2023-09-23T09:44:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.642+0000 7f2c1cc97700 1 -- 172.21.15.45:0/1445459330 wait complete. 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 Processor -- start 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 -- start start 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c18100160 0x7f2c18197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18105050 0x7f2c181983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c181a0d00 0x7f2c181a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a36c0 con 0x7f2c181a0d00 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a3830 con 0x7f2c18105050 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1cc97700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a39f0 con 0x7f2c18100160 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c181a0d00 0x7f2c181a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c181a0d00 0x7f2c181a30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37180/0 (socket says 172.21.15.45:37180) 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2595048524 learned_addr learned my addr 172.21.15.45:0/2595048524 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18105050 0x7f2c181983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1659c700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c18100160 0x7f2c18197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2595048524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c18100160 msgr2=0x7f2c18197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c18100160 0x7f2c18197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2595048524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18105050 msgr2=0x7f2c181983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18105050 0x7f2c181983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2595048524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080095f0 con 0x7f2c181a0d00 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.643+0000 7f2c1659c700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c18100160 0x7f2c18197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:44:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.644+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c181a0d00 0x7f2c181a30f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2c10007ee0 tx=0x7f2c1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.644+0000 7f2c0f7fe700 1 -- 172.21.15.45:0/2595048524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10017070 con 0x7f2c181a0d00 2023-09-23T09:44:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.644+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c181a3cd0 con 0x7f2c181a0d00 2023-09-23T09:44:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.644+0000 7f2c0f7fe700 1 -- 172.21.15.45:0/2595048524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c1000f040 con 0x7f2c181a0d00 2023-09-23T09:44:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.644+0000 7f2c0f7fe700 1 -- 172.21.15.45:0/2595048524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10005740 con 0x7f2c181a0d00 2023-09-23T09:44:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.644+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c181a4190 con 0x7f2c181a0d00 2023-09-23T09:44:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.646+0000 7f2c0f7fe700 1 -- 172.21.15.45:0/2595048524 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f2c1000dc60 con 0x7f2c181a0d00 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.646+0000 7f2c0f7fe700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2c040a1740 0x7f2c040a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.647+0000 7f2c0f7fe700 1 -- 172.21.15.45:0/2595048524 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c100ccab0 con 0x7f2c181a0d00 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.647+0000 7f2c1659c700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2c040a1740 0x7f2c040a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.647+0000 7f2c1659c700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2c040a1740 0x7f2c040a3c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006010 tx=0x7f2c08009f10 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.647+0000 7f2c28e81b80 1 Processor -- start 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.647+0000 7f2c28e81b80 1 -- start start 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a86320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4a93b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c49ef400 con 0x5572c4a91710 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c49e0bc0 con 0x5572c4a86860 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c49de4e0 con 0x5572c4a85f40 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4a93b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4a93b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37188/0 (socket says 172.21.15.45:37188) 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2776178313 learned_addr learned my addr 172.21.15.45:0/2776178313 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c1659c700 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a86320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2776178313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 msgr2=0x5572c4a86320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a86320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2776178313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 msgr2=0x5572c4a911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 -- 172.21.15.45:0/2776178313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c4a3c870 con 0x5572c4a91710 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.648+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4a93b00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000d7b0 tx=0x7f2c1000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ab2852371c3a4e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c0cff9700 1 -- 172.21.15.45:0/2776178313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1009db30 con 0x5572c4a91710 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c0cff9700 1 -- 172.21.15.45:0/2776178313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c100a1d70 con 0x5572c4a91710 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c0cff9700 1 -- 172.21.15.45:0/2776178313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10005aa0 con 0x5572c4a91710 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2776178313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 msgr2=0x5572c4a93b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4a93b00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000d7b0 tx=0x7f2c1000df60 comp rx=0 tx=0).stop 2023-09-23T09:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2776178313 shutdown_connections 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4a93b00 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a86320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2776178313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2776178313 >> 172.21.15.45:0/2776178313 conn(0x5572c49fb200 msgr2=0x5572c4a823c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2776178313 shutdown_connections 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.649+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2776178313 wait complete. 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 -- start start 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 Processor -- start 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a85f40 0x5572c4a911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a86860 0x5572c4a8e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a91710 0x5572c4a8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c49d9dd0 con 0x5572c4a85f40 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c49766b0 con 0x5572c4a91710 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4976350 con 0x5572c4a86860 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c28e81b80 10 client.0 _ll_register_callbacks cb 0x5572c47d37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c1659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a85f40 0x5572c4a911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c1659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a85f40 0x5572c4a911b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37190/0 (socket says 172.21.15.45:37190) 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c1659c700 1 -- 172.21.15.45:0/234452774 learned_addr learned my addr 172.21.15.45:0/234452774 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.650+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a91710 0x5572c4a8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a86860 0x5572c4a8e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c1659c700 1 -- 172.21.15.45:0/234452774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a86860 msgr2=0x5572c4a8e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c1659c700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a86860 0x5572c4a8e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c1659c700 1 -- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a91710 msgr2=0x5572c4a8eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c1659c700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a91710 0x5572c4a8eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c1659c700 1 -- 172.21.15.45:0/234452774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c181a4190 con 0x5572c4a85f40 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a86860 0x5572c4a8e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c1659c700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a85f40 0x5572c4a911b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2c08015000 tx=0x7f2c08009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a589f7a5763a04 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2bfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2bfdffb700 1 -- 172.21.15.45:0/234452774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0802c070 con 0x5572c4a85f40 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2bfdffb700 1 -- 172.21.15.45:0/234452774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c08003ad0 con 0x5572c4a85f40 2023-09-23T09:44:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572c4a7a090 con 0x5572c4a85f40 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2bfdffb700 1 -- 172.21.15.45:0/234452774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0801e070 con 0x5572c4a85f40 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c28e81b80 10 client.17151 fetch_fsmap learned FSMap version 56 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5572c4804d80 con 0x5572c4a85f40 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.651+0000 7f2bfdffb700 1 -- 172.21.15.45:0/234452774 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572c4a7a090 con 0x5572c4a85f40 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2bfdffb700 1 -- 172.21.15.45:0/234452774 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2c08004a20 con 0x5572c4a85f40 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c28e81b80 10 client.17151 fetch_fsmap finished waiting for FSMap version 56 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c28e81b80 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c28e81b80 20 client.17151 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4aa3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c28e81b80 4 client.17151 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572c49d2db0 con 0x5572c4aa1310 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4aa3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2bfdffb700 10 client.17151 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.652+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4aa3730 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d59fad1e75d4ca5 server_cookie=cc41fdf2d00a2340 in_seq=0 out_seq=0 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2bfdffb700 1 -- 172.21.15.45:0/234452774 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5572c49d2db0 con 0x5572c4aa1310 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2bfdffb700 10 client.17151 handle_command_reply: tid=0 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 client.17151 shutdown 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a85f40 msgr2=0x5572c4a911b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a85f40 0x5572c4a911b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2c08015000 tx=0x7f2c08009310 comp rx=0 tx=0).stop 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 shutdown_connections 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4aa3730 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a91710 0x5572c4a8eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a85f40 0x5572c4a911b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2bfdffb700 0 client.17151 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/234452774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a86860 0x5572c4a8e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 >> 172.21.15.45:0/234452774 conn(0x5572c49fb200 msgr2=0x5572c4a81fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 shutdown_connections 2023-09-23T09:44:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 1 -- 172.21.15.45:0/234452774 wait complete. 2023-09-23T09:44:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.653+0000 7f2c28e81b80 20 client.17151 trim_cache size 0 max 16384 2023-09-23T09:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 Processor -- start 2023-09-23T09:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 -- start start 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a85f40 0x5572c4a93b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a86860 0x5572c4a9d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a91710 0x5572c4a90d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c485a8c0 con 0x5572c4a86860 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4a8a350 con 0x5572c4a85f40 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4a892e0 con 0x5572c4a91710 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a86860 0x5572c4a9d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a86860 0x5572c4a9d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37200/0 (socket says 172.21.15.45:37200) 2023-09-23T09:44:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 -- 172.21.15.45:0/2848805992 learned_addr learned my addr 172.21.15.45:0/2848805992 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a91710 0x5572c4a90d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c1659c700 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a85f40 0x5572c4a93b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 -- 172.21.15.45:0/2848805992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a91710 msgr2=0x5572c4a90d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a91710 0x5572c4a90d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 -- 172.21.15.45:0/2848805992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a85f40 msgr2=0x5572c4a93b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a85f40 0x5572c4a93b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.668+0000 7f2c15d9b700 1 -- 172.21.15.45:0/2848805992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c47edc00 con 0x5572c4a86860 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c1659c700 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a85f40 0x5572c4a93b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c15d9b700 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a86860 0x5572c4a9d4f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2c18073d50 tx=0x7f2c00004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcc50e8de785946 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c0d7fa700 1 -- 172.21.15.45:0/2848805992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00016040 con 0x5572c4a86860 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c0d7fa700 1 -- 172.21.15.45:0/2848805992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0001b5a0 con 0x5572c4a86860 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c0d7fa700 1 -- 172.21.15.45:0/2848805992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000f680 con 0x5572c4a86860 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2848805992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a86860 msgr2=0x5572c4a9d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a86860 0x5572c4a9d4f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2c18073d50 tx=0x7f2c00004320 comp rx=0 tx=0).stop 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2848805992 shutdown_connections 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a91710 0x5572c4a90d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a85f40 0x5572c4a93b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/2848805992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a86860 0x5572c4a9d4f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2848805992 >> 172.21.15.45:0/2848805992 conn(0x5572c49fb200 msgr2=0x5572c4a80af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2848805992 shutdown_connections 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.669+0000 7f2c28e81b80 1 -- 172.21.15.45:0/2848805992 wait complete. 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 -- start start 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 Processor -- start 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a97410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a97950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4af4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4a98260 con 0x5572c4a91710 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4af4680 con 0x5572c4a86860 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c4af47c0 con 0x5572c4a85f40 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c28e81b80 10 client.0 _ll_register_callbacks cb 0x7f2c080055e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4af4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c15d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a97950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c1659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a97410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4af4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37208/0 (socket says 172.21.15.45:37208) 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c15d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a97950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58556/0 (socket says 172.21.15.45:58556) 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.670+0000 7f2c16d9d700 1 -- 172.21.15.45:0/406916551 learned_addr learned my addr 172.21.15.45:0/406916551 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c16d9d700 1 -- 172.21.15.45:0/406916551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 msgr2=0x5572c4a97410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a97410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c16d9d700 1 -- 172.21.15.45:0/406916551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 msgr2=0x5572c4a97950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a97950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c16d9d700 1 -- 172.21.15.45:0/406916551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0000ac90 con 0x5572c4a91710 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c16d9d700 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4af4140 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2c100ca270 tx=0x7f2c1000dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abda7da47b7b1be server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2bfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2bfdffb700 1 -- 172.21.15.45:0/406916551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10005aa0 con 0x5572c4a91710 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2bfdffb700 1 -- 172.21.15.45:0/406916551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c1009db30 con 0x5572c4a91710 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c100a1d70 con 0x5572c4a91710 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2bfdffb700 1 -- 172.21.15.45:0/406916551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c100b2120 con 0x5572c4a91710 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c28e81b80 10 client.17157 fetch_fsmap learned FSMap version 56 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5572c482b1f0 con 0x5572c4a91710 2023-09-23T09:44:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2bfdffb700 1 -- 172.21.15.45:0/406916551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c100a1d70 con 0x5572c4a91710 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2bfdffb700 1 -- 172.21.15.45:0/406916551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2c10099c40 con 0x5572c4a91710 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c28e81b80 10 client.17157 fetch_fsmap finished waiting for FSMap version 56 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c28e81b80 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.671+0000 7f2c28e81b80 20 client.17157 populate_metadata read hostname 'smithi045' 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.672+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4af7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.672+0000 7f2c28e81b80 4 client.17157 mds_command: new command op to 24530 tid=0[{"prefix": "flush journal"}] 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.672+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5572c49d2ad0 con 0x5572c4aa1310 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.672+0000 7f2c1659c700 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4af7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.672+0000 7f2bfdffb700 10 client.17157 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.672+0000 7f2c1659c700 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4af7c50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb82e6aa2ff3e8d server_cookie=a1317fda6cbe358c in_seq=0 out_seq=0 2023-09-23T09:44:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.711+0000 7f2bfdffb700 1 -- 172.21.15.45:0/406916551 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5572c49d2ad0 con 0x5572c4aa1310 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.711+0000 7f2bfdffb700 10 client.17157 handle_command_reply: tid=0 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.711+0000 7f2c28e81b80 1 client.17157 shutdown 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 msgr2=0x5572c4af4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4af4140 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2c100ca270 tx=0x7f2c1000dbd0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 shutdown_connections 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x5572c4aa1310 0x5572c4af7c50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572c4a91710 0x5572c4af4140 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572c4a85f40 0x5572c4a97410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 --2- 172.21.15.45:0/406916551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572c4a86860 0x5572c4a97950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2bfdffb700 0 client.17157 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 >> 172.21.15.45:0/406916551 conn(0x5572c49fb200 msgr2=0x5572c4a89810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 shutdown_connections 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 1 -- 172.21.15.45:0/406916551 wait complete. 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c28e81b80 20 client.17157 trim_cache size 0 max 16384 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-09-23T09:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-09-23T09:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2c040a1740 msgr2=0x7f2c040a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2c040a1740 0x7f2c040a3c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006010 tx=0x7f2c08009f10 comp rx=0 tx=0).stop 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c181a0d00 msgr2=0x7f2c181a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c181a0d00 0x7f2c181a30f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2c10007ee0 tx=0x7f2c1000d3b0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 shutdown_connections 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2c040a1740 0x7f2c040a3c00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c181a0d00 0x7f2c181a30f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c18100160 0x7f2c18197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 --2- 172.21.15.45:0/2595048524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18105050 0x7f2c181983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.712+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 >> 172.21.15.45:0/2595048524 conn(0x7f2c18076840 msgr2=0x7f2c180fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.713+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 shutdown_connections 2023-09-23T09:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-09-23T09:44:53.713+0000 7f2c1cc97700 1 -- 172.21.15.45:0/2595048524 wait complete. 2023-09-23T09:44:54.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:54 smithi098 ceph-mon[102386]: pgmap v485: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-09-23T09:44:54.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:54 smithi045 ceph-mon[98627]: pgmap v485: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-09-23T09:44:54.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:54 smithi123 ceph-mon[103577]: pgmap v485: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-09-23T09:44:56.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:56 smithi098 ceph-mon[102386]: pgmap v486: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-09-23T09:44:56.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:56 smithi045 ceph-mon[98627]: pgmap v486: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-09-23T09:44:56.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:56 smithi123 ceph-mon[103577]: pgmap v486: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-09-23T09:44:57.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:57 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:44:57.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:57 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:44:57.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:57 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:44:58.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:44:58 smithi098 ceph-mon[102386]: pgmap v487: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2023-09-23T09:44:58.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:44:58 smithi045 ceph-mon[98627]: pgmap v487: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2023-09-23T09:44:58.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:44:58 smithi123 ceph-mon[103577]: pgmap v487: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2023-09-23T09:45:00.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:00 smithi098 ceph-mon[102386]: pgmap v488: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-09-23T09:45:00.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:00 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:00 smithi045 ceph-mon[98627]: pgmap v488: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-09-23T09:45:00.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:00 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:00.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:00 smithi123 ceph-mon[103577]: pgmap v488: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-09-23T09:45:00.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:00 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:02.447 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/: 2023-09-23T09:45:02.447 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.447 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:45:02.447 INFO:teuthology.orchestra.run.smithi045.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 .. 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 data 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data: 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:total 31 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 . 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 .. 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 23 09:29 base 2023-09-23T09:45:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 23 09:29 current_logfiles 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Sep 23 09:29 global 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 log 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_commit_ts 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_dynshmem 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 23 09:29 pg_hba.conf 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 23 09:29 pg_ident.conf 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:44 pg_logical 2023-09-23T09:45:02.449 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 pg_multixact 2023-09-23T09:45:02.450 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 pg_notify 2023-09-23T09:45:02.450 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_replslot 2023-09-23T09:45:02.450 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_serial 2023-09-23T09:45:02.450 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_snapshots 2023-09-23T09:45:02.450 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:44 pg_stat 2023-09-23T09:45:02.450 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:44 pg_stat_tmp 2023-09-23T09:45:02.450 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:44 pg_subtrans 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_tblspc 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pg_twophase 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 23 09:44 pg_wal 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:37 pg_xact 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 23 09:29 postgresql.auto.conf 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 23 09:29 postgresql.conf 2023-09-23T09:45:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 23 09:29 postmaster.opts 2023-09-23T09:45:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.452 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base: 2023-09-23T09:45:02.452 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.452 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 23 09:29 . 2023-09-23T09:45:02.452 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.452 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 1 2023-09-23T09:45:02.452 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 12346 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Sep 23 09:29 12347 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 pgsql_tmp 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:total 7201 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 . 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 23 09:29 .. 2023-09-23T09:45:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 112 2023-09-23T09:45:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 113 2023-09-23T09:45:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 23 09:29 12182 2023-09-23T09:45:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12182_fsm 2023-09-23T09:45:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12182_vm 2023-09-23T09:45:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12184 2023-09-23T09:45:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12186 2023-09-23T09:45:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12187_fsm 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187_vm 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12189 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12191 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12192_fsm 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192_vm 2023-09-23T09:45:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12194 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12196 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12197_fsm 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197_vm 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12199 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12201 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202 2023-09-23T09:45:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12202_fsm 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202_vm 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12204 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12206 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12207_fsm 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207_vm 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12209 2023-09-23T09:45:02.457 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12211 2023-09-23T09:45:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12212 2023-09-23T09:45:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12214 2023-09-23T09:45:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12216 2023-09-23T09:45:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 23 09:29 1247 2023-09-23T09:45:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1247_fsm 2023-09-23T09:45:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1247_vm 2023-09-23T09:45:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 23 09:29 1249 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1249_fsm 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1249_vm 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 23 09:29 1255 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1255_fsm 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1255_vm 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 23 09:29 1259 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1259_fsm 2023-09-23T09:45:02.459 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1259_vm 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417_vm 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418_vm 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 174 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 175 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2187 2023-09-23T09:45:02.460 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224_vm 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328_vm 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336_vm 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2337 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2600 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2600_fsm 2023-09-23T09:45:02.461 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2600_vm 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2601_fsm 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601_vm 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2602 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2602_fsm 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2602_vm 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2603 2023-09-23T09:45:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2603_fsm 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2603_vm 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604_vm 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2605 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2605_fsm 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2605_vm 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606 2023-09-23T09:45:02.463 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2606_fsm 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606_vm 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607_fsm 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2607_vm 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 23 09:29 2608 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2608_fsm 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2608_vm 2023-09-23T09:45:02.464 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 23 09:29 2609 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2609_fsm 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2609_vm 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610_fsm 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2610_vm 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611_vm 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612 2023-09-23T09:45:02.465 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2612_fsm 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612_vm 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613_vm 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2615_fsm 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615_vm 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616 2023-09-23T09:45:02.466 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616_fsm 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2616_vm 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 23 09:29 2617 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2617_fsm 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2617_vm 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 23 09:29 2618 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2618_fsm 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2618_vm 2023-09-23T09:45:02.467 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 23 09:29 2619 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2619_fsm 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2619_vm 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620_vm 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2650 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2651 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2652 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2653 2023-09-23T09:45:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2654 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2655 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2656 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2657 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 23 09:29 2658 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2659 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2660 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2661 2023-09-23T09:45:02.469 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2662 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2663 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2664 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2665 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2666 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2667 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2668 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2669 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2670 2023-09-23T09:45:02.470 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Sep 23 09:29 2673 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 23 09:29 2674 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 23 09:29 2675 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2678 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2679 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2680 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2681 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2682 2023-09-23T09:45:02.471 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2683 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2684 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2685 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2686 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2687 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2688 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2689 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2690 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 23 09:29 2691 2023-09-23T09:45:02.472 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2692 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2693 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2696 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2699 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2701 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2702 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2703 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2704 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2753 2023-09-23T09:45:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2753_fsm 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2753_vm 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2754 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2755 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2756 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2757 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830_vm 2023-09-23T09:45:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2831 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832_vm 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2833 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834_vm 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2835 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836_vm 2023-09-23T09:45:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2837 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 23 09:29 2838 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2838_fsm 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2838_vm 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2839 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840_fsm 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2840_vm 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2841 2023-09-23T09:45:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995_vm 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2996 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3079_fsm 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079_vm 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3080 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3081 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3085 2023-09-23T09:45:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118_vm 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3119 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3164 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256_vm 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3257 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3258 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350 2023-09-23T09:45:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350_vm 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3351 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3379 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3380 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381_vm 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3394 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3394_fsm 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3394_vm 2023-09-23T09:45:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3395 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439_vm 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3440 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 3455 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3456 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3456_fsm 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3456_vm 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466 2023-09-23T09:45:02.480 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466_vm 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3467 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3468 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501_vm 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3502 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3503 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3534 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3541_fsm 2023-09-23T09:45:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541_vm 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3542 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3574 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3575 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576_vm 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596_vm 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3597 2023-09-23T09:45:02.482 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598_vm 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3599 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3600_fsm 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600_vm 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3601_fsm 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601_vm 2023-09-23T09:45:02.483 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602 2023-09-23T09:45:02.484 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3602_fsm 2023-09-23T09:45:02.484 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602_vm 2023-09-23T09:45:02.484 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3603 2023-09-23T09:45:02.484 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3603_fsm 2023-09-23T09:45:02.484 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3603_vm 2023-09-23T09:45:02.485 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3604 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3605 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3606 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3607 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3608 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 3609 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3712 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764 2023-09-23T09:45:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3764_fsm 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764_vm 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3766 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3767 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3997 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 5002 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 548 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 549 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102 2023-09-23T09:45:02.487 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102_vm 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104_vm 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106_vm 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6110 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6111 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6112 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6113 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6117 2023-09-23T09:45:02.488 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826_vm 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 827 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 828 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:total 7201 2023-09-23T09:45:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 23 09:29 . 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 23 09:29 .. 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 112 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 113 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 23 09:29 12182 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12182_fsm 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12182_vm 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12184 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12186 2023-09-23T09:45:02.490 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12187_fsm 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187_vm 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12189 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12191 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12192_fsm 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192_vm 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12194 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12196 2023-09-23T09:45:02.491 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12197_fsm 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197_vm 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12199 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12201 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12202_fsm 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202_vm 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12204 2023-09-23T09:45:02.492 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12206 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12207_fsm 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207_vm 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12209 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12211 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12212 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12214 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12216 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 23 09:29 1247 2023-09-23T09:45:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1247_fsm 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1247_vm 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 23 09:29 1249 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1249_fsm 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1249_vm 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 23 09:29 1255 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1255_fsm 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1255_vm 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 23 09:29 1259 2023-09-23T09:45:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1259_fsm 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1259_vm 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417_vm 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418_vm 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 174 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 175 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2187 2023-09-23T09:45:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224_vm 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328_vm 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336_vm 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2337 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2600 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2600_fsm 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2600_vm 2023-09-23T09:45:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2601_fsm 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601_vm 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2602 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2602_fsm 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2602_vm 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2603 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2603_fsm 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2603_vm 2023-09-23T09:45:02.497 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604_vm 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2605 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2605_fsm 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2605_vm 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2606_fsm 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606_vm 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607_fsm 2023-09-23T09:45:02.498 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2607_vm 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 23 09:29 2608 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2608_fsm 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2608_vm 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 23 09:29 2609 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2609_fsm 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2609_vm 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610_fsm 2023-09-23T09:45:02.499 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2610_vm 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611_vm 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2612_fsm 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612_vm 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613_vm 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2615_fsm 2023-09-23T09:45:02.500 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615_vm 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616_fsm 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2616_vm 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 23 09:29 2617 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2617_fsm 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2617_vm 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 23 09:29 2618 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2618_fsm 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2618_vm 2023-09-23T09:45:02.501 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 23 09:29 2619 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2619_fsm 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2619_vm 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620_vm 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2650 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2651 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2652 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2653 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2654 2023-09-23T09:45:02.502 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2655 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2656 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2657 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 23 09:29 2658 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2659 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2660 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2661 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2662 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2663 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2664 2023-09-23T09:45:02.503 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2665 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2666 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2667 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2668 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2669 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2670 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Sep 23 09:29 2673 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 23 09:29 2674 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 23 09:29 2675 2023-09-23T09:45:02.504 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2678 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2679 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2680 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2681 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2682 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2683 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2684 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2685 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2686 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2687 2023-09-23T09:45:02.505 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2688 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2689 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2690 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 23 09:29 2691 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2692 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2693 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2696 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2699 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2701 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2702 2023-09-23T09:45:02.506 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2703 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2704 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2753 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2753_fsm 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2753_vm 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2754 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2755 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2756 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2757 2023-09-23T09:45:02.507 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830_vm 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2831 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832_vm 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2833 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834_vm 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2835 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836 2023-09-23T09:45:02.508 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836_vm 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2837 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 23 09:29 2838 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2838_fsm 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2838_vm 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2839 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840_fsm 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2840_vm 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2841 2023-09-23T09:45:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995_vm 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2996 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3079_fsm 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079_vm 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3080 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3081 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3085 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118 2023-09-23T09:45:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118_vm 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3119 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3164 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256_vm 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3257 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3258 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350_vm 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3351 2023-09-23T09:45:02.511 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3379 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3380 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381_vm 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3394 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3394_fsm 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3394_vm 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3395 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439_vm 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3440 2023-09-23T09:45:02.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 3455 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3456 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3456_fsm 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3456_vm 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466_vm 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3467 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3468 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501 2023-09-23T09:45:02.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501_vm 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3502 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3503 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3534 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3541_fsm 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541_vm 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3542 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3574 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3575 2023-09-23T09:45:02.514 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576_vm 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596_vm 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3597 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598_vm 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3599 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3600_fsm 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600_vm 2023-09-23T09:45:02.515 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3601_fsm 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601_vm 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3602_fsm 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602_vm 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3603 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3603_fsm 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3603_vm 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3604 2023-09-23T09:45:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3605 2023-09-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3606 2023-09-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3607 2023-09-23T09:45:02.517 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3608 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 3609 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3712 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3764_fsm 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764_vm 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3766 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3767 2023-09-23T09:45:02.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3997 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 5002 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 548 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 549 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102_vm 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104_vm 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106_vm 2023-09-23T09:45:02.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6110 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6111 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6112 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6113 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6117 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826_vm 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 827 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 828 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:45:02.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:total 633328 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Sep 23 09:29 . 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 23 09:29 .. 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 112 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 113 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 23 09:29 12182 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12182_fsm 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12182_vm 2023-09-23T09:45:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12184 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12186 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12187_fsm 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12187_vm 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12189 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12191 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12192_fsm 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12192_vm 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12194 2023-09-23T09:45:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12196 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12197_fsm 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12197_vm 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12199 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12201 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12202_fsm 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12202_vm 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12204 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12206 2023-09-23T09:45:02.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 12207_fsm 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12207_vm 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12209 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12211 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12212 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 12214 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 12216 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 23 09:29 1247 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1247_fsm 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1247_vm 2023-09-23T09:45:02.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 23 09:29 1249 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1249_fsm 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1249_vm 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 23 09:29 1255 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1255_fsm 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1255_vm 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 23 09:44 1259 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1259_fsm 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1259_vm 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417 2023-09-23T09:45:02.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1417_vm 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 1418_vm 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 5529600 Sep 23 09:44 16387 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:44 16387_fsm 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:44 16387_vm 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 7888896 Sep 23 09:44 16393 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:44 16393_fsm 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:44 16393_vm 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 465428480 Sep 23 09:44 16396 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 23 09:44 16396_fsm 2023-09-23T09:45:02.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:32 16396_vm 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 23 09:44 16397 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 565248 Sep 23 09:44 16399 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71917568 Sep 23 09:44 16401 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 89276416 Sep 23 09:44 16403 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:44 16403_fsm 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 174 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 175 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2187 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2224_vm 2023-09-23T09:45:02.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2328_vm 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2336_vm 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2337 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2600 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2600_fsm 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2600_vm 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601 2023-09-23T09:45:02.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2601_fsm 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2601_vm 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2602 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2602_fsm 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2602_vm 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2603 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2603_fsm 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2603_vm 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2604_vm 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2605 2023-09-23T09:45:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2605_fsm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2605_vm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2606_fsm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2606_vm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2607_fsm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2607_vm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Sep 23 09:29 2608 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2608_fsm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2608_vm 2023-09-23T09:45:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 23 09:29 2609 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2609_fsm 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2609_vm 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2610 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2610_fsm 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2610_vm 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2611_vm 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2612_fsm 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2612_vm 2023-09-23T09:45:02.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2613_vm 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2615_fsm 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2615_vm 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2616_fsm 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2616_vm 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 23 09:29 2617 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2617_fsm 2023-09-23T09:45:02.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2617_vm 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 23 09:29 2618 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2618_fsm 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2618_vm 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Sep 23 09:44 2619 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:34 2619_fsm 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2619_vm 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2620_vm 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2650 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2651 2023-09-23T09:45:02.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2652 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2653 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2654 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2655 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2656 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2657 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 23 09:29 2658 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2659 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2660 2023-09-23T09:45:02.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2661 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2662 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2663 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2664 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2665 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2666 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2667 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2668 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2669 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2670 2023-09-23T09:45:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Sep 23 09:29 2673 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 23 09:29 2674 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 23 09:29 2675 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2678 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2679 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2680 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2681 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2682 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2683 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2684 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2685 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2686 2023-09-23T09:45:02.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2687 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2688 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 23 09:29 2689 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 23 09:29 2690 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 23 09:29 2691 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2692 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2693 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:35 2696 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2699 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2701 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2702 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2703 2023-09-23T09:45:02.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 2704 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2753 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2753_fsm 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2753_vm 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2754 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2755 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2756 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 2757 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2830_vm 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2831 2023-09-23T09:45:02.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2832_vm 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2833 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2834_vm 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2835 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2836_vm 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2837 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 23 09:29 2838 2023-09-23T09:45:02.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2838_fsm 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2838_vm 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2839 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2840_fsm 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2840_vm 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2841 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2995_vm 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2996 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3079_fsm 2023-09-23T09:45:02.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3079_vm 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3080 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3081 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3085 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3118_vm 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3119 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3164 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3256_vm 2023-09-23T09:45:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3257 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3258 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3350_vm 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3351 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3379 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3380 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3381_vm 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3394 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3394_fsm 2023-09-23T09:45:02.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3394_vm 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3395 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3439_vm 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3440 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 23 09:29 3455 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3456 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3456_fsm 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3456_vm 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3466_vm 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3467 2023-09-23T09:45:02.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3468 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3501_vm 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3502 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3503 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3534 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3541_fsm 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3541_vm 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3542 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3574 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3575 2023-09-23T09:45:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3576_vm 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3596_vm 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3597 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3598_vm 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3599 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3600_fsm 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3600_vm 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601 2023-09-23T09:45:02.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3601_fsm 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3601_vm 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3602_fsm 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3602_vm 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3603 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3603_fsm 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3603_vm 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3604 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3605 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3606 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3607 2023-09-23T09:45:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3608 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 23 09:29 3609 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3712 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 3764_fsm 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3764_vm 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3766 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 3767 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3997 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 5002 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 548 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 549 2023-09-23T09:45:02.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6102_vm 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6104_vm 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6106_vm 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6110 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6111 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6112 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6113 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6117 2023-09-23T09:45:02.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826 2023-09-23T09:45:02.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 826_vm 2023-09-23T09:45:02.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 827 2023-09-23T09:45:02.564 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:02 smithi098 ceph-mon[102386]: pgmap v489: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 828 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Sep 23 09:29 pg_internal.init 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:29 PG_VERSION 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 23 09:29 .. 2023-09-23T09:45:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/global: 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:total 574 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Sep 23 09:29 . 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1136 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1136_fsm 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1136_vm 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 1137 2023-09-23T09:45:02.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1213 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1213_fsm 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1213_vm 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1214 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1214_fsm 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1214_vm 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 1232 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 1233 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1260 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1260_fsm 2023-09-23T09:45:02.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1260_vm 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1261 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1261_fsm 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1261_vm 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1262 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 1262_fsm 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 1262_vm 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2396 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 23 09:29 2396_fsm 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2396_vm 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2397 2023-09-23T09:45:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2671 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2672 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2676 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2677 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2694 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2695 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2697 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 23 09:29 2698 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2846 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2846_vm 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2847 2023-09-23T09:45:02.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2964 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2964_vm 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2965 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2966 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 2966_vm 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 2967 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3592 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 3592_vm 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 3593 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 4060 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 4060_vm 2023-09-23T09:45:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 4061 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6000 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6000_vm 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6001 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6002 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6100 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 6100_vm 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6114 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 6115 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:44 pg_control 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 23 09:29 pg_filenode.map 2023-09-23T09:45:02.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Sep 23 09:29 pg_internal.init 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/log: 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:total 3 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2314 Sep 23 09:44 postgresql-Sat.log 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.572 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:total 1 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:44 . 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.573 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 mappings 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 23 09:44 replorigin_checkpoint 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 snapshots 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:44 .. 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.574 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.575 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:44 .. 2023-09-23T09:45:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.575 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-23T09:45:02.575 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 . 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 members 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 offsets 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 .. 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:45:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:29 .. 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:29 . 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.577 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 23 09:29 0000 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:total 13 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 23 09:44 . 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Sep 23 09:44 db_0.stat 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Sep 23 09:44 db_12347.stat 2023-09-23T09:45:02.579 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Sep 23 09:44 global.stat 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:44 . 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:total 248 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 23 09:44 . 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 23 09:44 0019 2023-09-23T09:45:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-23T09:45:02.581 INFO:teuthology.orchestra.run.smithi045.stdout:total 1048576 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 23 09:44 . 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000F5 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000F6 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000F7 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000F8 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000F9 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000FA 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000FB 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000FC 2023-09-23T09:45:02.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000FD 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000FE 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 0000000100000001000000FF 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 000000010000000200000000 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 000000010000000200000001 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 000000010000000200000002 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000003 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000004 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000005 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000006 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000007 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:41 000000010000000200000008 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000009 2023-09-23T09:45:02.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 00000001000000020000000A 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 00000001000000020000000B 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 00000001000000020000000C 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 00000001000000020000000D 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:41 00000001000000020000000E 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:41 00000001000000020000000F 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:41 000000010000000200000010 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000011 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000012 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000013 2023-09-23T09:45:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000014 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000015 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000016 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000017 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000018 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 000000010000000200000019 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:41 00000001000000020000001A 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:42 00000001000000020000001B 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000001C 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 00000001000000020000001D 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000001E 2023-09-23T09:45:02.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000001F 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000020 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000021 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 000000010000000200000022 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000023 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 000000010000000200000024 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000025 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000026 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000027 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000028 2023-09-23T09:45:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:44 000000010000000200000029 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000002A 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000002B 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000002C 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000002D 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000002E 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 00000001000000020000002F 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000030 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000031 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000032 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000033 2023-09-23T09:45:02.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 23 09:43 000000010000000200000034 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 archive_status 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 23 09:29 . 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 23 09:44 .. 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:total 416 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 23 09:37 . 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 23 09:44 .. 2023-09-23T09:45:02.588 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 23 09:37 0000 2023-09-23T09:45:02.589 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 23 09:44 0001 2023-09-23T09:45:02.589 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:02 smithi045 ceph-mon[98627]: pgmap v489: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-09-23T09:45:02.590 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-09-23T09:45:02.600 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:45:02.601 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:02 smithi123 ceph-mon[103577]: pgmap v489: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-09-23T09:45:02.608 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-09-23T09:45:02.651 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-09-23T09:45:02.651 INFO:teuthology.task.exec:Executing custom commands... 2023-09-23T09:45:02.651 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:45:02.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-09-23T09:45:02.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-09-23T09:45:02.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-23T09:45:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 -- 172.21.15.45:0/263403988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c10d690 msgr2=0x7f1d6c10fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 --2- 172.21.15.45:0/263403988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c10d690 0x7f1d6c10fa80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b0a0 tx=0x7f1d6400b3b0 comp rx=0 tx=0).stop 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 -- 172.21.15.45:0/263403988 shutdown_connections 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 --2- 172.21.15.45:0/263403988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c10d690 0x7f1d6c10fa80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 --2- 172.21.15.45:0/263403988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6c101d10 0x7f1d6c068210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 --2- 172.21.15.45:0/263403988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c0687e0 0x7f1d6c068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 -- 172.21.15.45:0/263403988 >> 172.21.15.45:0/263403988 conn(0x7f1d6c076840 msgr2=0x7f1d6c076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 -- 172.21.15.45:0/263403988 shutdown_connections 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 -- 172.21.15.45:0/263403988 wait complete. 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.913+0000 7f1d71852700 1 Processor -- start 2023-09-23T09:45:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d71852700 1 -- start start 2023-09-23T09:45:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d71852700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0687e0 0x7f1d6c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d71852700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6c10d690 0x7f1d6c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d71852700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c19dd60 0x7f1d6c19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d71852700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c19e7e0 con 0x7f1d6c0687e0 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d71852700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c19e950 con 0x7f1d6c10d690 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d71852700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c19eb10 con 0x7f1d6c19dd60 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0687e0 0x7f1d6c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0687e0 0x7f1d6c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37218/0 (socket says 172.21.15.45:37218) 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 -- 172.21.15.45:0/1902921422 learned_addr learned my addr 172.21.15.45:0/1902921422 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6a7fc700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6c10d690 0x7f1d6c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6b7fe700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c19dd60 0x7f1d6c19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 -- 172.21.15.45:0/1902921422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c19dd60 msgr2=0x7f1d6c19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c19dd60 0x7f1d6c19e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:02.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 -- 172.21.15.45:0/1902921422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6c10d690 msgr2=0x7f1d6c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6c10d690 0x7f1d6c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 -- 172.21.15.45:0/1902921422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60009520 con 0x7f1d6c0687e0 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6a7fc700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6c10d690 0x7f1d6c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.914+0000 7f1d6affd700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0687e0 0x7f1d6c1991b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9a0 tx=0x7f1d5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.915+0000 7f1d70850700 1 -- 172.21.15.45:0/1902921422 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c009790 con 0x7f1d6c0687e0 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.915+0000 7f1d70850700 1 -- 172.21.15.45:0/1902921422 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5c010430 con 0x7f1d6c0687e0 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.915+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d64009d00 con 0x7f1d6c0687e0 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.915+0000 7f1d70850700 1 -- 172.21.15.45:0/1902921422 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c018680 con 0x7f1d6c0687e0 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.915+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c1a6410 con 0x7f1d6c0687e0 2023-09-23T09:45:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.916+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c077420 con 0x7f1d6c0687e0 2023-09-23T09:45:02.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.917+0000 7f1d70850700 1 -- 172.21.15.45:0/1902921422 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f1d5c0105a0 con 0x7f1d6c0687e0 2023-09-23T09:45:02.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.918+0000 7f1d70850700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1d540a1790 0x7f1d540a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:02.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.918+0000 7f1d6a7fc700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1d540a1790 0x7f1d540a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:02.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.918+0000 7f1d70850700 1 -- 172.21.15.45:0/1902921422 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1d5c014070 con 0x7f1d6c0687e0 2023-09-23T09:45:02.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.918+0000 7f1d6a7fc700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1d540a1790 0x7f1d540a3c50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009fd0 tx=0x7f1d60009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:02.922+0000 7f1d70850700 1 -- 172.21.15.45:0/1902921422 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1d5c096680 con 0x7f1d6c0687e0 2023-09-23T09:45:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.155+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 --> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d6c101d60 con 0x7f1d540a1790 2023-09-23T09:45:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.163+0000 7f1d70850700 1 -- 172.21.15.45:0/1902921422 <== mgr.15405 v2:172.21.15.45:6800/441481757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f1d6c101d60 con 0x7f1d540a1790 2023-09-23T09:45:03.164 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-09-23T00:00:00", "created": "2023-09-23T09:29:26", "first": "2023-09-23T09:30:00", "last": "2023-09-23T09:45:00", "last_pruned": "2023-09-23T09:45:00", "created_count": 16, "pruned_count": 10, "active": true} 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1d540a1790 msgr2=0x7f1d540a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1d540a1790 0x7f1d540a3c50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009fd0 tx=0x7f1d60009260 comp rx=0 tx=0).stop 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0687e0 msgr2=0x7f1d6c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0687e0 0x7f1d6c1991b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9a0 tx=0x7f1d5c00ecb0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 shutdown_connections 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c19dd60 0x7f1d6c19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1d540a1790 0x7f1d540a3c50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0687e0 0x7f1d6c1991b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 --2- 172.21.15.45:0/1902921422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6c10d690 0x7f1d6c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 >> 172.21.15.45:0/1902921422 conn(0x7f1d6c076840 msgr2=0x7f1d6c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.165+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 shutdown_connections 2023-09-23T09:45:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.166+0000 7f1d71852700 1 -- 172.21.15.45:0/1902921422 wait complete. 2023-09-23T09:45:03.179 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-23T09:45:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 -- 172.21.15.45:0/3549933856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 msgr2=0x7f1a3c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 --2- 172.21.15.45:0/3549933856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 0x7f1a3c1013d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009990 tx=0x7f1a30009ca0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 -- 172.21.15.45:0/3549933856 shutdown_connections 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 --2- 172.21.15.45:0/3549933856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 0x7f1a3c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 --2- 172.21.15.45:0/3549933856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 0x7f1a3c1013d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 --2- 172.21.15.45:0/3549933856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a3c1019a0 0x7f1a3c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 -- 172.21.15.45:0/3549933856 >> 172.21.15.45:0/3549933856 conn(0x7f1a3c0fc870 msgr2=0x7f1a3c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 -- 172.21.15.45:0/3549933856 shutdown_connections 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.305+0000 7f1a41374700 1 -- 172.21.15.45:0/3549933856 wait complete. 2023-09-23T09:45:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 Processor -- start 2023-09-23T09:45:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 -- start start 2023-09-23T09:45:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 0x7f1a3c196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 0x7f1a3c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a3c19ba70 0x7f1a3c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c19c4f0 con 0x7f1a3c106110 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c19c660 con 0x7f1a3c19ba70 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a41374700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c19c820 con 0x7f1a3c100ff0 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 0x7f1a3c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 0x7f1a3c196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 0x7f1a3c197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37234/0 (socket says 172.21.15.45:37234) 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 -- 172.21.15.45:0/1599229857 learned_addr learned my addr 172.21.15.45:0/1599229857 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3b7fe700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a3c19ba70 0x7f1a3c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 -- 172.21.15.45:0/1599229857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 msgr2=0x7f1a3c196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 0x7f1a3c196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 -- 172.21.15.45:0/1599229857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a3c19ba70 msgr2=0x7f1a3c19bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a3c19ba70 0x7f1a3c19bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3a7fc700 1 -- 172.21.15.45:0/1599229857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c009520 con 0x7f1a3c106110 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3b7fe700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a3c19ba70 0x7f1a3c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.306+0000 7f1a3affd700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 0x7f1a3c196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.307+0000 7f1a3a7fc700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 0x7f1a3c197450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00ea30 tx=0x7f1a2c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.307+0000 7f1a23fff700 1 -- 172.21.15.45:0/1599229857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00cbd0 con 0x7f1a3c106110 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.307+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a34009c40 con 0x7f1a3c106110 2023-09-23T09:45:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.307+0000 7f1a23fff700 1 -- 172.21.15.45:0/1599229857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a2c00eec0 con 0x7f1a3c106110 2023-09-23T09:45:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.307+0000 7f1a23fff700 1 -- 172.21.15.45:0/1599229857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c003d20 con 0x7f1a3c106110 2023-09-23T09:45:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.307+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a300095f0 con 0x7f1a3c106110 2023-09-23T09:45:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.308+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c1004c0 con 0x7f1a3c106110 2023-09-23T09:45:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.309+0000 7f1a23fff700 1 -- 172.21.15.45:0/1599229857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f1a2c003ea0 con 0x7f1a3c106110 2023-09-23T09:45:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.309+0000 7f1a23fff700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1a240a1710 0x7f1a240a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:03.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.310+0000 7f1a3affd700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1a240a1710 0x7f1a240a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:03.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.310+0000 7f1a23fff700 1 -- 172.21.15.45:0/1599229857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1a2c014070 con 0x7f1a3c106110 2023-09-23T09:45:03.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.310+0000 7f1a3affd700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1a240a1710 0x7f1a240a3bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000b2e0 tx=0x7f1a30005960 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.313+0000 7f1a23fff700 1 -- 172.21.15.45:0/1599229857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1a2c096a20 con 0x7f1a3c106110 2023-09-23T09:45:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.540+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 --> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1a3c101a10 con 0x7f1a240a1710 2023-09-23T09:45:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:03 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:45:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:03 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:45:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:03 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:45:03.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:03 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:45:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.547+0000 7f1a23fff700 1 -- 172.21.15.45:0/1599229857 <== mgr.15405 v2:172.21.15.45:6800/441481757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f1a3c101a10 con 0x7f1a240a1710 2023-09-23T09:45:03.548 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2023-09-23T09:45:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1a240a1710 msgr2=0x7f1a240a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1a240a1710 0x7f1a240a3bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000b2e0 tx=0x7f1a30005960 comp rx=0 tx=0).stop 2023-09-23T09:45:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 msgr2=0x7f1a3c197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 0x7f1a3c197450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00ea30 tx=0x7f1a2c00ed40 comp rx=0 tx=0).stop 2023-09-23T09:45:03.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 shutdown_connections 2023-09-23T09:45:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a3c19ba70 0x7f1a3c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c100ff0 0x7f1a3c196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.549+0000 7f1a41374700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1a240a1710 0x7f1a240a3bd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.550+0000 7f1a41374700 1 --2- 172.21.15.45:0/1599229857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a3c106110 0x7f1a3c197450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.550+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 >> 172.21.15.45:0/1599229857 conn(0x7f1a3c0fc870 msgr2=0x7f1a3c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.550+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 shutdown_connections 2023-09-23T09:45:03.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:03.550+0000 7f1a41374700 1 -- 172.21.15.45:0/1599229857 wait complete. 2023-09-23T09:45:03.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:03 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:45:03.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:03 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:45:03.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:03 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:45:03.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:03 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:45:03.563 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-23T09:45:03.573 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-23T09:45:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:03 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:45:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:03 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:45:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:03 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:45:03.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:03 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:45:04.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:04 smithi098 ceph-mon[102386]: pgmap v490: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 8.4 MiB/s wr, 839 op/s 2023-09-23T09:45:04.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:04 smithi098 ceph-mon[102386]: from='client.17163 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:45:04.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:04 smithi045 ceph-mon[98627]: pgmap v490: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 8.4 MiB/s wr, 839 op/s 2023-09-23T09:45:04.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:04 smithi045 ceph-mon[98627]: from='client.17163 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:45:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:04 smithi123 ceph-mon[103577]: pgmap v490: 97 pgs: 97 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 8.4 MiB/s wr, 839 op/s 2023-09-23T09:45:04.589 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:04 smithi123 ceph-mon[103577]: from='client.17163 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:45:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:05 smithi098 ceph-mon[102386]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:45:05.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:05 smithi098 ceph-mon[102386]: osdmap e106: 12 total, 12 up, 12 in 2023-09-23T09:45:05.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:05 smithi045 ceph-mon[98627]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:45:05.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:05 smithi045 ceph-mon[98627]: osdmap e106: 12 total, 12 up, 12 in 2023-09-23T09:45:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:05 smithi123 ceph-mon[103577]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T09:45:05.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:05 smithi123 ceph-mon[103577]: osdmap e106: 12 total, 12 up, 12 in 2023-09-23T09:45:06.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:06 smithi098 ceph-mon[102386]: pgmap v492: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.5 MiB/s wr, 472 op/s 2023-09-23T09:45:06.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:06 smithi045 ceph-mon[98627]: pgmap v492: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.5 MiB/s wr, 472 op/s 2023-09-23T09:45:06.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:06 smithi123 ceph-mon[103577]: pgmap v492: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.5 MiB/s wr, 472 op/s 2023-09-23T09:45:08.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:08 smithi098 ceph-mon[102386]: pgmap v493: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 301 KiB/s wr, 56 op/s 2023-09-23T09:45:08.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:08 smithi045 ceph-mon[98627]: pgmap v493: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 301 KiB/s wr, 56 op/s 2023-09-23T09:45:08.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:08 smithi123 ceph-mon[103577]: pgmap v493: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 301 KiB/s wr, 56 op/s 2023-09-23T09:45:10.495 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T09:45:10.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:10 smithi045 ceph-mon[98627]: pgmap v494: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 37 op/s 2023-09-23T09:45:10.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:10 smithi098 ceph-mon[102386]: pgmap v494: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 37 op/s 2023-09-23T09:45:10.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:10 smithi123 ceph-mon[103577]: pgmap v494: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 37 op/s 2023-09-23T09:45:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.636+0000 7f0111131700 1 -- 172.21.15.45:0/3957605689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c100880 msgr2=0x7f010c1048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.636+0000 7f0111131700 1 --2- 172.21.15.45:0/3957605689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c100880 0x7f010c1048c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009990 tx=0x7f00f4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.636+0000 7f0111131700 1 -- 172.21.15.45:0/3957605689 shutdown_connections 2023-09-23T09:45:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.636+0000 7f0111131700 1 --2- 172.21.15.45:0/3957605689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f010c104ff0 0x7f010c10cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.636+0000 7f0111131700 1 --2- 172.21.15.45:0/3957605689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c0ffed0 0x7f010c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.636+0000 7f0111131700 1 --2- 172.21.15.45:0/3957605689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c100880 0x7f010c1048c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.636+0000 7f0111131700 1 -- 172.21.15.45:0/3957605689 >> 172.21.15.45:0/3957605689 conn(0x7f010c0fb770 msgr2=0x7f010c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 -- 172.21.15.45:0/3957605689 shutdown_connections 2023-09-23T09:45:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 -- 172.21.15.45:0/3957605689 wait complete. 2023-09-23T09:45:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 Processor -- start 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 -- start start 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f010c0ffed0 0x7f010c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c104ff0 0x7f010c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c19b960 0x7f010c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19c3e0 con 0x7f010c19b960 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19c550 con 0x7f010c0ffed0 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.637+0000 7f0111131700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19c710 con 0x7f010c104ff0 2023-09-23T09:45:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c104ff0 0x7f010c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f010b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c19b960 0x7f010c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f010c0ffed0 0x7f010c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c104ff0 0x7f010c197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39354/0 (socket says 172.21.15.45:39354) 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 -- 172.21.15.45:0/3726571716 learned_addr learned my addr 172.21.15.45:0/3726571716 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 -- 172.21.15.45:0/3726571716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f010c0ffed0 msgr2=0x7f010c196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f010c0ffed0 0x7f010c196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 -- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c19b960 msgr2=0x7f010c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c19b960 0x7f010c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 -- 172.21.15.45:0/3726571716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40095f0 con 0x7f010c104ff0 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f010b59e700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c19b960 0x7f010c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0103fff700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c104ff0 0x7f010c197500 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f00f40059d0 tx=0x7f00f4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0108d99700 1 -- 172.21.15.45:0/3726571716 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f401d070 con 0x7f010c104ff0 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c1a3c10 con 0x7f010c104ff0 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0108d99700 1 -- 172.21.15.45:0/3726571716 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f4022470 con 0x7f010c104ff0 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0108d99700 1 -- 172.21.15.45:0/3726571716 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4017650 con 0x7f010c104ff0 2023-09-23T09:45:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.638+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1a4090 con 0x7f010c104ff0 2023-09-23T09:45:10.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.641+0000 7f0108d99700 1 -- 172.21.15.45:0/3726571716 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f00f40225e0 con 0x7f010c104ff0 2023-09-23T09:45:10.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.641+0000 7f0108d99700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f00f80a1710 0x7f00f80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.641+0000 7f0108d99700 1 -- 172.21.15.45:0/3726571716 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f00f40ce7d0 con 0x7f010c104ff0 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.641+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f00f80a1710 0x7f00f80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f00f80a1710 0x7f00f80a3bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009c60 tx=0x7f00fc009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 Processor -- start 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 -- start start 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce2867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce295aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce2983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1ef460 con 0x558dce286400 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1e0c20 con 0x558dce286d20 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f011d31bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1de540 con 0x558dce295fe0 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.642+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce2867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f0103fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce295aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce2867e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37254/0 (socket says 172.21.15.45:37254) 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce2983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 -- 172.21.15.45:0/1692333058 learned_addr learned my addr 172.21.15.45:0/1692333058 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 -- 172.21.15.45:0/1692333058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 msgr2=0x558dce2983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce2983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 -- 172.21.15.45:0/1692333058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 msgr2=0x558dce295aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce295aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 -- 172.21.15.45:0/1692333058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dce23c8d0 con 0x558dce286400 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f0103fff700 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce295aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f010ad9d700 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce2867e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00e4e0 tx=0x7f00fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4847b2759544acf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f01017fa700 1 -- 172.21.15.45:0/1692333058 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00ee00 con 0x558dce286400 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f01017fa700 1 -- 172.21.15.45:0/1692333058 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc021070 con 0x558dce286400 2023-09-23T09:45:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.643+0000 7f01017fa700 1 -- 172.21.15.45:0/1692333058 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc01d7b0 con 0x558dce286400 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 -- 172.21.15.45:0/1692333058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 msgr2=0x558dce2867e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce2867e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00e4e0 tx=0x7f00fc014000 comp rx=0 tx=0).stop 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 -- 172.21.15.45:0/1692333058 shutdown_connections 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce2983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce2867e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 --2- 172.21.15.45:0/1692333058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce295aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 -- 172.21.15.45:0/1692333058 >> 172.21.15.45:0/1692333058 conn(0x558dce1fb260 msgr2=0x558dce282790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 -- 172.21.15.45:0/1692333058 shutdown_connections 2023-09-23T09:45:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 -- 172.21.15.45:0/1692333058 wait complete. 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 -- start start 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.644+0000 7f011d31bb80 1 Processor -- start 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce295a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce292c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce293350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1d9e30 con 0x558dce286400 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce176710 con 0x558dce286d20 2023-09-23T09:45:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1763b0 con 0x558dce295fe0 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 10 client.0 _ll_register_callbacks cb 0x558dcdfd3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce295a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce295a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37260/0 (socket says 172.21.15.45:37260) 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 -- 172.21.15.45:0/3207581722 learned_addr learned my addr 172.21.15.45:0/3207581722 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f0103fff700 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce292c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010b59e700 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce293350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 -- 172.21.15.45:0/3207581722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 msgr2=0x558dce293350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce293350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 -- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 msgr2=0x558dce292c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce292c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 -- 172.21.15.45:0/3207581722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1a4090 con 0x558dce286400 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce295a50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00ec80 tx=0x7f00fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b78577de5911f07 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f00f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f00f27fc700 1 -- 172.21.15.45:0/3207581722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc005250 con 0x558dce286400 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f00f27fc700 1 -- 172.21.15.45:0/3207581722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f00fc0053b0 con 0x558dce286400 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.645+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dce27a250 con 0x558dce286400 2023-09-23T09:45:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f00f27fc700 1 -- 172.21.15.45:0/3207581722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00fc005570 con 0x558dce286400 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 10 client.17181 fetch_fsmap learned FSMap version 58 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7f00fc017070 con 0x558dce286400 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f00f27fc700 1 -- 172.21.15.45:0/3207581722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558dce27a250 con 0x558dce286400 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f00f27fc700 1 -- 172.21.15.45:0/3207581722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 58) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f00fc01c6f0 con 0x558dce286400 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 10 client.17181 fetch_fsmap finished waiting for FSMap version 58 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 20 client.17181 populate_metadata read hostname 'smithi045' 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2a58b0 0x558dce2a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 4 client.17181 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dce1d2e10 con 0x558dce2a58b0 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.646+0000 7f0103fff700 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2a58b0 0x558dce2a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f00f27fc700 10 client.17181 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f0103fff700 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2a58b0 0x558dce2a7cd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c86ece48a3b2f87 server_cookie=465cf9836a8fb949 in_seq=0 out_seq=0 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f00f27fc700 1 -- 172.21.15.45:0/3207581722 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558dce1d2e10 con 0x558dce2a58b0 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f00f27fc700 10 client.17181 handle_command_reply: tid=0 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 client.17181 shutdown 2023-09-23T09:45:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 msgr2=0x558dce295a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce295a50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00ec80 tx=0x7f00fc00ecb0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 shutdown_connections 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2a58b0 0x558dce2a7cd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce295fe0 0x558dce293350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce295a50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3207581722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce292c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 >> 172.21.15.45:0/3207581722 conn(0x558dce1fb260 msgr2=0x558dce282380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f00f27fc700 0 client.17181 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:45:10.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 shutdown_connections 2023-09-23T09:45:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.647+0000 7f011d31bb80 1 -- 172.21.15.45:0/3207581722 wait complete. 2023-09-23T09:45:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.648+0000 7f011d31bb80 20 client.17181 trim_cache size 0 max 16384 2023-09-23T09:45:10.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 Processor -- start 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 -- start start 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce29cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286d20 0x558dce295580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce295fe0 0x558dce292500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce05a920 con 0x558dce286400 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce289380 con 0x558dce295fe0 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f011d31bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce287bf0 con 0x558dce286d20 2023-09-23T09:45:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce29cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce29cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37268/0 (socket says 172.21.15.45:37268) 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 -- 172.21.15.45:0/492787891 learned_addr learned my addr 172.21.15.45:0/492787891 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f0103fff700 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286d20 0x558dce295580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010b59e700 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce295fe0 0x558dce292500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 -- 172.21.15.45:0/492787891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286d20 msgr2=0x558dce295580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286d20 0x558dce295580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 -- 172.21.15.45:0/492787891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce295fe0 msgr2=0x558dce292500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce295fe0 0x558dce292500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010ad9d700 1 -- 172.21.15.45:0/492787891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dce004de0 con 0x558dce286400 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.665+0000 7f010b59e700 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce295fe0 0x558dce292500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f010ad9d700 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce29cb20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f00fc038040 tx=0x7f00fc0264b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b6f574de7fc10 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f0101ffb700 1 -- 172.21.15.45:0/492787891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00ed20 con 0x558dce286400 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f0101ffb700 1 -- 172.21.15.45:0/492787891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc01cc70 con 0x558dce286400 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f0101ffb700 1 -- 172.21.15.45:0/492787891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0038e0 con 0x558dce286400 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f011d31bb80 1 -- 172.21.15.45:0/492787891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 msgr2=0x558dce29cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f011d31bb80 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce29cb20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f00fc038040 tx=0x7f00fc0264b0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f011d31bb80 1 -- 172.21.15.45:0/492787891 shutdown_connections 2023-09-23T09:45:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f011d31bb80 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce295fe0 0x558dce292500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f011d31bb80 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce286400 0x558dce29cb20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.666+0000 7f011d31bb80 1 --2- 172.21.15.45:0/492787891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286d20 0x558dce295580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 -- 172.21.15.45:0/492787891 >> 172.21.15.45:0/492787891 conn(0x558dce1fb260 msgr2=0x558dce28dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 -- 172.21.15.45:0/492787891 shutdown_connections 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 -- 172.21.15.45:0/492787891 wait complete. 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 -- start start 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 Processor -- start 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286400 0x558dce298cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce299230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce295fe0 0x558dce2f8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce299940 con 0x558dce295fe0 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce2f8bd0 con 0x558dce286d20 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce2f8d40 con 0x558dce286400 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f011d31bb80 10 client.0 _ll_register_callbacks cb 0x7f00f4005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286400 0x558dce298cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f010b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce295fe0 0x558dce2f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f0103fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce299230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f0103fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce299230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58626/0 (socket says 172.21.15.45:58626) 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f010b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce295fe0 0x558dce2f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37278/0 (socket says 172.21.15.45:37278) 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f010ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286400 0x558dce298cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39380/0 (socket says 172.21.15.45:39380) 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.667+0000 7f0103fff700 1 -- 172.21.15.45:0/3000594283 learned_addr learned my addr 172.21.15.45:0/3000594283 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f0103fff700 1 -- 172.21.15.45:0/3000594283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286400 msgr2=0x558dce298cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f0103fff700 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286400 0x558dce298cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f0103fff700 1 -- 172.21.15.45:0/3000594283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce295fe0 msgr2=0x558dce2f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f0103fff700 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce295fe0 0x558dce2f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f0103fff700 1 -- 172.21.15.45:0/3000594283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc017070 con 0x558dce286d20 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286400 0x558dce298cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f010b59e700 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce295fe0 0x558dce2f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f0103fff700 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce299230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00f40cbeb0 tx=0x7f00f4022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaaaa6bc28a92298 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f00f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f00f27fc700 1 -- 172.21.15.45:0/3000594283 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4097a40 con 0x558dce286d20 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f00f27fc700 1 -- 172.21.15.45:0/3000594283 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f00f409be00 con 0x558dce286d20 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00fc021070 con 0x558dce286d20 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f00f27fc700 1 -- 172.21.15.45:0/3000594283 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f409b700 con 0x558dce286d20 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f011d31bb80 10 client.26819 fetch_fsmap learned FSMap version 58 2023-09-23T09:45:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.668+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x558dcdfedc60 con 0x558dce286d20 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f00f27fc700 1 -- 172.21.15.45:0/3000594283 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f40b7080 con 0x558dce286d20 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f00f27fc700 1 -- 172.21.15.45:0/3000594283 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 58) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f00f40b0060 con 0x558dce286d20 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f011d31bb80 10 client.26819 fetch_fsmap finished waiting for FSMap version 58 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f011d31bb80 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f011d31bb80 20 client.26819 populate_metadata read hostname 'smithi045' 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2fa1e0 0x558dce2fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f011d31bb80 4 client.26819 mds_command: new command op to 24530 tid=0[{"prefix": "scrub status"}] 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 --> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558dce1d2b30 con 0x558dce2fa1e0 2023-09-23T09:45:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2fa1e0 0x558dce2fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f00f27fc700 10 client.26819 ms_handle_connect on v2:172.21.15.98:6838/2348516337 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f010ad9d700 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2fa1e0 0x558dce2fc600 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11a2d36feb43c21e server_cookie=e673ee2ff1546 in_seq=0 out_seq=0 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f00f27fc700 1 -- 172.21.15.45:0/3000594283 <== mds.0 v2:172.21.15.98:6838/2348516337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558dce1d2b30 con 0x558dce2fa1e0 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f00f27fc700 10 client.26819 handle_command_reply: tid=0 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.669+0000 7f011d31bb80 1 client.26819 shutdown 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 msgr2=0x558dce299230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce299230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00f40cbeb0 tx=0x7f00f4022f90 comp rx=0 tx=0).stop 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 shutdown_connections 2023-09-23T09:45:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:6838/2348516337,v1:172.21.15.98:6839/2348516337] conn(0x558dce2fa1e0 0x558dce2fc600 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce295fe0 0x558dce2f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f00f27fc700 0 client.26819 ms_handle_reset on v2:172.21.15.98:6838/2348516337 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dce286400 0x558dce298cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 --2- 172.21.15.45:0/3000594283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dce286d20 0x558dce299230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 >> 172.21.15.45:0/3000594283 conn(0x558dce1fb260 msgr2=0x558dce288370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 shutdown_connections 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 1 -- 172.21.15.45:0/3000594283 wait complete. 2023-09-23T09:45:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f011d31bb80 20 client.26819 trim_cache size 0 max 16384 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout: "bdc80eb3-adc3-4567-9146-7ffaaf39690b": { 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "bdc80eb3-adc3-4567-9146-7ffaaf39690b", 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-09-23T09:45:10.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:10.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:10.684 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:45:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f00f80a1710 msgr2=0x7f00f80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f00f80a1710 0x7f00f80a3bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009c60 tx=0x7f00fc009260 comp rx=0 tx=0).stop 2023-09-23T09:45:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c104ff0 msgr2=0x7f010c197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c104ff0 0x7f010c197500 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f00f40059d0 tx=0x7f00f4005740 comp rx=0 tx=0).stop 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 shutdown_connections 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f00f80a1710 0x7f00f80a3bd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c19b960 0x7f010c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f010c0ffed0 0x7f010c196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 --2- 172.21.15.45:0/3726571716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c104ff0 0x7f010c197500 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 >> 172.21.15.45:0/3726571716 conn(0x7f010c0fb770 msgr2=0x7f010c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 shutdown_connections 2023-09-23T09:45:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.670+0000 7f0111131700 1 -- 172.21.15.45:0/3726571716 wait complete. 2023-09-23T09:45:10.687 INFO:tasks.cephfs.filesystem:scrub status for tag:bdc80eb3-adc3-4567-9146-7ffaaf39690b - {'path': '/', 'tag': 'bdc80eb3-adc3-4567-9146-7ffaaf39690b', 'options': 'recursive,force'} 2023-09-23T09:45:10.687 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-09-23T09:45:10.715 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-23T09:45:10.716 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-23T09:45:10.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:45:10.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 -- 172.21.15.45:0/3265278138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 msgr2=0x7f0ba40ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 --2- 172.21.15.45:0/3265278138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 0x7f0ba40ffca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009990 tx=0x7f0b94009ca0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 -- 172.21.15.45:0/3265278138 shutdown_connections 2023-09-23T09:45:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 --2- 172.21.15.45:0/3265278138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4104760 0x7f0ba410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 --2- 172.21.15.45:0/3265278138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 0x7f0ba40ffca0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 --2- 172.21.15.45:0/3265278138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 0x7f0ba4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 -- 172.21.15.45:0/3265278138 >> 172.21.15.45:0/3265278138 conn(0x7f0ba4075fb0 msgr2=0x7f0ba40763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 -- 172.21.15.45:0/3265278138 shutdown_connections 2023-09-23T09:45:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.852+0000 7f0ba9743700 1 -- 172.21.15.45:0/3265278138 wait complete. 2023-09-23T09:45:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 Processor -- start 2023-09-23T09:45:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 -- start start 2023-09-23T09:45:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 0x7f0ba419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 0x7f0ba419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4104760 0x7f0ba4195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba4196450 con 0x7f0ba4100270 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba41965c0 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba9743700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba4196780 con 0x7f0ba4104760 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0b9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 0x7f0ba419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0b9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 0x7f0ba419c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37290/0 (socket says 172.21.15.45:37290) 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 0x7f0ba419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0b9bfff700 1 -- 172.21.15.45:0/2642802727 learned_addr learned my addr 172.21.15.45:0/2642802727 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba37fe700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4104760 0x7f0ba4195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba2ffd700 1 -- 172.21.15.45:0/2642802727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4104760 msgr2=0x7f0ba4195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba2ffd700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4104760 0x7f0ba4195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba2ffd700 1 -- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 msgr2=0x7f0ba419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba2ffd700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 0x7f0ba419c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0ba2ffd700 1 -- 172.21.15.45:0/2642802727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940095f0 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.853+0000 7f0b9bfff700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 0x7f0ba419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.854+0000 7f0ba2ffd700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 0x7f0ba419bc80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.854+0000 7f0ba0ff9700 1 -- 172.21.15.45:0/2642802727 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9401d070 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.854+0000 7f0ba0ff9700 1 -- 172.21.15.45:0/2642802727 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b94022470 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.854+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba4196a00 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.854+0000 7f0ba0ff9700 1 -- 172.21.15.45:0/2642802727 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94017650 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.854+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba41a2c70 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.855+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba4067380 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.856+0000 7f0ba0ff9700 1 -- 172.21.15.45:0/2642802727 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f0b940225e0 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.857+0000 7f0ba0ff9700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0b900a17d0 0x7f0b900a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.857+0000 7f0b9bfff700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0b900a17d0 0x7f0b900a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.857+0000 7f0ba0ff9700 1 -- 172.21.15.45:0/2642802727 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0b940ce970 con 0x7f0ba40ff8c0 2023-09-23T09:45:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.857+0000 7f0b9bfff700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0b900a17d0 0x7f0b900a3c90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4197490 tx=0x7f0b8c009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:10.860+0000 7f0ba0ff9700 1 -- 172.21.15.45:0/2642802727 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0b94097be0 con 0x7f0ba40ff8c0 2023-09-23T09:45:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.065+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba410f380 con 0x7f0ba40ff8c0 2023-09-23T09:45:11.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.067+0000 7f0ba0ff9700 1 -- 172.21.15.45:0/2642802727 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+23078 (secure 0 0 0) 0x7f0b9409b200 con 0x7f0ba40ff8c0 2023-09-23T09:45:11.069 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:11.069 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:45:04.163341+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/230875070":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3269510756":"2023-09-24T09:29:25.699095+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/2324483632":"2023-09-24T09:29:25.699095+0000","172.21.15.45:6800/944196506":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/65945615":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/944196506":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/1970790569":"2023-09-24T09:29:25.699095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:45:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0b900a17d0 msgr2=0x7f0b900a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0b900a17d0 0x7f0b900a3c90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4197490 tx=0x7f0b8c009260 comp rx=0 tx=0).stop 2023-09-23T09:45:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 msgr2=0x7f0ba419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 0x7f0ba419bc80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94009580 comp rx=0 tx=0).stop 2023-09-23T09:45:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 shutdown_connections 2023-09-23T09:45:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4104760 0x7f0ba4195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff8c0 0x7f0ba419bc80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f0b900a17d0 0x7f0b900a3c90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 --2- 172.21.15.45:0/2642802727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ba4100270 0x7f0ba419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 >> 172.21.15.45:0/2642802727 conn(0x7f0ba4075fb0 msgr2=0x7f0ba40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 shutdown_connections 2023-09-23T09:45:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.069+0000 7f0ba9743700 1 -- 172.21.15.45:0/2642802727 wait complete. 2023-09-23T09:45:11.084 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:45:11.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:45:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 -- 172.21.15.45:0/492177442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db80ffed0 msgr2=0x7f1db81002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/492177442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db80ffed0 0x7f1db81002b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009990 tx=0x7f1da8009ca0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 -- 172.21.15.45:0/492177442 shutdown_connections 2023-09-23T09:45:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/492177442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db8104ff0 0x7f1db810cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/492177442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db80ffed0 0x7f1db81002b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/492177442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db8100880 0x7f1db81048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 -- 172.21.15.45:0/492177442 >> 172.21.15.45:0/492177442 conn(0x7f1db80fb770 msgr2=0x7f1db80fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 -- 172.21.15.45:0/492177442 shutdown_connections 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.212+0000 7f1dbcf62700 1 -- 172.21.15.45:0/492177442 wait complete. 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 Processor -- start 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 -- start start 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db80ffed0 0x7f1db8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db8104ff0 0x7f1db8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db819daf0 0x7f1db819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819e570 con 0x7f1db8104ff0 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819e6e0 con 0x7f1db80ffed0 2023-09-23T09:45:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1dbcf62700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819e8a0 con 0x7f1db819daf0 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1db5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db8104ff0 0x7f1db8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1db659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db80ffed0 0x7f1db8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1db5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db8104ff0 0x7f1db8199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37304/0 (socket says 172.21.15.45:37304) 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1db5d9b700 1 -- 172.21.15.45:0/2202016148 learned_addr learned my addr 172.21.15.45:0/2202016148 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1db6d9d700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db819daf0 0x7f1db819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1db659c700 1 -- 172.21.15.45:0/2202016148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db819daf0 msgr2=0x7f1db819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.213+0000 7f1db659c700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db819daf0 0x7f1db819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1db659c700 1 -- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db8104ff0 msgr2=0x7f1db8199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1db659c700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db8104ff0 0x7f1db8199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1db659c700 1 -- 172.21.15.45:0/2202016148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80095f0 con 0x7f1db80ffed0 2023-09-23T09:45:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1db5d9b700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db8104ff0 0x7f1db8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:45:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1db659c700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db80ffed0 0x7f1db8199150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000c00 tx=0x7f1da8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1da77fe700 1 -- 172.21.15.45:0/2202016148 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da801d070 con 0x7f1db80ffed0 2023-09-23T09:45:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db81a5da0 con 0x7f1db80ffed0 2023-09-23T09:45:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1da77fe700 1 -- 172.21.15.45:0/2202016148 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1da8021410 con 0x7f1db80ffed0 2023-09-23T09:45:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1da77fe700 1 -- 172.21.15.45:0/2202016148 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8017650 con 0x7f1db80ffed0 2023-09-23T09:45:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.214+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db81a61d0 con 0x7f1db80ffed0 2023-09-23T09:45:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.215+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db8067ba0 con 0x7f1db80ffed0 2023-09-23T09:45:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.216+0000 7f1da77fe700 1 -- 172.21.15.45:0/2202016148 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f1da8021580 con 0x7f1db80ffed0 2023-09-23T09:45:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.217+0000 7f1da77fe700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1da00a1710 0x7f1da00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.217+0000 7f1db5d9b700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1da00a1710 0x7f1da00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.217+0000 7f1da77fe700 1 -- 172.21.15.45:0/2202016148 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f1da80ce760 con 0x7f1db80ffed0 2023-09-23T09:45:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.217+0000 7f1db5d9b700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1da00a1710 0x7f1da00a3bd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1db819a760 tx=0x7f1dac009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.224+0000 7f1da77fe700 1 -- 172.21.15.45:0/2202016148 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1da80979d0 con 0x7f1db80ffed0 2023-09-23T09:45:11.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.427+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1db81007f0 con 0x7f1db80ffed0 2023-09-23T09:45:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.428+0000 7f1da77fe700 1 -- 172.21.15.45:0/2202016148 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1da809aff0 con 0x7f1db80ffed0 2023-09-23T09:45:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:45:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1da00a1710 msgr2=0x7f1da00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1da00a1710 0x7f1da00a3bd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1db819a760 tx=0x7f1dac009260 comp rx=0 tx=0).stop 2023-09-23T09:45:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db80ffed0 msgr2=0x7f1db8199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db80ffed0 0x7f1db8199150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000c00 tx=0x7f1da8005740 comp rx=0 tx=0).stop 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 shutdown_connections 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db819daf0 0x7f1db819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db80ffed0 0x7f1db8199150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f1da00a1710 0x7f1da00a3bd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 --2- 172.21.15.45:0/2202016148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db8104ff0 0x7f1db8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 >> 172.21.15.45:0/2202016148 conn(0x7f1db80fb770 msgr2=0x7f1db80687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 shutdown_connections 2023-09-23T09:45:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.430+0000 7f1dbcf62700 1 -- 172.21.15.45:0/2202016148 wait complete. 2023-09-23T09:45:11.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:45:11.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 -- 172.21.15.45:0/1764128194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 msgr2=0x7f954810cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 --2- 172.21.15.45:0/1764128194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f954810cb70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f954000b0a0 tx=0x7f954000b3b0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 -- 172.21.15.45:0/1764128194 shutdown_connections 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 --2- 172.21.15.45:0/1764128194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f954810cb70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 --2- 172.21.15.45:0/1764128194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480ffed0 0x7f95481002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 --2- 172.21.15.45:0/1764128194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9548100880 0x7f95481048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 -- 172.21.15.45:0/1764128194 >> 172.21.15.45:0/1764128194 conn(0x7f95480fb770 msgr2=0x7f95480fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 -- 172.21.15.45:0/1764128194 shutdown_connections 2023-09-23T09:45:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.570+0000 7f954e3ec700 1 -- 172.21.15.45:0/1764128194 wait complete. 2023-09-23T09:45:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 Processor -- start 2023-09-23T09:45:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 -- start start 2023-09-23T09:45:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480ffed0 0x7f9548195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f9548196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f954819a860 0x7f954819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954819b2e0 con 0x7f9548104ff0 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954819b450 con 0x7f954819a860 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f954e3ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954819b610 con 0x7f95480ffed0 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f95477fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f9548196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f9547fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480ffed0 0x7f9548195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.571+0000 7f95477fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f9548196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37318/0 (socket says 172.21.15.45:37318) 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f9547fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480ffed0 0x7f9548195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39422/0 (socket says 172.21.15.45:39422) 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95477fe700 1 -- 172.21.15.45:0/234190693 learned_addr learned my addr 172.21.15.45:0/234190693 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f954c989700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f954819a860 0x7f954819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95477fe700 1 -- 172.21.15.45:0/234190693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480ffed0 msgr2=0x7f9548195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95477fe700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480ffed0 0x7f9548195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95477fe700 1 -- 172.21.15.45:0/234190693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f954819a860 msgr2=0x7f954819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95477fe700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f954819a860 0x7f954819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95477fe700 1 -- 172.21.15.45:0/234190693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540009d00 con 0x7f9548104ff0 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95477fe700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f9548196470 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f953000e970 tx=0x7f953000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95457fa700 1 -- 172.21.15.45:0/234190693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530009790 con 0x7f9548104ff0 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95481a2b70 con 0x7f9548104ff0 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95457fa700 1 -- 172.21.15.45:0/234190693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f953000c7e0 con 0x7f9548104ff0 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f95457fa700 1 -- 172.21.15.45:0/234190693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95300054e0 con 0x7f9548104ff0 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.572+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481a30a0 con 0x7f9548104ff0 2023-09-23T09:45:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.573+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548197280 con 0x7f9548104ff0 2023-09-23T09:45:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.574+0000 7f95457fa700 1 -- 172.21.15.45:0/234190693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f953000c950 con 0x7f9548104ff0 2023-09-23T09:45:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.575+0000 7f95457fa700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95340a1710 0x7f95340a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.575+0000 7f9547fff700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95340a1710 0x7f95340a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.575+0000 7f95457fa700 1 -- 172.21.15.45:0/234190693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9530014070 con 0x7f9548104ff0 2023-09-23T09:45:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.575+0000 7f9547fff700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95340a1710 0x7f95340a3bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9538009c60 tx=0x7f9538009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.578+0000 7f95457fa700 1 -- 172.21.15.45:0/234190693 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9530096980 con 0x7f9548104ff0 2023-09-23T09:45:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.777+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9548100ad0 con 0x7f9548104ff0 2023-09-23T09:45:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.778+0000 7f95457fa700 1 -- 172.21.15.45:0/234190693 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9548100ad0 con 0x7f9548104ff0 2023-09-23T09:45:11.779 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:45:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95340a1710 msgr2=0x7f95340a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95340a1710 0x7f95340a3bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9538009c60 tx=0x7f9538009260 comp rx=0 tx=0).stop 2023-09-23T09:45:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 msgr2=0x7f9548196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f9548196470 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f953000e970 tx=0x7f953000ec80 comp rx=0 tx=0).stop 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 shutdown_connections 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f954819a860 0x7f954819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480ffed0 0x7f9548195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f95340a1710 0x7f95340a3bd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 --2- 172.21.15.45:0/234190693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9548104ff0 0x7f9548196470 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 >> 172.21.15.45:0/234190693 conn(0x7f95480fb770 msgr2=0x7f95480fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 shutdown_connections 2023-09-23T09:45:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.780+0000 7f954e3ec700 1 -- 172.21.15.45:0/234190693 wait complete. 2023-09-23T09:45:11.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 -- 172.21.15.45:0/3923865942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 msgr2=0x7f25681058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 --2- 172.21.15.45:0/3923865942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 0x7f25681058b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f255c009990 tx=0x7f255c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 -- 172.21.15.45:0/3923865942 shutdown_connections 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 --2- 172.21.15.45:0/3923865942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25680ffdf0 0x7f256810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 --2- 172.21.15.45:0/3923865942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 0x7f25681058b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 --2- 172.21.15.45:0/3923865942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 0x7f25680ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 -- 172.21.15.45:0/3923865942 >> 172.21.15.45:0/3923865942 conn(0x7f2568077290 msgr2=0x7f25680fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 -- 172.21.15.45:0/3923865942 shutdown_connections 2023-09-23T09:45:11.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.921+0000 7f2570435700 1 -- 172.21.15.45:0/3923865942 wait complete. 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.922+0000 7f2570435700 1 Processor -- start 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.922+0000 7f2570435700 1 -- start start 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f2570435700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 0x7f2568196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f2570435700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25680ffdf0 0x7f25681974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f2570435700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 0x7f256819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f2570435700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256819bf70 con 0x7f25681054d0 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f2570435700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256819c0e0 con 0x7f25680ffdf0 2023-09-23T09:45:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f2570435700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256819c2a0 con 0x7f25680ff430 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 0x7f2568196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 0x7f2568196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39436/0 (socket says 172.21.15.45:39436) 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 -- 172.21.15.45:0/93134258 learned_addr learned my addr 172.21.15.45:0/93134258 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256d9d0700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25680ffdf0 0x7f25681974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e9d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 0x7f256819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 -- 172.21.15.45:0/93134258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25680ffdf0 msgr2=0x7f25681974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25680ffdf0 0x7f25681974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 -- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 msgr2=0x7f256819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 0x7f256819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 -- 172.21.15.45:0/93134258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0095f0 con 0x7f25680ff430 2023-09-23T09:45:11.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f256e1d1700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 0x7f2568196f90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f255c005910 tx=0x7f255c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:11.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f255b7fe700 1 -- 172.21.15.45:0/93134258 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c01d070 con 0x7f25680ff430 2023-09-23T09:45:11.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256819c520 con 0x7f25680ff430 2023-09-23T09:45:11.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f255b7fe700 1 -- 172.21.15.45:0/93134258 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255c022470 con 0x7f25680ff430 2023-09-23T09:45:11.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.923+0000 7f255b7fe700 1 -- 172.21.15.45:0/93134258 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c017650 con 0x7f25680ff430 2023-09-23T09:45:11.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.924+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a3dc0 con 0x7f25680ff430 2023-09-23T09:45:11.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.924+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2568107a30 con 0x7f25680ff430 2023-09-23T09:45:11.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.926+0000 7f255b7fe700 1 -- 172.21.15.45:0/93134258 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f255c0225e0 con 0x7f25680ff430 2023-09-23T09:45:11.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.926+0000 7f255b7fe700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f25540a1710 0x7f25540a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:11.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.927+0000 7f256d9d0700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f25540a1710 0x7f25540a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:11.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.927+0000 7f255b7fe700 1 -- 172.21.15.45:0/93134258 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f255c0ce830 con 0x7f25680ff430 2023-09-23T09:45:11.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.927+0000 7f256d9d0700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f25540a1710 0x7f25540a3bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f25681985a0 tx=0x7f2564009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:11.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:11.932+0000 7f255b7fe700 1 -- 172.21.15.45:0/93134258 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f255c097b00 con 0x7f25680ff430 2023-09-23T09:45:12.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:11 smithi045 ceph-mon[98627]: pgmap v495: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 105 KiB/s wr, 36 op/s 2023-09-23T09:45:12.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:11 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2642802727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:45:12.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:11 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2202016148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:45:12.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:11 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/234190693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:45:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.136+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2568198380 con 0x7f25680ff430 2023-09-23T09:45:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.136+0000 7f255b7fe700 1 -- 172.21.15.45:0/93134258 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f255c09b120 con 0x7f25680ff430 2023-09-23T09:45:12.138 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f25540a1710 msgr2=0x7f25540a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f25540a1710 0x7f25540a3bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f25681985a0 tx=0x7f2564009260 comp rx=0 tx=0).stop 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 msgr2=0x7f2568196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 0x7f2568196f90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f255c005910 tx=0x7f255c005720 comp rx=0 tx=0).stop 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 shutdown_connections 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f25540a1710 0x7f25540a3bd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25681054d0 0x7f256819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25680ff430 0x7f2568196f90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 --2- 172.21.15.45:0/93134258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25680ffdf0 0x7f25681974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.138+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 >> 172.21.15.45:0/93134258 conn(0x7f2568077290 msgr2=0x7f2568103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.139+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 shutdown_connections 2023-09-23T09:45:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.139+0000 7f2570435700 1 -- 172.21.15.45:0/93134258 wait complete. 2023-09-23T09:45:12.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-23T09:45:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 -- 172.21.15.45:0/734658208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 msgr2=0x7faf901061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 --2- 172.21.15.45:0/734658208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 0x7faf901061b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009990 tx=0x7faf8c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 -- 172.21.15.45:0/734658208 shutdown_connections 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 --2- 172.21.15.45:0/734658208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf90100870 0x7faf9010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 --2- 172.21.15.45:0/734658208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 0x7faf901061b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 --2- 172.21.15.45:0/734658208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf900ffeb0 0x7faf90100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 -- 172.21.15.45:0/734658208 >> 172.21.15.45:0/734658208 conn(0x7faf90077ca0 msgr2=0x7faf900780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 -- 172.21.15.45:0/734658208 shutdown_connections 2023-09-23T09:45:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.277+0000 7faf97775700 1 -- 172.21.15.45:0/734658208 wait complete. 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 Processor -- start 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 -- start start 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf900ffeb0 0x7faf90101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf90100870 0x7faf90101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 0x7faf901a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901026f0 con 0x7faf90105dd0 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a55b0 con 0x7faf900ffeb0 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf97775700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a56f0 con 0x7faf90100870 2023-09-23T09:45:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf95511700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf900ffeb0 0x7faf90101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf95d12700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 0x7faf901a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf94d10700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf90100870 0x7faf90101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf95511700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf900ffeb0 0x7faf90101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58694/0 (socket says 172.21.15.45:58694) 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.278+0000 7faf95511700 1 -- 172.21.15.45:0/388778576 learned_addr learned my addr 172.21.15.45:0/388778576 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf95d12700 1 -- 172.21.15.45:0/388778576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf90100870 msgr2=0x7faf90101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf95d12700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf90100870 0x7faf90101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf95d12700 1 -- 172.21.15.45:0/388778576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf900ffeb0 msgr2=0x7faf90101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf95d12700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf900ffeb0 0x7faf90101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf95d12700 1 -- 172.21.15.45:0/388778576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0095f0 con 0x7faf90105dd0 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf95d12700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 0x7faf901a5070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7faf88007f10 tx=0x7faf8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf867fc700 1 -- 172.21.15.45:0/388778576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88009ea0 con 0x7faf90105dd0 2023-09-23T09:45:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf867fc700 1 -- 172.21.15.45:0/388778576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf8800f040 con 0x7faf90105dd0 2023-09-23T09:45:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf90196c50 con 0x7faf90105dd0 2023-09-23T09:45:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf867fc700 1 -- 172.21.15.45:0/388778576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88004350 con 0x7faf90105dd0 2023-09-23T09:45:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.279+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901970e0 con 0x7faf90105dd0 2023-09-23T09:45:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.280+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf90068060 con 0x7faf90105dd0 2023-09-23T09:45:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.281+0000 7faf867fc700 1 -- 172.21.15.45:0/388778576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7faf8800db20 con 0x7faf90105dd0 2023-09-23T09:45:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.282+0000 7faf867fc700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7faf7c0a1750 0x7faf7c0a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:45:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.282+0000 7faf95511700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7faf7c0a1750 0x7faf7c0a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:45:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.282+0000 7faf867fc700 1 -- 172.21.15.45:0/388778576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8629+0+0 (secure 0 0 0) 0x7faf880cd6a0 con 0x7faf90105dd0 2023-09-23T09:45:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.282+0000 7faf95511700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7faf7c0a1750 0x7faf7c0a3c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00b2e0 tx=0x7faf8c005fd0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:45:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.285+0000 7faf867fc700 1 -- 172.21.15.45:0/388778576 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7faf88096880 con 0x7faf90105dd0 2023-09-23T09:45:12.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:11 smithi098 ceph-mon[102386]: pgmap v495: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 105 KiB/s wr, 36 op/s 2023-09-23T09:45:12.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:11 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2642802727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:45:12.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:11 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2202016148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:45:12.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:11 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/234190693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:45:12.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:11 smithi123 ceph-mon[103577]: pgmap v495: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 105 KiB/s wr, 36 op/s 2023-09-23T09:45:12.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:11 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2642802727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:45:12.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:11 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2202016148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:45:12.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:11 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/234190693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:45:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.550+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7faf9010fcd0 con 0x7faf90105dd0 2023-09-23T09:45:12.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.558+0000 7faf867fc700 1 -- 172.21.15.45:0/388778576 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+734592 (secure 0 0 0) 0x7faf88099ea0 con 0x7faf90105dd0 2023-09-23T09:45:12.567 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:45:12.567 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:45:12.567 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 25, 2023-09-23T09:45:12.567 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 15405, 2023-09-23T09:45:12.567 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2023-09-23T09:45:12.567 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2023-09-23T09:45:12.567 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 441481757 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 441481757 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.568 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/441481757", 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2023-09-23T09:29:25.699323+0000", 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540138322906710015, 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 15399, 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540138322906710015, 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2023-09-23T09:45:12.569 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.572 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.574 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2023-09-23T09:45:12.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2023-09-23T09:45:12.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.580 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2023-09-23T09:45:12.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2023-09-23T09:45:12.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2023-09-23T09:45:12.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.587 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.588 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-23T09:45:12.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2023-09-23T09:45:12.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.595 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.597 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-23T09:45:12.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2023-09-23T09:45:12.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2023-09-23T09:45:12.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-09-23T09:45:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2023-09-23T09:45:12.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2023-09-23T09:45:12.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.614 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-23T09:45:12.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-23T09:45:12.616 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-23T09:45:12.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-23T09:45:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2023-09-23T09:45:12.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2023-09-23T09:45:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2023-09-23T09:45:12.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.629 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2023-09-23T09:45:12.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.634 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2023-09-23T09:45:12.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2023-09-23T09:45:12.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2023-09-23T09:45:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2023-09-23T09:45:12.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2023-09-23T09:45:12.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.645 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2023-09-23T09:45:12.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.657 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-23T09:45:12.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2023-09-23T09:45:12.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2023-09-23T09:45:12.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2023-09-23T09:45:12.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.675 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.680 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.686 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-23T09:45:12.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-23T09:45:12.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.695 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-23T09:45:12.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-23T09:45:12.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.710 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.714 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.717 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.718 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-23T09:45:12.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-23T09:45:12.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2023-09-23T09:45:12.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.725 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.726 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2023-09-23T09:45:12.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.735 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.737 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2023-09-23T09:45:12.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2023-09-23T09:45:12.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:12.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.746 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.754 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.756 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-23T09:45:12.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.759 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.768 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2023-09-23T09:45:12.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.771 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.772 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.773 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.779 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2023-09-23T09:45:12.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2023-09-23T09:45:12.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.798 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.803 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.806 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.809 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2023-09-23T09:45:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2023-09-23T09:45:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.847 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-09-23T09:45:12.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2023-09-23T09:45:12.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.852 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.858 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.859 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.864 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "noautoscale": { 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "noautoscale", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "global autoscale flag", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-23T09:45:12.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.872 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:12.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2023-09-23T09:45:12.882 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-23T09:45:12.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.886 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.888 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2023-09-23T09:45:12.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-09-23T09:45:12.899 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.900 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.902 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.903 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2023-09-23T09:45:12.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.914 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.919 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.920 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.925 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.927 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.930 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.934 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.937 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2023-09-23T09:45:12.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2023-09-23T09:45:12.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2023-09-23T09:45:12.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.944 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-09-23T09:45:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2023-09-23T09:45:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2023-09-23T09:45:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2023-09-23T09:45:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2023-09-23T09:45:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2023-09-23T09:45:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 25094, 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540138322906710015, 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2023-09-23T09:45:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2023-09-23T09:45:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-23T09:45:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2023-09-23T09:45:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2023-09-23T09:45:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2023-09-23T09:45:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2023-09-23T09:45:13.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.081 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2023-09-23T09:45:13.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-23T09:45:13.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2023-09-23T09:45:13.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2023-09-23T09:45:13.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-23T09:45:13.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2023-09-23T09:45:13.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.097 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-23T09:45:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2023-09-23T09:45:13.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.102 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2023-09-23T09:45:13.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2023-09-23T09:45:13.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2023-09-23T09:45:13.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.113 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2023-09-23T09:45:13.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.118 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-23T09:45:13.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2023-09-23T09:45:13.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.130 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2023-09-23T09:45:13.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.134 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2023-09-23T09:45:13.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.139 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-23T09:45:13.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2023-09-23T09:45:13.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-23T09:45:13.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2023-09-23T09:45:13.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-23T09:45:13.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.169 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-23T09:45:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.176 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-23T09:45:13.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-23T09:45:13.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.188 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.190 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.191 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-23T09:45:13.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.194 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.196 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.200 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-23T09:45:13.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-23T09:45:13.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.218 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.222 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.226 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.230 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.232 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2023-09-23T09:45:13.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.235 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.237 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-09-23T09:45:13.240 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.246 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.250 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2023-09-23T09:45:13.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-23T09:45:13.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.255 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2023-09-23T09:45:13.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.263 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.268 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.270 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-23T09:45:13.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.277 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.280 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.281 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.284 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.287 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-23T09:45:13.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2023-09-23T09:45:13.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.293 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.304 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.307 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-23T09:45:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2023-09-23T09:45:13.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.311 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.314 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.315 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.317 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2023-09-23T09:45:13.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.323 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2023-09-23T09:45:13.329 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.330 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2023-09-23T09:45:13.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.334 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.342 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.355 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.358 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.361 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.364 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.367 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.370 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.377 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.380 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.381 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "noautoscale": { 2023-09-23T09:45:13.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "noautoscale", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "global autoscale flag", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-09-23T09:45:13.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.395 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.399 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-23T09:45:13.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.412 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2023-09-23T09:45:13.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-09-23T09:45:13.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.426 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.429 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2023-09-23T09:45:13.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2023-09-23T09:45:13.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.445 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.446 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.449 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.451 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.453 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.456 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.460 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.463 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2023-09-23T09:45:13.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2023-09-23T09:45:13.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2023-09-23T09:45:13.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.477 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2023-09-23T09:45:13.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.482 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2023-09-23T09:45:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2023-09-23T09:45:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2023-09-23T09:45:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "restful", 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_schedule" 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2023-09-23T09:45:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2023-09-23T09:45:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2023-09-23T09:45:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2023-09-23T09:45:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-23T09:45:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T09:45:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.588 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2023-09-23T09:45:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.591 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2023-09-23T09:45:13.597 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-23T09:45:13.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-23T09:45:13.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2023-09-23T09:45:13.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2023-09-23T09:45:13.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2023-09-23T09:45:13.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.622 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2023-09-23T09:45:13.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2023-09-23T09:45:13.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.633 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.636 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2023-09-23T09:45:13.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.640 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-23T09:45:13.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-23T09:45:13.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2023-09-23T09:45:13.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2023-09-23T09:45:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2023-09-23T09:45:13.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.651 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2023-09-23T09:45:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2023-09-23T09:45:13.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2023-09-23T09:45:13.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.662 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.668 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-23T09:45:13.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.672 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-23T09:45:13.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.687 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-23T09:45:13.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-23T09:45:13.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.700 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-23T09:45:13.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-23T09:45:13.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-23T09:45:13.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.711 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.717 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2023-09-23T09:45:13.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-09-23T09:45:13.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.725 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2023-09-23T09:45:13.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-09-23T09:45:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2023-09-23T09:45:13.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.742 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-23T09:45:13.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-09-23T09:45:13.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.756 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.758 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2023-09-23T09:45:13.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2023-09-23T09:45:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.762 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2023-09-23T09:45:13.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.769 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2023-09-23T09:45:13.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-09-23T09:45:13.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.782 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2023-09-23T09:45:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-23T09:45:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2023-09-23T09:45:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2023-09-23T09:45:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2023-09-23T09:45:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2023-09-23T09:45:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2023-09-23T09:45:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "noautoscale": { 2023-09-23T09:45:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "noautoscale", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "global autoscale flag", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-23T09:45:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-09-23T09:45:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.867 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2023-09-23T09:45:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-09-23T09:45:13.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2023-09-23T09:45:13.874 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.878 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.879 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.880 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2023-09-23T09:45:13.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.883 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-09-23T09:45:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.892 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2023-09-23T09:45:13.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.926 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2023-09-23T09:45:13.932 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-09-23T09:45:13.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2023-09-23T09:45:13.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2023-09-23T09:45:13.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2023-09-23T09:45:13.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.939 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2023-09-23T09:45:13.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.947 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.953 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2023-09-23T09:45:13.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.968 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.971 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2023-09-23T09:45:13.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2023-09-23T09:45:13.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.983 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2023-09-23T09:45:13.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:13.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.992 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:13.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:13.995 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:13.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2023-09-23T09:45:13.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2023-09-23T09:45:14.002 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:14.007 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:14.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:14.014 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:14.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2023-09-23T09:45:14.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2023-09-23T09:45:14.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-09-23T09:45:14.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:14.026 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:14.027 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-09-23T09:45:14.029 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-09-23T09:45:14.032 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-09-23T09:45:14.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-09-23T09:45:14.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.036 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-09-23T09:45:14.037 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-09-23T09:45:14.038 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-09-23T09:45:14.039 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-09-23T09:45:14.040 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 86, 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3823656313 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2023-09-23T09:45:14.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2303082840 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-09-23T09:45:14.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 76686505 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-09-23T09:45:14.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1646072753 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-09-23T09:45:14.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2451686359 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-09-23T09:45:14.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 33663376 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-09-23T09:45:14.047 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:45:14.049 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/93134258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:45:14.049 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/388778576' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-23T09:45:14.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/93134258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:45:14.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/388778576' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7faf7c0a1750 msgr2=0x7faf7c0a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7faf7c0a1750 0x7faf7c0a3c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00b2e0 tx=0x7faf8c005fd0 comp rx=0 tx=0).stop 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 msgr2=0x7faf901a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 0x7faf901a5070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7faf88007f10 tx=0x7faf8800d3b0 comp rx=0 tx=0).stop 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 shutdown_connections 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7faf7c0a1750 0x7faf7c0a3c10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf90105dd0 0x7faf901a5070 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf900ffeb0 0x7faf90101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 --2- 172.21.15.45:0/388778576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf90100870 0x7faf90101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:45:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 >> 172.21.15.45:0/388778576 conn(0x7faf90077ca0 msgr2=0x7faf90069240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:45:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 shutdown_connections 2023-09-23T09:45:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:45:12.564+0000 7faf97775700 1 -- 172.21.15.45:0/388778576 wait complete. 2023-09-23T09:45:14.058 DEBUG:tasks.check_counter:Getting stats from a 2023-09-23T09:45:14.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-09-23T09:45:14.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/93134258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:45:14.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/388778576' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-23T09:45:14.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:13 smithi098 ceph-mon[102386]: pgmap v496: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 34 op/s 2023-09-23T09:45:14.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:13 smithi045 ceph-mon[98627]: pgmap v496: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 34 op/s 2023-09-23T09:45:14.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:13 smithi123 ceph-mon[103577]: pgmap v496: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 34 op/s 2023-09-23T09:45:14.637 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:45:16.063 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:45:16.063 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:16.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 291, 2023-09-23T09:45:16.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 281, 2023-09-23T09:45:16.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1446224, 2023-09-23T09:45:16.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 169248, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.050330755, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.015680203, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.033999713, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.007448976, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 280, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.082155029, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000293410 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1446224, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 169248 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 7, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 218, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3424, 2023-09-23T09:45:16.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 268192, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015394554, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010314191, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002622260, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 216, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.070334643, 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000325623 2023-09-23T09:45:16.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 268192 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2023-09-23T09:45:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000403211, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000330959, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:16.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-09-23T09:45:16.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:16.071 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:16.072 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 147977, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 29, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:16.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-09-23T09:45:16.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 294, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1405157, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 294, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 294, 2023-09-23T09:45:16.075 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1405157, 2023-09-23T09:45:16.076 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-09-23T09:45:16.076 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:16.076 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:16.076 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:16.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:16.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:16.076 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:45:16.240 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:15 smithi045 ceph-mon[98627]: pgmap v497: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 96 KiB/s wr, 34 op/s 2023-09-23T09:45:16.240 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:15 smithi045 ceph-mon[98627]: osdmap e107: 12 total, 12 up, 12 in 2023-09-23T09:45:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:15 smithi098 ceph-mon[102386]: pgmap v497: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 96 KiB/s wr, 34 op/s 2023-09-23T09:45:16.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:15 smithi098 ceph-mon[102386]: osdmap e107: 12 total, 12 up, 12 in 2023-09-23T09:45:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:15 smithi123 ceph-mon[103577]: pgmap v497: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 96 KiB/s wr, 34 op/s 2023-09-23T09:45:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:15 smithi123 ceph-mon[103577]: osdmap e107: 12 total, 12 up, 12 in 2023-09-23T09:45:16.506 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-09-23T09:45:16.506 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-09-23T09:45:16.506 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-09-23T09:45:16.506 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-09-23T09:45:16.506 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-09-23T09:45:16.506 DEBUG:tasks.check_counter:Getting stats from d 2023-09-23T09:45:16.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-09-23T09:45:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:45:18.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:17 smithi045 ceph-mon[98627]: pgmap v499: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:18.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:17 smithi098 ceph-mon[102386]: pgmap v499: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:18.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:17 smithi123 ceph-mon[103577]: pgmap v499: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:18.498 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:45:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 296, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 286, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1447584, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 172368, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.051917176, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.016451828, 2023-09-23T09:45:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034302857, 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.007855402, 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 285, 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.092518724, 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000324627 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:18.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1447584, 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 172368 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 7, 2023-09-23T09:45:18.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 222, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3424, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 268896, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015422578, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010555201, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002385744, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:18.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 220, 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.056061406, 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000254824 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:18.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 268896 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2023-09-23T09:45:18.504 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000360513, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000266419, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:18.505 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:45:18.506 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:45:18.507 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:18.508 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:18.509 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:18.510 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 147920, 2023-09-23T09:45:18.511 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 27, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:18.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:18.513 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 299, 2023-09-23T09:45:18.514 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1405807, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 299, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 299, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1405807, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:18.515 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:18.516 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:18.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:18.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:18.516 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:45:18.840 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-09-23T09:45:18.841 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-09-23T09:45:18.841 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-09-23T09:45:18.841 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-09-23T09:45:18.841 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-09-23T09:45:18.841 DEBUG:tasks.check_counter:Getting stats from g 2023-09-23T09:45:18.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-09-23T09:45:19.415 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:45:20.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:19 smithi098 ceph-mon[102386]: pgmap v500: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:20.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:19 smithi045 ceph-mon[98627]: pgmap v500: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:20.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:19 smithi123 ceph-mon[103577]: pgmap v500: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:20.651 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2944, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.002093931, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000042712, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000805011, 2023-09-23T09:45:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000763881, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000254627 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2023-09-23T09:45:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 217, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 480, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 267920, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015382675, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010471861, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002411824, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 214, 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.102814789, 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000480442 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 480, 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 267920 2023-09-23T09:45:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 293, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 283, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1446768, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 170496, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.049653709, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.015990386, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034305074, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.007492401, 2023-09-23T09:45:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 282, 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.107764610, 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000382144 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1446768, 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 170496 2023-09-23T09:45:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 147987, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 296, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1405417, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 296, 2023-09-23T09:45:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 296, 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1405417, 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:45:21.072 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-09-23T09:45:21.072 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-09-23T09:45:21.072 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-09-23T09:45:21.072 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-09-23T09:45:21.072 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-09-23T09:45:21.072 DEBUG:tasks.check_counter:Getting stats from j 2023-09-23T09:45:21.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-09-23T09:45:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T09:45:22.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:21 smithi045 ceph-mon[98627]: pgmap v501: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-23T09:45:22.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:21 smithi098 ceph-mon[102386]: pgmap v501: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-23T09:45:22.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:21 smithi123 ceph-mon[103577]: pgmap v501: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-23T09:45:22.887 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-09-23T09:45:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 297, 2023-09-23T09:45:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 287, 2023-09-23T09:45:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1447856, 2023-09-23T09:45:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 172992, 2023-09-23T09:45:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.050791825, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.015905311, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034639841, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.007342740, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 286, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.073364005, 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000256517 2023-09-23T09:45:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1447856, 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 172992 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 7, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 223, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3424, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 269072, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015838263, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010690662, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002702228, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 221, 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.025847461, 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000116956 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 269072 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000551185, 2023-09-23T09:45:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000470037, 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 147920, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 27, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-09-23T09:45:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-09-23T09:45:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-09-23T09:45:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 300, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1405937, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 300, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 300, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1405937, 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-09-23T09:45:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-09-23T09:45:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-09-23T09:45:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-09-23T09:45:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-09-23T09:45:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-09-23T09:45:23.301 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-09-23T09:45:23.301 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-09-23T09:45:23.301 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-09-23T09:45:23.302 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-09-23T09:45:23.302 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-09-23T09:45:23.302 DEBUG:tasks.check_counter:Getting stats from b 2023-09-23T09:45:23.302 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-09-23T09:45:24.010 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:45:24.269 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:23 smithi098 ceph-mon[102386]: pgmap v502: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-23T09:45:24.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:23 smithi045 ceph-mon[98627]: pgmap v502: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-23T09:45:24.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:23 smithi123 ceph-mon[103577]: pgmap v502: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-23T09:45:26.150 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-09-23T09:45:26.150 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:26.150 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 108169, 2023-09-23T09:45:26.150 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 109075, 2023-09-23T09:45:26.150 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 20743425, 2023-09-23T09:45:26.150 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 37109824, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 214, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 4.380891244, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 1.854862343, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 2.960604049, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.100856814, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 109076, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3.643095930, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000033399 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:26.151 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 209648, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.020644158, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000098 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 43120, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 2880 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 3740, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 3797, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1314168, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 24641132, 2023-09-23T09:45:26.152 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 13, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 7, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.344568592, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.091649167, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.053722334, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.185759695, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3794, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2.067759482, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000545007 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 291, 2023-09-23T09:45:26.153 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000043315, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000148 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 100736, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 584960 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 3789157, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 3792074, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 604865011, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 637836771, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 25, 2023-09-23T09:45:26.154 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 7, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 88.722657719, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 35.426466621, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 62.805318915, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.199940501, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3792071, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 93.495589460, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000024655 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 7007688, 2023-09-23T09:45:26.155 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.592717149, 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000227 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 5321056, 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 7736960 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-09-23T09:45:26.156 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 7472, 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 4.446635423, 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000595106 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 397, 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.017072990, 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000043005 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.157 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 5515, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 5043, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5043, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 84.387369674, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.016733565 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 472, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 23, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 2, 2023-09-23T09:45:26.158 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 1044, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 5, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 315, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 128, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 171, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 16, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 125, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 3146, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 99, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 99, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 15, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 18058, 2023-09-23T09:45:26.159 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 13778, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 446, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 124, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 2, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 2159, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 1645, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 86, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 2739, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 37, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 1691, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2023-09-23T09:45:26.160 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 804, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 5, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1016, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 1352210931, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 6, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 804, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 69, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 3, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 66, 2023-09-23T09:45:26.161 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 735, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 96, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 1303, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 2950, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 5, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 3113, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 1628, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 120, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 981 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.162 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 159, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 202, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 3, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 302, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 90, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 855, 2023-09-23T09:45:26.163 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 38, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 3, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 86, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 5, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2023-09-23T09:45:26.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 5816, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 5747, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 5747, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 69, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 105, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 80, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 79, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 79, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 1, 2023-09-23T09:45:26.165 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 1, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 1, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 40362470, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 40979328, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3391, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 31.430529210, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.009268808 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.166 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 252, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 2846, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 2594, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 65, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 159, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 94, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 317, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 4331, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 4014, 2023-09-23T09:45:26.167 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 99, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 1485, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 1386, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 73440, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 207152 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 12600, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 2376, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 5515, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 844, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 3699, 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2023-09-23T09:45:26.168 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1130, 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 6.714356114, 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005941908 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 561, 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 6.363137314, 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.011342490 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.169 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 216, 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.093641625, 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005063155 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1147, 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 14.909945234, 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.012999080 2023-09-23T09:45:26.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000130835, 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000130835 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.171 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2023-09-23T09:45:26.172 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 16, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.003048609, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000190538 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 35, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.338675762, 2023-09-23T09:45:26.173 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.009676450 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 16, 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.201558394, 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.012597399 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.174 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 602, 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.347350504, 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000576994 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 101, 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.312953975, 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003098554 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 964, 2023-09-23T09:45:26.175 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 52.957974483, 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.054935658 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 10, 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.285558259, 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.028555825 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2023-09-23T09:45:26.177 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002777602, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002777602 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.178 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2023-09-23T09:45:26.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.017467711, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002911285 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 237, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.838793253, 2023-09-23T09:45:26.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003539212 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 5, 2023-09-23T09:45:26.181 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 3, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 2, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 163, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 81, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 949, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 360, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 360, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:26.182 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:26.183 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 460522, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 445, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 88, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 1, 2023-09-23T09:45:26.185 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8352, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 190, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:26.186 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 1135858, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 24606, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 7963, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 42476431, 2023-09-23T09:45:26.187 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 37, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 7963, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 7963, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 139.477297736, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.017515672 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 7963, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 12876 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.188 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 7963, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 887, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 7076, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 957, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 1357, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 55, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 3443, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 335, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2023-09-23T09:45:26.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 4, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 634, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 817, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 2714, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2023-09-23T09:45:26.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2023-09-23T09:45:26.191 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 107, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 25, 2023-09-23T09:45:26.192 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 2260, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 50, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 250 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.193 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 19, 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 10, 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 90, 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:26.194 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 3901062, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 275302515, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 3901062, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3901062, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 275302515, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.195 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:26.196 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 7963, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 42485337, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 7963, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 42485337, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2023-09-23T09:45:26.197 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 7963, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 7963, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 7963, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 7963, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 90, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 9282, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 90, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:26.199 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 52, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 9282, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x56439435e6e0": { 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 5816, 2023-09-23T09:45:26.200 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 36785024, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 5816, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3391, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 36785024, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:26.201 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-09-23T09:45:26.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:25 smithi098 ceph-mon[102386]: pgmap v503: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-23T09:45:26.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:25 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:26.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:25 smithi045 ceph-mon[98627]: pgmap v503: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-23T09:45:26.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:25 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:26.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:25 smithi123 ceph-mon[103577]: pgmap v503: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-23T09:45:26.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:25 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:28.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:27 smithi098 ceph-mon[102386]: pgmap v504: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-23T09:45:28.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:27 smithi045 ceph-mon[98627]: pgmap v504: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-23T09:45:28.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:27 smithi123 ceph-mon[103577]: pgmap v504: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-23T09:45:29.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:28 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:29.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:28 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:29.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:28 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:29.522 INFO:tasks.check_counter:Daemon mds.b mds.exported=86 2023-09-23T09:45:29.523 INFO:tasks.check_counter:Daemon mds.b mds.imported=37 2023-09-23T09:45:29.523 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-09-23T09:45:29.523 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2023-09-23T09:45:29.523 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2023-09-23T09:45:29.523 DEBUG:tasks.check_counter:Getting stats from e 2023-09-23T09:45:29.523 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-09-23T09:45:30.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:29 smithi098 ceph-mon[102386]: pgmap v505: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-23T09:45:30.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:29 smithi045 ceph-mon[98627]: pgmap v505: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-23T09:45:30.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:29 smithi123 ceph-mon[103577]: pgmap v505: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-23T09:45:32.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:31 smithi098 ceph-mon[102386]: pgmap v506: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:32.298 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:45:32.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:31 smithi045 ceph-mon[98627]: pgmap v506: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:32.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:31 smithi123 ceph-mon[103577]: pgmap v506: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-23T09:45:34.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:33 smithi098 ceph-mon[102386]: pgmap v507: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-23T09:45:34.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:33 smithi098 ceph-mon[102386]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:34.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:33 smithi045 ceph-mon[98627]: pgmap v507: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-23T09:45:34.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:33 smithi045 ceph-mon[98627]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:34.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:33 smithi123 ceph-mon[103577]: pgmap v507: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-23T09:45:34.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:33 smithi123 ceph-mon[103577]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-23T09:45:36.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:35 smithi098 ceph-mon[102386]: pgmap v508: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 8 op/s 2023-09-23T09:45:36.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:35 smithi045 ceph-mon[98627]: pgmap v508: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 8 op/s 2023-09-23T09:45:36.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:35 smithi123 ceph-mon[103577]: pgmap v508: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 8 op/s 2023-09-23T09:45:38.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:37 smithi098 ceph-mon[102386]: pgmap v509: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-23T09:45:38.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:37 smithi045 ceph-mon[98627]: pgmap v509: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-23T09:45:38.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:37 smithi123 ceph-mon[103577]: pgmap v509: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-23T09:45:40.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:39 smithi098 ceph-mon[102386]: pgmap v510: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-23T09:45:40.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:39 smithi045 ceph-mon[98627]: pgmap v510: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-23T09:45:40.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:39 smithi123 ceph-mon[103577]: pgmap v510: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-23T09:45:41.349 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-09-23T09:45:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 415, 2023-09-23T09:45:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 364, 2023-09-23T09:45:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1657756, 2023-09-23T09:45:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 193934, 2023-09-23T09:45:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.054604689, 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.015700278, 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.037952800, 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.008816473, 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 364, 2023-09-23T09:45:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.054391461, 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000149427 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1647328, 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 181488 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1109, 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1329, 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 4912170, 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 888538, 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2023-09-23T09:45:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.094867956, 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.026656131, 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.025409695, 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.034295932, 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1326, 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.215107687, 2023-09-23T09:45:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000162222 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-23T09:45:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 544256 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2278, 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2280, 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 9055331, 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 708516, 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 5, 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2023-09-23T09:45:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.188148071, 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.053618315, 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.049276045, 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.070473779, 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2278, 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.404934392, 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000177758 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5813, 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.147669741, 2023-09-23T09:45:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000025403 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2023-09-23T09:45:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 1931, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 320, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 1470, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 141, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 1473, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 1, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2023-09-23T09:45:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 51, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2023-09-23T09:45:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2023-09-23T09:45:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T09:45:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2023-09-23T09:45:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2023-09-23T09:45:41.368 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2023-09-23T09:45:41.369 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2023-09-23T09:45:41.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 5434, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2023-09-23T09:45:41.371 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 8, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 17215824, 2023-09-23T09:45:41.372 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 17209061, 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 5434 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2023-09-23T09:45:41.373 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 1626, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 2078, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 452, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 65, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 85, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 20, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 1931, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 2784, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 853, 2023-09-23T09:45:41.374 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 56296, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 207152 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2023-09-23T09:45:41.375 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.376 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2023-09-23T09:45:41.377 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.378 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.379 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.382 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.383 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.384 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.385 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2023-09-23T09:45:41.386 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2023-09-23T09:45:41.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2023-09-23T09:45:41.388 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2023-09-23T09:45:41.389 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.390 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.391 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 4, 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 2, 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 3, 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2023-09-23T09:45:41.392 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 2959, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:41.393 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:41.394 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:41.395 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:41.396 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 270317, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 1771, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:41.397 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8352, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 190, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 5199065, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 55366, 2023-09-23T09:45:41.398 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 3425, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 21, 2023-09-23T09:45:41.399 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 3, 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 3425, 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3425, 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 12.773850793, 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003729591 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3425, 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3426 2023-09-23T09:45:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 3425, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 3425, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 2017, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 1406, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2023-09-23T09:45:41.401 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 1, 2023-09-23T09:45:41.402 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2023-09-23T09:45:41.403 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2023-09-23T09:45:41.404 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 107, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 24, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 7, 2023-09-23T09:45:41.405 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 8, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 1, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2023-09-23T09:45:41.406 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2023-09-23T09:45:41.407 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 3798, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 15261669, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:41.408 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 3798, 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3798, 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 15261669, 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3425, 2023-09-23T09:45:41.410 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 13021959, 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3425, 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 13021959, 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:41.411 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3425, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3425, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3425, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3425, 2023-09-23T09:45:41.412 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:41.413 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.414 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55b8e260e960": { 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:41.415 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:41.416 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-09-23T09:45:42.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:41 smithi098 ceph-mon[102386]: pgmap v511: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-23T09:45:42.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:41 smithi045 ceph-mon[98627]: pgmap v511: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-23T09:45:42.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:41 smithi123 ceph-mon[103577]: pgmap v511: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-23T09:45:43.302 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-09-23T09:45:43.303 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-09-23T09:45:43.303 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-09-23T09:45:43.303 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-09-23T09:45:43.303 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T09:45:43.303 DEBUG:tasks.check_counter:Getting stats from h 2023-09-23T09:45:43.304 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-09-23T09:45:44.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:43 smithi098 ceph-mon[102386]: pgmap v512: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-09-23T09:45:44.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:43 smithi045 ceph-mon[98627]: pgmap v512: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-09-23T09:45:44.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:43 smithi123 ceph-mon[103577]: pgmap v512: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-09-23T09:45:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:45:46.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:45 smithi098 ceph-mon[102386]: pgmap v513: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2023-09-23T09:45:46.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:45 smithi045 ceph-mon[98627]: pgmap v513: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2023-09-23T09:45:46.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:45 smithi123 ceph-mon[103577]: pgmap v513: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2023-09-23T09:45:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:47 smithi098 ceph-mon[102386]: pgmap v514: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-23T09:45:48.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:47 smithi045 ceph-mon[98627]: pgmap v514: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-23T09:45:48.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:47 smithi123 ceph-mon[103577]: pgmap v514: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-23T09:45:48.593 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-09-23T09:45:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 7772021, 2023-09-23T09:45:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 7771806, 2023-09-23T09:45:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1236342841, 2023-09-23T09:45:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1297622026, 2023-09-23T09:45:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 9, 2023-09-23T09:45:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2023-09-23T09:45:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 174.979869727, 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 70.499434815, 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 123.012541003, 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.069500727, 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 7771807, 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 161.653072508, 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000020799 2023-09-23T09:45:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 14334215, 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3.152965414, 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000219 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1656336, 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 190240 2023-09-23T09:45:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 344, 2023-09-23T09:45:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 593, 2023-09-23T09:45:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 217837, 2023-09-23T09:45:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 833770, 2023-09-23T09:45:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 16, 2023-09-23T09:45:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2023-09-23T09:45:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.034042430, 2023-09-23T09:45:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.013608222, 2023-09-23T09:45:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.010938590, 2023-09-23T09:45:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.006443566, 2023-09-23T09:45:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 590, 2023-09-23T09:45:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.091084482, 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000154380 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 72, 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000017967, 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000249 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 5264, 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 653728 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 7116, 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 9786, 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 2363940, 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 7906064, 2023-09-23T09:45:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 15, 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 7, 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.463579488, 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.207187326, 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.293234877, 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.022982921, 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 9783, 2023-09-23T09:45:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.986665279, 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000100855 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 12365, 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001944767, 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000157 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 2028544, 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 4724416 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2023-09-23T09:45:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5136, 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2.173470993, 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000423183 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2023-09-23T09:45:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 420, 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.022033764, 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000052461 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2023-09-23T09:45:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 3599, 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 3055, 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3055, 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 151.643069345, 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.049637665 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2023-09-23T09:45:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 544, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 8, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 27, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 2, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 1107, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 242, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 728, 2023-09-23T09:45:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 137, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 759, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 1917, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 223, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 223, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 58, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 25096, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 19007, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 484, 2023-09-23T09:45:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 242, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 2, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 4257, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 2255, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 2, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 26, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 1412, 2023-09-23T09:45:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 43, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 1403, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 570, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 12, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1033, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 1607202986, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 6, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 570, 2023-09-23T09:45:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 7, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 7, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 563, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 191, 2023-09-23T09:45:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 1366, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 2149, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 10, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 2374, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 791, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 225, 2023-09-23T09:45:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 900 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 304, 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 266, 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 9, 2023-09-23T09:45:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 251, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 72, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 941, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 84, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2023-09-23T09:45:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 27, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 2, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2023-09-23T09:45:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 4318, 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 4318, 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2023-09-23T09:45:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 7, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 7, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2023-09-23T09:45:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 19507859, 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2230, 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 13.504993737, 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.006056051 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.618 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 580, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 2037, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 1457, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 106, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 158, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 52, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 1118, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 3131, 2023-09-23T09:45:48.619 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 2013, 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 223, 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 1199, 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 976, 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 63200, 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 198960 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 16621, 2023-09-23T09:45:48.620 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 4880, 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 3599, 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 351, 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 7480, 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 73, 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.194212483, 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002660444 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.621 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 235, 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 10.000300270, 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.042554469 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 173, 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.651113509, 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.009544008 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1061, 2023-09-23T09:45:48.623 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 31.700564746, 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.029878006 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2023-09-23T09:45:48.624 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000100804, 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000100804 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.625 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.627 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.628 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 149, 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.657501025, 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004412758 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2023-09-23T09:45:48.629 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 87, 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3.045803324, 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.035009233 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1078, 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 103.527801860, 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.096036921 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.630 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.632 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.633 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.634 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.635 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.636 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 198, 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.865671324, 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004372077 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.637 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 4, 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 2, 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 2, 2023-09-23T09:45:48.638 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 308, 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 154, 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 970, 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2023-09-23T09:45:48.639 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:48.640 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:48.641 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:48.642 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 1317158, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 1182, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 88, 2023-09-23T09:45:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 1, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8352, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 190, 2023-09-23T09:45:48.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 2896059, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 97568, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:48.645 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 3346, 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 15556362, 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 33, 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 3346, 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2023-09-23T09:45:48.646 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3346, 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 19.031777250, 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005687919 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3346, 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3672 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.647 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 3346, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 636, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 2710, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 58, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 2288, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 121, 2023-09-23T09:45:48.648 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 89, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 571, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2023-09-23T09:45:48.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2023-09-23T09:45:48.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2023-09-23T09:45:48.651 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 107, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 24, 2023-09-23T09:45:48.652 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 355, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 16, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 144 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2023-09-23T09:45:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2023-09-23T09:45:48.654 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 8, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 8, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 72, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 68, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:48.655 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 7779481, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 538476878, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 7779481, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 7779479, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 538476742, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.656 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:48.657 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3346, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 15563754, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3346, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 15563754, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.658 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:48.659 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3346, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3346, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3346, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3346, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.660 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 72, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 7520, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 72, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:48.661 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 58, 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 7520, 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x561a23a4e6e0": { 2023-09-23T09:45:48.662 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 4318, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 15313555, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 4318, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2230, 2023-09-23T09:45:48.663 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 15313555, 2023-09-23T09:45:48.664 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:48.664 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:48.664 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:48.664 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:48.664 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.664 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:48.664 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-09-23T09:45:49.084 INFO:tasks.check_counter:Daemon mds.h mds.exported=26 2023-09-23T09:45:49.084 INFO:tasks.check_counter:Daemon mds.h mds.imported=43 2023-09-23T09:45:49.084 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-09-23T09:45:49.084 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-09-23T09:45:49.084 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T09:45:49.084 DEBUG:tasks.check_counter:Getting stats from k 2023-09-23T09:45:49.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-09-23T09:45:49.758 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T09:45:50.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:49 smithi098 ceph-mon[102386]: pgmap v515: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 6 op/s 2023-09-23T09:45:50.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:49 smithi045 ceph-mon[98627]: pgmap v515: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 6 op/s 2023-09-23T09:45:50.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:49 smithi123 ceph-mon[103577]: pgmap v515: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 6 op/s 2023-09-23T09:45:51.187 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-09-23T09:45:51.187 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 0, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 0, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 0, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.001184583, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000000000, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000260469, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:51.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.189 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 7, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 228, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3424, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 269952, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.011332997, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.007096012, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.002136998, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:51.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 226, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.078669744, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000348096 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 269952 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.191 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 307, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 297, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1451264, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 178480, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.045691747, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.013524061, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.031662468, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.007624697, 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:51.192 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 296, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.084577972, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000285736 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1451264, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 178480 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-09-23T09:45:51.193 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:45:51.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:45:51.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-09-23T09:45:51.196 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148794, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 32, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-09-23T09:45:51.197 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 928, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-09-23T09:45:51.198 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 310, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1407938, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 310, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-09-23T09:45:51.199 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 310, 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1407938, 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-09-23T09:45:51.200 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-09-23T09:45:51.625 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-09-23T09:45:51.625 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-09-23T09:45:51.625 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-09-23T09:45:51.625 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-09-23T09:45:51.625 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-09-23T09:45:51.625 DEBUG:tasks.check_counter:Getting stats from c 2023-09-23T09:45:51.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-09-23T09:45:52.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:51 smithi123 ceph-mon[103577]: pgmap v516: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 KiB/s wr, 7 op/s 2023-09-23T09:45:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:45:52.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:51 smithi098 ceph-mon[102386]: pgmap v516: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 KiB/s wr, 7 op/s 2023-09-23T09:45:52.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:51 smithi045 ceph-mon[98627]: pgmap v516: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 KiB/s wr, 7 op/s 2023-09-23T09:45:54.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:53 smithi098 ceph-mon[102386]: pgmap v517: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 8 op/s 2023-09-23T09:45:54.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:53 smithi045 ceph-mon[98627]: pgmap v517: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 8 op/s 2023-09-23T09:45:54.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:53 smithi123 ceph-mon[103577]: pgmap v517: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 8 op/s 2023-09-23T09:45:56.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:55 smithi098 ceph-mon[102386]: pgmap v518: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 7 op/s 2023-09-23T09:45:56.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:55 smithi045 ceph-mon[98627]: pgmap v518: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 7 op/s 2023-09-23T09:45:56.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:55 smithi123 ceph-mon[103577]: pgmap v518: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 7 op/s 2023-09-23T09:45:58.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:57 smithi098 ceph-mon[102386]: pgmap v519: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-23T09:45:58.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:57 smithi045 ceph-mon[98627]: pgmap v519: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-23T09:45:58.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:57 smithi123 ceph-mon[103577]: pgmap v519: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-23T09:45:59.963 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-23T09:45:59.963 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:45:59.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1772, 2023-09-23T09:45:59.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1738, 2023-09-23T09:45:59.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 8684627, 2023-09-23T09:45:59.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 605641, 2023-09-23T09:45:59.963 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.150352467, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.039021381, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.047583666, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.052650529, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1737, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.374411157, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000215550 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.964 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 374096, 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 146912 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1181, 2023-09-23T09:45:59.965 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1358, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 16651396, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 763690, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.098538687, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.028127354, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.031702460, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.032226838, 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1357, 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.229062900, 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000168800 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-09-23T09:45:59.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 397888 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 392, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 394, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 4294018, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 121919, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.033837044, 2023-09-23T09:45:59.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.007786233, 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.011389646, 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.012174636, 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 392, 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.090149350, 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000229972 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:45:59.969 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-23T09:45:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5130, 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.143385863, 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000027950 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-23T09:45:59.971 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2023-09-23T09:45:59.972 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2023-09-23T09:45:59.973 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 1613, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 852, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 734, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 27, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 395, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 20, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 9, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2023-09-23T09:45:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2023-09-23T09:45:59.975 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1016, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1352474790, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 6, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T09:45:59.976 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T09:45:59.977 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2023-09-23T09:45:59.978 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2023-09-23T09:45:59.979 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2023-09-23T09:45:59.980 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 2537, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2023-09-23T09:45:59.981 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 63, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 40362470, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 41418371, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 41381622, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 2537 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 292, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 1706, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 1414, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 48, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 97, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 49, 2023-09-23T09:45:59.983 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 1613, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 3260, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 1647, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 51052, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 207152 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2023-09-23T09:45:59.984 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.987 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.988 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.989 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2023-09-23T09:45:59.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2023-09-23T09:45:59.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2023-09-23T09:45:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2023-09-23T09:45:59.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.994 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.995 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2023-09-23T09:45:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 2, 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 0, 2023-09-23T09:46:00.000 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2023-09-23T09:46:00.001 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2023-09-23T09:46:00.001 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2023-09-23T09:46:00.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 2310, 2023-09-23T09:46:00.001 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2023-09-23T09:46:00.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.001 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-23T09:46:00.001 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:46:00.002 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:46:00.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-23T09:46:00.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 225091, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 521, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2023-09-23T09:46:00.005 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8352, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 190, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:46:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 2069579, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 68609, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2023-09-23T09:46:00.007 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 3043, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 26, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 4, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 3043, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3043, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 13.940684166, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004581230 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.008 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3043, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3044 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 3043, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 3043, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-09-23T09:46:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 1740, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1300, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-09-23T09:46:00.010 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 2, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-09-23T09:46:00.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-09-23T09:46:00.012 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 107, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-09-23T09:46:00.013 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 20, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 7, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 7, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2023-09-23T09:46:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2023-09-23T09:46:00.015 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:00.016 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3341, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 29311350, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3341, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3341, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 29311350, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3043, 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 28408067, 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3043, 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 28408067, 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.019 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3043, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 3043, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3043, 2023-09-23T09:46:00.020 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3043, 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:00.021 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:00.022 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55adbd230960": { 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:00.025 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-23T09:46:00.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:45:59 smithi098 ceph-mon[102386]: pgmap v520: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 KiB/s wr, 6 op/s 2023-09-23T09:46:00.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:45:59 smithi045 ceph-mon[98627]: pgmap v520: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 KiB/s wr, 6 op/s 2023-09-23T09:46:00.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:45:59 smithi123 ceph-mon[103577]: pgmap v520: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 KiB/s wr, 6 op/s 2023-09-23T09:46:00.379 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-09-23T09:46:00.379 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-09-23T09:46:00.379 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-09-23T09:46:00.379 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-09-23T09:46:00.379 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T09:46:00.380 DEBUG:tasks.check_counter:Getting stats from f 2023-09-23T09:46:00.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-09-23T09:46:01.017 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:46:02.088 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:01 smithi123 ceph-mon[103577]: pgmap v521: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2023-09-23T09:46:02.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:01 smithi098 ceph-mon[102386]: pgmap v521: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2023-09-23T09:46:02.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:01 smithi045 ceph-mon[98627]: pgmap v521: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2023-09-23T09:46:02.965 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3139, 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 3141, 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 13468039, 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 976236, 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.235460977, 2023-09-23T09:46:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.063269951, 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.062431131, 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.092673627, 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3139, 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.508770198, 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000162080 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 799, 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 748, 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 4495763, 2023-09-23T09:46:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 314979, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.084429982, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.022282194, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.047999844, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.020952630, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 748, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.164555643, 2023-09-23T09:46:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000219994 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1648688, 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 184608 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 229, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 748, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 640871, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.011652508, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.007052753, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002446320, 2023-09-23T09:46:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000028618, 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 226, 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.058499406, 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000258846 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 480, 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 640560 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-23T09:46:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5995, 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.144657142, 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000024129 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 1186, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 395, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 776, 2023-09-23T09:46:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 15, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 621, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 4, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 54, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2023-09-23T09:46:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2023-09-23T09:46:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T09:46:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2023-09-23T09:46:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2023-09-23T09:46:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2023-09-23T09:46:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2023-09-23T09:46:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 4323, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 7, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2023-09-23T09:46:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 19518865, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 19518291, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 4323 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 556, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 2004, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 1448, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 79, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 157, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 78, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 1186, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 2625, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 1439, 2023-09-23T09:46:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 56300, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 207152 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2023-09-23T09:46:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2023-09-23T09:46:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2023-09-23T09:46:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2023-09-23T09:46:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2023-09-23T09:46:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 4, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 3, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 4821, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-23T09:46:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:46:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:46:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-23T09:46:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 221358, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 877, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-23T09:46:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8352, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 190, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-23T09:46:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 2493975, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 66809, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-09-23T09:46:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 3559, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 3559, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3559, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 9.126792972, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002564426 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-09-23T09:46:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3559, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3560 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 3559, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 3559, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 2083, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-09-23T09:46:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1474, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-09-23T09:46:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-09-23T09:46:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-09-23T09:46:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 107, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-09-23T09:46:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 24, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 5, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 6, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 1, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2023-09-23T09:46:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2023-09-23T09:46:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3938, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 17588059, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3938, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3938, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 17588059, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3559, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 15324566, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3559, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 15324566, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3559, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 3559, 2023-09-23T09:46:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3559, 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3559, 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-23T09:46:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55de1472c960": { 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:03.027 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-23T09:46:03.899 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-09-23T09:46:03.900 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-09-23T09:46:03.900 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-09-23T09:46:03.900 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-09-23T09:46:03.900 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T09:46:03.900 DEBUG:tasks.check_counter:Getting stats from i 2023-09-23T09:46:03.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-09-23T09:46:04.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:03 smithi098 ceph-mon[102386]: pgmap v522: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2023-09-23T09:46:04.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:03 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:46:04.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:03 smithi045 ceph-mon[98627]: pgmap v522: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2023-09-23T09:46:04.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:03 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:46:04.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:03 smithi123 ceph-mon[103577]: pgmap v522: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2023-09-23T09:46:04.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:03 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T09:46:04.525 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:46:04.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:04 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:46:04.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:04 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:46:04.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:04 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:46:05.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:04 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:46:05.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:04 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:46:05.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:04 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:46:05.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:04 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T09:46:05.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:04 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T09:46:05.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:04 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' 2023-09-23T09:46:06.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:05 smithi098 ceph-mon[102386]: pgmap v523: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 23 KiB/s wr, 7 op/s 2023-09-23T09:46:06.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:05 smithi098 ceph-mon[102386]: osdmap e108: 12 total, 12 up, 12 in 2023-09-23T09:46:06.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:05 smithi045 ceph-mon[98627]: pgmap v523: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 23 KiB/s wr, 7 op/s 2023-09-23T09:46:06.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:05 smithi045 ceph-mon[98627]: osdmap e108: 12 total, 12 up, 12 in 2023-09-23T09:46:06.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:05 smithi123 ceph-mon[103577]: pgmap v523: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 23 KiB/s wr, 7 op/s 2023-09-23T09:46:06.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:05 smithi123 ceph-mon[103577]: osdmap e108: 12 total, 12 up, 12 in 2023-09-23T09:46:06.841 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-23T09:46:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 853, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 856, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 255895, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 3919246, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 14, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.141116393, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.019077835, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.014778406, 2023-09-23T09:46:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.101458301, 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 854, 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.058638909, 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000068663 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 22, 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000003046, 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000138 2023-09-23T09:46:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2672, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 2368 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 8023932, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 8023378, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1325070587, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1276570378, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 11, 2023-09-23T09:46:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 176.767917669, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 70.710483532, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 121.498367165, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.030769230, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 8023376, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 139.963972683, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000017444 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 13661575, 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2.248001242, 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000164 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 7024, 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 578704 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3784385, 2023-09-23T09:46:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 3787362, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 628548789, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 609780321, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 14, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 84.675994489, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 33.845613638, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 58.677134036, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.033481055, 2023-09-23T09:46:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3787361, 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 79.986155855, 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000021119 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6435767, 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.065456586, 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000165 2023-09-23T09:46:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3770832, 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 5759600 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-23T09:46:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4363, 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.956033987, 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000448323 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-23T09:46:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 364, 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.016861890, 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000046323 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 4119, 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 3916, 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2023-09-23T09:46:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3916, 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 281.026244287, 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.071763596 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 203, 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 3, 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2023-09-23T09:46:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 35, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 3, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 1527, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 13, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 6, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 1508, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 1508, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 2436, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 702, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 702, 2023-09-23T09:46:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 57, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 27892, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 20496, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 161, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 254, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 2, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 4476, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 1062, 2023-09-23T09:46:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 1, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 16, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 1454, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 44, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 1363, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 348, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 25, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1033, 2023-09-23T09:46:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1607202986, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 6, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 348, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T09:46:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 348, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 168, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 1332, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 2226, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 1, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 17, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 2313, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 660, 2023-09-23T09:46:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 211, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 1072 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 273, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 277, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 11, 2023-09-23T09:46:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 287, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 115, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 954, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 76, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2023-09-23T09:46:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 16, 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 3, 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2023-09-23T09:46:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 5473, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 5473, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 8, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2023-09-23T09:46:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 8, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 17390735, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1995, 2023-09-23T09:46:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 11.444030163, 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005736355 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 1373, 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 2144, 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 771, 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 88, 2023-09-23T09:46:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 108, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 20, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 1532, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 4126, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 2594, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 702, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1855, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 1153, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 66708, 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 198960 2023-09-23T09:46:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 17523, 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 4458, 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 4119, 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 296, 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 7265, 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2023-09-23T09:46:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 115, 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.240787634, 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002093805 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 110, 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.133869036, 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001216991 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2023-09-23T09:46:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 175, 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.821319037, 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.010407537 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1866, 2023-09-23T09:46:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 14.921404128, 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.007996465 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 202, 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.625501713, 2023-09-23T09:46:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003096543 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 121, 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 14.139115595, 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.116852195 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1148, 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 248.283692531, 2023-09-23T09:46:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.216274993 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003263102, 2023-09-23T09:46:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003263102 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.877 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 178, 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.857291511, 2023-09-23T09:46:06.879 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004816244 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2023-09-23T09:46:06.880 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 4, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 325, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 162, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 978, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.881 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:46:06.882 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:46:06.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-23T09:46:06.884 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 609245, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 1069, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 88, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 1, 2023-09-23T09:46:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8352, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 190, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-23T09:46:06.886 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 4703259, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 61231, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-09-23T09:46:06.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 2889, 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 13490680, 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 28, 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 2889, 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2889, 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 17.423603535, 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006031015 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.888 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2889, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3195 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 2889, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 395, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 2494, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-09-23T09:46:06.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 44, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 2039, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 106, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 1, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 132, 2023-09-23T09:46:06.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 348, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-09-23T09:46:06.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 2, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-09-23T09:46:06.892 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-09-23T09:46:06.893 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 108, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 25, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 353, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 6, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 134 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.894 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 1 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2023-09-23T09:46:06.895 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 10, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 10, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 115, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 72, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-23T09:46:06.896 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 11809173, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 890744883, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 11809173, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 11809172, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 890744811, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.897 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:06.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2889, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 13502487, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2889, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 13502487, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.899 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-23T09:46:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2889, 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 2889, 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2889, 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2889, 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.901 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 115, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 11855, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 115, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 44, 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 11855, 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5560466bc6e0": { 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 32257, 2023-09-23T09:46:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 5473, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 13196431, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 5473, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1995, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 13164174, 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:06.904 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:06.905 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:06.905 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:06.905 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.905 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:06.905 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-23T09:46:07.312 INFO:tasks.check_counter:Daemon mds.i mds.exported=16 2023-09-23T09:46:07.312 INFO:tasks.check_counter:Daemon mds.i mds.imported=44 2023-09-23T09:46:07.313 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-09-23T09:46:07.313 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-09-23T09:46:07.313 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T09:46:07.313 DEBUG:tasks.check_counter:Getting stats from l 2023-09-23T09:46:07.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-09-23T09:46:08.036 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T09:46:08.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:07 smithi123 ceph-mon[103577]: pgmap v525: 97 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 71 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 25 op/s 2023-09-23T09:46:08.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:07 smithi098 ceph-mon[102386]: pgmap v525: 97 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 71 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 25 op/s 2023-09-23T09:46:08.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:07 smithi045 ceph-mon[98627]: pgmap v525: 97 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 71 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 25 op/s 2023-09-23T09:46:09.465 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-23T09:46:09.465 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T09:46:09.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2023-09-23T09:46:09.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001140114, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000307955, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:09.466 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T09:46:09.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 231, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 480, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 270384, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.010806649, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.006736149, 2023-09-23T09:46:09.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001930721, 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 228, 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.075333675, 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000330410 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:09.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 480, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 270384 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 316, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 304, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1455568, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 182048, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2023-09-23T09:46:09.470 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.046557332, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.013603838, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.032531698, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.007733411, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 304, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.066574017, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000218993 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-23T09:46:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1455568, 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 182048 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-23T09:46:09.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T09:46:09.473 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T09:46:09.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-23T09:46:09.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148744, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-23T09:46:09.476 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-23T09:46:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-23T09:46:09.478 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 315, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1408588, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 315, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 315, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1408588, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-23T09:46:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-23T09:46:09.480 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:09.480 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-23T09:46:09.480 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-23T09:46:09.901 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-09-23T09:46:09.901 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-09-23T09:46:09.901 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-09-23T09:46:09.902 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-09-23T09:46:09.902 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-09-23T09:46:09.902 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-09-23T09:46:09.913 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-23T09:46:09.913 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:46:09.913 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T09:46:09.964 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:46:09.964 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T09:46:10.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:46:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 -- 172.21.15.45:0/1522027363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6481008a0 msgr2=0x7fa6481048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/1522027363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6481008a0 0x7fa6481048e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098f0 tx=0x7fa638009c00 comp rx=0 tx=0).stop 2023-09-23T09:46:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 -- 172.21.15.45:0/1522027363 shutdown_connections 2023-09-23T09:46:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/1522027363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa648105010 0x7fa64810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/1522027363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6480ffef0 0x7fa6481002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/1522027363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6481008a0 0x7fa6481048e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 -- 172.21.15.45:0/1522027363 >> 172.21.15.45:0/1522027363 conn(0x7fa6480fb730 msgr2=0x7fa6480fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 -- 172.21.15.45:0/1522027363 shutdown_connections 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.218+0000 7fa64dcc4700 1 -- 172.21.15.45:0/1522027363 wait complete. 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 Processor -- start 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 -- start start 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6480ffef0 0x7fa648199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa648105010 0x7fa648199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64819dac0 0x7fa64819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64819e540 con 0x7fa648105010 2023-09-23T09:46:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64819e6b0 con 0x7fa64819dac0 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa64dcc4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64819e870 con 0x7fa6480ffef0 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa6477fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6480ffef0 0x7fa648199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa647fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64819dac0 0x7fa64819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa646ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa648105010 0x7fa648199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa6477fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6480ffef0 0x7fa648199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.45:39460/0 (socket says 172.21.15.45:39460) 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa6477fe700 1 -- 172.21.15.45:0/496595259 learned_addr learned my addr 172.21.15.45:0/496595259 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.219+0000 7fa647fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64819dac0 0x7fa64819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58708/0 (socket says 172.21.15.45:58708) 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa6477fe700 1 -- 172.21.15.45:0/496595259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64819dac0 msgr2=0x7fa64819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa6477fe700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64819dac0 0x7fa64819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa6477fe700 1 -- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa648105010 msgr2=0x7fa648199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa6477fe700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa648105010 0x7fa648199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa6477fe700 1 -- 172.21.15.45:0/496595259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630009520 con 0x7fa6480ffef0 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa646ffd700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa648105010 0x7fa648199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa6477fe700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6480ffef0 0x7fa648199120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa63000eb30 tx=0x7fa63000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa644ff9700 1 -- 172.21.15.45:0/496595259 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630009880 con 0x7fa6480ffef0 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa644ff9700 1 -- 172.21.15.45:0/496595259 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6300044e0 con 0x7fa6480ffef0 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa644ff9700 1 -- 172.21.15.45:0/496595259 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630003d80 con 0x7fa6480ffef0 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa640009d00 con 0x7fa6480ffef0 2023-09-23T09:46:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.220+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa638009520 con 0x7fa6480ffef0 2023-09-23T09:46:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.221+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa648067380 con 0x7fa6480ffef0 2023-09-23T09:46:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.222+0000 7fa644ff9700 1 -- 172.21.15.45:0/496595259 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fa630003f00 con 0x7fa6480ffef0 2023-09-23T09:46:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.223+0000 7fa644ff9700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa6340a17d0 0x7fa6340a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.223+0000 7fa646ffd700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa6340a17d0 0x7fa6340a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.223+0000 7fa644ff9700 1 -- 172.21.15.45:0/496595259 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa630014070 con 0x7fa6480ffef0 2023-09-23T09:46:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.223+0000 7fa646ffd700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa6340a17d0 0x7fa6340a3c90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098c0 tx=0x7fa638019040 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.226+0000 7fa644ff9700 1 -- 172.21.15.45:0/496595259 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa630096be0 con 0x7fa6480ffef0 2023-09-23T09:46:10.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:09 smithi098 ceph-mon[102386]: pgmap v526: 97 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 71 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 25 op/s 2023-09-23T09:46:10.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:09 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:46:10.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:09 smithi098 ceph-mon[102386]: scrub summary: active paths [/] 2023-09-23T09:46:10.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:09 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:46:10.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:09 smithi098 ceph-mon[102386]: scrub summary: idle+waiting paths [/] 2023-09-23T09:46:10.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:09 smithi045 ceph-mon[98627]: pgmap v526: 97 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 71 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 25 op/s 2023-09-23T09:46:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:09 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:46:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:09 smithi045 ceph-mon[98627]: scrub summary: active paths [/] 2023-09-23T09:46:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:09 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:46:10.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:09 smithi045 ceph-mon[98627]: scrub summary: idle+waiting paths [/] 2023-09-23T09:46:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:09 smithi123 ceph-mon[103577]: pgmap v526: 97 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 71 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 25 op/s 2023-09-23T09:46:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:09 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:46:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:09 smithi123 ceph-mon[103577]: scrub summary: active paths [/] 2023-09-23T09:46:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:09 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:46:10.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:09 smithi123 ceph-mon[103577]: scrub summary: idle+waiting paths [/] 2023-09-23T09:46:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.432+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa648100aa0 con 0x7fa6480ffef0 2023-09-23T09:46:10.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.434+0000 7fa644ff9700 1 -- 172.21.15.45:0/496595259 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+23078 (secure 0 0 0) 0x7fa63009a200 con 0x7fa6480ffef0 2023-09-23T09:46:10.436 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:46:10.436 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":108,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:46:04.972961+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/230875070":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3269510756":"2023-09-24T09:29:25.699095+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/2324483632":"2023-09-24T09:29:25.699095+0000","172.21.15.45:6800/944196506":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/65945615":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/944196506":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/1970790569":"2023-09-24T09:29:25.699095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa6340a17d0 msgr2=0x7fa6340a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa6340a17d0 0x7fa6340a3c90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098c0 tx=0x7fa638019040 comp rx=0 tx=0).stop 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6480ffef0 msgr2=0x7fa648199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6480ffef0 0x7fa648199120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa63000eb30 tx=0x7fa63000ee40 comp rx=0 tx=0).stop 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 shutdown_connections 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64819dac0 0x7fa64819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6480ffef0 0x7fa648199120 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa6340a17d0 0x7fa6340a3c90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 --2- 172.21.15.45:0/496595259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa648105010 0x7fa648199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 >> 172.21.15.45:0/496595259 conn(0x7fa6480fb730 msgr2=0x7fa648068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 shutdown_connections 2023-09-23T09:46:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.436+0000 7fa64dcc4700 1 -- 172.21.15.45:0/496595259 wait complete. 2023-09-23T09:46:10.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T09:24:00.782778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T09:27:48.176304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T09:27:49.453516+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T09:46:10.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T09:46:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 -- 172.21.15.45:0/3920319486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc10b350 msgr2=0x7f8bbc10d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 --2- 172.21.15.45:0/3920319486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc10b350 0x7f8bbc10d810 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009f90 tx=0x7f8bb400b330 comp rx=0 tx=0).stop 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 -- 172.21.15.45:0/3920319486 shutdown_connections 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 --2- 172.21.15.45:0/3920319486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc10b350 0x7f8bbc10d810 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 --2- 172.21.15.45:0/3920319486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bbc101f30 0x7f8bbc102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 --2- 172.21.15.45:0/3920319486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc102850 0x7f8bbc10ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 -- 172.21.15.45:0/3920319486 >> 172.21.15.45:0/3920319486 conn(0x7f8bbc0fb770 msgr2=0x7f8bbc0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 -- 172.21.15.45:0/3920319486 shutdown_connections 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 -- 172.21.15.45:0/3920319486 wait complete. 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 Processor -- start 2023-09-23T09:46:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.579+0000 7f8bc4394700 1 -- start start 2023-09-23T09:46:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc4394700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc101f30 0x7f8bbc196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc4394700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc102850 0x7f8bbc197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc4394700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bbc10b350 0x7f8bbc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc4394700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc19bfb0 con 0x7f8bbc102850 2023-09-23T09:46:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc4394700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc19c120 con 0x7f8bbc101f30 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc4394700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc19c2e0 con 0x7f8bbc10b350 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc102850 0x7f8bbc197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc102850 0x7f8bbc197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37370/0 (socket says 172.21.15.45:37370) 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 -- 172.21.15.45:0/31865468 learned_addr learned my addr 172.21.15.45:0/31865468 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc2130700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc101f30 0x7f8bbc196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc2931700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bbc10b350 0x7f8bbc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 -- 172.21.15.45:0/31865468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bbc10b350 msgr2=0x7f8bbc19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bbc10b350 0x7f8bbc19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 -- 172.21.15.45:0/31865468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc101f30 msgr2=0x7f8bbc196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc101f30 0x7f8bbc196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 -- 172.21.15.45:0/31865468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0095f0 con 0x7f8bbc102850 2023-09-23T09:46:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bc192f700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc102850 0x7f8bbc197510 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009fd0 tx=0x7f8bac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.580+0000 7f8bb37fe700 1 -- 172.21.15.45:0/31865468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac009960 con 0x7f8bbc102850 2023-09-23T09:46:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.581+0000 7f8bb37fe700 1 -- 172.21.15.45:0/31865468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bac0044e0 con 0x7f8bbc102850 2023-09-23T09:46:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.581+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb4009c40 con 0x7f8bbc102850 2023-09-23T09:46:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.581+0000 7f8bb37fe700 1 -- 172.21.15.45:0/31865468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac003e90 con 0x7f8bbc102850 2023-09-23T09:46:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.581+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc19c7b0 con 0x7f8bbc102850 2023-09-23T09:46:10.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.582+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc067ba0 con 0x7f8bbc102850 2023-09-23T09:46:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.583+0000 7f8bb37fe700 1 -- 172.21.15.45:0/31865468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f8bac004000 con 0x7f8bbc102850 2023-09-23T09:46:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.583+0000 7f8bb37fe700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8ba80a1790 0x7f8ba80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.584+0000 7f8bc2130700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8ba80a1790 0x7f8ba80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.584+0000 7f8bb37fe700 1 -- 172.21.15.45:0/31865468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8bac014070 con 0x7f8bbc102850 2023-09-23T09:46:10.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.584+0000 7f8bc2130700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8ba80a1790 0x7f8ba80a3c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009c60 tx=0x7f8bb8009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:10.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.588+0000 7f8bb37fe700 1 -- 172.21.15.45:0/31865468 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8bac0968f0 con 0x7f8bbc102850 2023-09-23T09:46:10.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.791+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bbc110e00 con 0x7f8bbc102850 2023-09-23T09:46:10.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.791+0000 7f8bb37fe700 1 -- 172.21.15.45:0/31865468 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bac099f10 con 0x7f8bbc102850 2023-09-23T09:46:10.793 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-09-23T09:46:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8ba80a1790 msgr2=0x7f8ba80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8ba80a1790 0x7f8ba80a3c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009c60 tx=0x7f8bb8009260 comp rx=0 tx=0).stop 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc102850 msgr2=0x7f8bbc197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc102850 0x7f8bbc197510 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009fd0 tx=0x7f8bac00ee70 comp rx=0 tx=0).stop 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 shutdown_connections 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bbc10b350 0x7f8bbc19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc101f30 0x7f8bbc196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f8ba80a1790 0x7f8ba80a3c50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 --2- 172.21.15.45:0/31865468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bbc102850 0x7f8bbc197510 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 >> 172.21.15.45:0/31865468 conn(0x7f8bbc0fb770 msgr2=0x7f8bbc10d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 shutdown_connections 2023-09-23T09:46:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.794+0000 7f8bc4394700 1 -- 172.21.15.45:0/31865468 wait complete. 2023-09-23T09:46:10.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T09:46:10.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 -- 172.21.15.45:0/519827851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8073af0 msgr2=0x7f77b810d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 --2- 172.21.15.45:0/519827851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8073af0 0x7f77b810d110 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009990 tx=0x7f77b4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 -- 172.21.15.45:0/519827851 shutdown_connections 2023-09-23T09:46:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 --2- 172.21.15.45:0/519827851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 0x7f77b810fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 --2- 172.21.15.45:0/519827851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77b80731d0 0x7f77b80735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 --2- 172.21.15.45:0/519827851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8073af0 0x7f77b810d110 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 -- 172.21.15.45:0/519827851 >> 172.21.15.45:0/519827851 conn(0x7f77b80fc880 msgr2=0x7f77b80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:10.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 -- 172.21.15.45:0/519827851 shutdown_connections 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.936+0000 7f77c050f700 1 -- 172.21.15.45:0/519827851 wait complete. 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 Processor -- start 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 -- start start 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b80731d0 0x7f77b81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77b8073af0 0x7f77b8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 0x7f77b819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b819e0c0 con 0x7f77b80731d0 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b819e230 con 0x7f77b810d650 2023-09-23T09:46:10.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77c050f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b819e3f0 con 0x7f77b8073af0 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77be2ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b80731d0 0x7f77b81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 0x7f77b819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77bdaaa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77b8073af0 0x7f77b8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 0x7f77b819da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58736/0 (socket says 172.21.15.45:58736) 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77be2ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b80731d0 0x7f77b81990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37382/0 (socket says 172.21.15.45:37382) 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 -- 172.21.15.45:0/2282302926 learned_addr learned my addr 172.21.15.45:0/2282302926 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 -- 172.21.15.45:0/2282302926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77b8073af0 msgr2=0x7f77b8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77b8073af0 0x7f77b8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 -- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b80731d0 msgr2=0x7f77b81990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b80731d0 0x7f77b81990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 -- 172.21.15.45:0/2282302926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40095f0 con 0x7f77b810d650 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77be2ab700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b80731d0 0x7f77b81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.937+0000 7f77beaac700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 0x7f77b819da80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77b000ef30 tx=0x7f77b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.938+0000 7f77af7fe700 1 -- 172.21.15.45:0/2282302926 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0018070 con 0x7f77b810d650 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.938+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b819e670 con 0x7f77b810d650 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.938+0000 7f77af7fe700 1 -- 172.21.15.45:0/2282302926 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77b000f040 con 0x7f77b810d650 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.938+0000 7f77af7fe700 1 -- 172.21.15.45:0/2282302926 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b000cd00 con 0x7f77b810d650 2023-09-23T09:46:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.938+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b81a5f10 con 0x7f77b810d650 2023-09-23T09:46:10.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.939+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b8067ba0 con 0x7f77b810d650 2023-09-23T09:46:10.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.940+0000 7f77af7fe700 1 -- 172.21.15.45:0/2282302926 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f77b0003a00 con 0x7f77b810d650 2023-09-23T09:46:10.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.941+0000 7f77af7fe700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f77a40a1710 0x7f77a40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:10.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.941+0000 7f77be2ab700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f77a40a1710 0x7f77a40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:10.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.941+0000 7f77af7fe700 1 -- 172.21.15.45:0/2282302926 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f77b00cd880 con 0x7f77b810d650 2023-09-23T09:46:10.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.941+0000 7f77be2ab700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f77a40a1710 0x7f77a40a3bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77a8011490 tx=0x7f77a8009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:10.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:10.948+0000 7f77af7fe700 1 -- 172.21.15.45:0/2282302926 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f77b0096af0 con 0x7f77b810d650 2023-09-23T09:46:11.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:10 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/496595259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:46:11.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:10 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/31865468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:46:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.151+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77b819a430 con 0x7f77b810d650 2023-09-23T09:46:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.152+0000 7f77af7fe700 1 -- 172.21.15.45:0/2282302926 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77b009a110 con 0x7f77b810d650 2023-09-23T09:46:11.153 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f77a40a1710 msgr2=0x7f77a40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f77a40a1710 0x7f77a40a3bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77a8011490 tx=0x7f77a8009260 comp rx=0 tx=0).stop 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 msgr2=0x7f77b819da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 0x7f77b819da80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77b000ef30 tx=0x7f77b000c3a0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 shutdown_connections 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77b810d650 0x7f77b819da80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f77a40a1710 0x7f77a40a3bd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b80731d0 0x7f77b81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 --2- 172.21.15.45:0/2282302926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77b8073af0 0x7f77b8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 >> 172.21.15.45:0/2282302926 conn(0x7f77b80fc880 msgr2=0x7f77b810f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:11.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 shutdown_connections 2023-09-23T09:46:11.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.154+0000 7f77c050f700 1 -- 172.21.15.45:0/2282302926 wait complete. 2023-09-23T09:46:11.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T09:46:11.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:10 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/496595259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:46:11.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:10 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/31865468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:46:11.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 -- 172.21.15.45:0/3519773942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c4101010 msgr2=0x7f92c41013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/3519773942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c4101010 0x7f92c41013f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009990 tx=0x7f92b4009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 -- 172.21.15.45:0/3519773942 shutdown_connections 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/3519773942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4106130 0x7f92c410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/3519773942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c4101010 0x7f92c41013f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/3519773942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c41019c0 0x7f92c4105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 -- 172.21.15.45:0/3519773942 >> 172.21.15.45:0/3519773942 conn(0x7f92c40fc8f0 msgr2=0x7f92c40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 -- 172.21.15.45:0/3519773942 shutdown_connections 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.297+0000 7f92cc0ca700 1 -- 172.21.15.45:0/3519773942 wait complete. 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 Processor -- start 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 -- start start 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c4101010 0x7f92c4196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4106130 0x7f92c41974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c419b940 0x7f92c419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419c3c0 con 0x7f92c419b940 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419c530 con 0x7f92c4101010 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92cc0ca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419c6f0 con 0x7f92c4106130 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92ca667700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c419b940 0x7f92c419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92c9e66700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c4101010 0x7f92c4196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92c9665700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4106130 0x7f92c41974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92c9e66700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c4101010 0x7f92c4196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58744/0 (socket says 172.21.15.45:58744) 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92ca667700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c419b940 0x7f92c419bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37398/0 (socket says 172.21.15.45:37398) 2023-09-23T09:46:11.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.298+0000 7f92c9e66700 1 -- 172.21.15.45:0/401436963 learned_addr learned my addr 172.21.15.45:0/401436963 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92c9e66700 1 -- 172.21.15.45:0/401436963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4106130 msgr2=0x7f92c41974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92c9e66700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4106130 0x7f92c41974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92c9e66700 1 -- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c419b940 msgr2=0x7f92c419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92c9e66700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c419b940 0x7f92c419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92c9e66700 1 -- 172.21.15.45:0/401436963 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40095f0 con 0x7f92c4101010 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92ca667700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c419b940 0x7f92c419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92c9665700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4106130 0x7f92c41974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92c9e66700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c4101010 0x7f92c4196fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92b4004730 tx=0x7f92b4004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92baffd700 1 -- 172.21.15.45:0/401436963 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b401d070 con 0x7f92c4101010 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c41a3bf0 con 0x7f92c4101010 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92baffd700 1 -- 172.21.15.45:0/401436963 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92b4005a00 con 0x7f92c4101010 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92baffd700 1 -- 172.21.15.45:0/401436963 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4005300 con 0x7f92c4101010 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.299+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c41a4060 con 0x7f92c4101010 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.300+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c4067ba0 con 0x7f92c4101010 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.301+0000 7f92baffd700 1 -- 172.21.15.45:0/401436963 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f92b40049d0 con 0x7f92c4101010 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.302+0000 7f92baffd700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f92b00a1710 0x7f92b00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.302+0000 7f92c9665700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f92b00a1710 0x7f92b00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.302+0000 7f92baffd700 1 -- 172.21.15.45:0/401436963 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f92b40ce3a0 con 0x7f92c4101010 2023-09-23T09:46:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.302+0000 7f92c9665700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f92b00a1710 0x7f92b00a3bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f92c41985b0 tx=0x7f92c0009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.305+0000 7f92baffd700 1 -- 172.21.15.45:0/401436963 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f92b4097610 con 0x7f92c4101010 2023-09-23T09:46:11.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:10 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/496595259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:46:11.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:10 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/31865468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T09:46:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.507+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92c4101a60 con 0x7f92c4101010 2023-09-23T09:46:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.507+0000 7f92baffd700 1 -- 172.21.15.45:0/401436963 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92b409ac30 con 0x7f92c4101010 2023-09-23T09:46:11.508 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f92b00a1710 msgr2=0x7f92b00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f92b00a1710 0x7f92b00a3bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f92c41985b0 tx=0x7f92c0009260 comp rx=0 tx=0).stop 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c4101010 msgr2=0x7f92c4196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c4101010 0x7f92c4196fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92b4004730 tx=0x7f92b4004760 comp rx=0 tx=0).stop 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 shutdown_connections 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f92b00a1710 0x7f92b00a3bd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c419b940 0x7f92c419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c4101010 0x7f92c4196fa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 --2- 172.21.15.45:0/401436963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4106130 0x7f92c41974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 >> 172.21.15.45:0/401436963 conn(0x7f92c40fc8f0 msgr2=0x7f92c40fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 shutdown_connections 2023-09-23T09:46:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.509+0000 7f92cc0ca700 1 -- 172.21.15.45:0/401436963 wait complete. 2023-09-23T09:46:11.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T09:46:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 -- 172.21.15.45:0/201086759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62740ffed0 msgr2=0x7f62741002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 --2- 172.21.15.45:0/201086759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62740ffed0 0x7f62741002b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6268009990 tx=0x7f6268009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 -- 172.21.15.45:0/201086759 shutdown_connections 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 --2- 172.21.15.45:0/201086759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6274104ff0 0x7f627410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 --2- 172.21.15.45:0/201086759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62740ffed0 0x7f62741002b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 --2- 172.21.15.45:0/201086759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6274100880 0x7f62741048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 -- 172.21.15.45:0/201086759 >> 172.21.15.45:0/201086759 conn(0x7f62740fb770 msgr2=0x7f62740fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 -- 172.21.15.45:0/201086759 shutdown_connections 2023-09-23T09:46:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.648+0000 7f627b038700 1 -- 172.21.15.45:0/201086759 wait complete. 2023-09-23T09:46:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 Processor -- start 2023-09-23T09:46:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 -- start start 2023-09-23T09:46:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62740ffed0 0x7f6274197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6274104ff0 0x7f6274198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6274198900 0x7f62741a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62741a3610 con 0x7f6274198900 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62741a3780 con 0x7f62740ffed0 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f627b038700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62741a3940 con 0x7f6274104ff0 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f62795d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6274198900 0x7f62741a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f6278dd4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62740ffed0 0x7f6274197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f62795d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6274198900 0x7f62741a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37412/0 (socket says 172.21.15.45:37412) 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f6273fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6274104ff0 0x7f6274198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f62795d5700 1 -- 172.21.15.45:0/3222258012 learned_addr learned my addr 172.21.15.45:0/3222258012 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f6278dd4700 1 -- 172.21.15.45:0/3222258012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6274104ff0 msgr2=0x7f6274198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f6278dd4700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6274104ff0 0x7f6274198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f6278dd4700 1 -- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6274198900 msgr2=0x7f62741a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f6278dd4700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6274198900 0x7f62741a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f6278dd4700 1 -- 172.21.15.45:0/3222258012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62680095f0 con 0x7f62740ffed0 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.649+0000 7f62795d5700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6274198900 0x7f62741a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.650+0000 7f6278dd4700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62740ffed0 0x7f6274197de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6268009990 tx=0x7f6268004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.650+0000 7f6271ffb700 1 -- 172.21.15.45:0/3222258012 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626801d070 con 0x7f62740ffed0 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.650+0000 7f6271ffb700 1 -- 172.21.15.45:0/3222258012 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6268021410 con 0x7f62740ffed0 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.650+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62741a3bc0 con 0x7f62740ffed0 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.650+0000 7f6271ffb700 1 -- 172.21.15.45:0/3222258012 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268017650 con 0x7f62740ffed0 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.650+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62741a3fe0 con 0x7f62740ffed0 2023-09-23T09:46:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.651+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6274067ba0 con 0x7f62740ffed0 2023-09-23T09:46:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.652+0000 7f6271ffb700 1 -- 172.21.15.45:0/3222258012 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f62680217f0 con 0x7f62740ffed0 2023-09-23T09:46:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.653+0000 7f6271ffb700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62640a1710 0x7f62640a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.653+0000 7f6273fff700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62640a1710 0x7f62640a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.653+0000 7f6271ffb700 1 -- 172.21.15.45:0/3222258012 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f62680ced50 con 0x7f62740ffed0 2023-09-23T09:46:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.653+0000 7f6273fff700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62640a1710 0x7f62640a3bd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6274068850 tx=0x7f6260009260 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.656+0000 7f6271ffb700 1 -- 172.21.15.45:0/3222258012 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6268098020 con 0x7f62740ffed0 2023-09-23T09:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.874+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f62741008d0 con 0x7f62740ffed0 2023-09-23T09:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.875+0000 7f6271ffb700 1 -- 172.21.15.45:0/3222258012 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v108) v1 ==== 81+0+997 (secure 0 0 0) 0x7f626809b640 con 0x7f62740ffed0 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/230875070 2023-09-24T09:29:25.699095+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2302872888 2023-09-24T09:21:54.248141+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3011609987 2023-09-24T09:21:28.141138+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/258118400 2023-09-24T09:21:28.141138+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3656954721 2023-09-24T09:21:54.248141+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3269510756 2023-09-24T09:29:25.699095+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/4072483061 2023-09-24T09:21:28.141138+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/4072483061 2023-09-24T09:21:28.141138+0000 2023-09-23T09:46:11.877 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1791488577 2023-09-24T09:21:54.248141+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1791488577 2023-09-24T09:21:54.248141+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/690218910 2023-09-24T09:21:54.248141+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2324483632 2023-09-24T09:29:25.699095+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/944196506 2023-09-24T09:29:25.699095+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/65945615 2023-09-24T09:29:25.699095+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3832887419 2023-09-24T09:21:28.141138+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/944196506 2023-09-24T09:29:25.699095+0000 2023-09-23T09:46:11.878 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1970790569 2023-09-24T09:29:25.699095+0000 2023-09-23T09:46:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62640a1710 msgr2=0x7f62640a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62640a1710 0x7f62640a3bd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6274068850 tx=0x7f6260009260 comp rx=0 tx=0).stop 2023-09-23T09:46:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62740ffed0 msgr2=0x7f6274197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:11.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62740ffed0 0x7f6274197de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6268009990 tx=0x7f6268004730 comp rx=0 tx=0).stop 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 shutdown_connections 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f62640a1710 0x7f62640a3bd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6274198900 0x7f62741a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62740ffed0 0x7f6274197de0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 --2- 172.21.15.45:0/3222258012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6274104ff0 0x7f6274198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 >> 172.21.15.45:0/3222258012 conn(0x7f62740fb770 msgr2=0x7f62740fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 shutdown_connections 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:11.877+0000 7f627b038700 1 -- 172.21.15.45:0/3222258012 wait complete. 2023-09-23T09:46:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:listed 17 entries 2023-09-23T09:46:11.891 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-09-23T09:46:11.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:46:11.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /tmp/cephfs 2023-09-23T09:46:12.035 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:46:12.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:46:12.035 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-09-23T09:46:12.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:46:12.220 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-09-23T09:46:12.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:12 smithi045 ceph-mon[98627]: pgmap v527: 97 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 83 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 28 op/s 2023-09-23T09:46:12.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2282302926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:46:12.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/401436963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:46:12.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:12 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/3222258012' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T09:46:12.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:12 smithi123 ceph-mon[103577]: pgmap v527: 97 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 83 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 28 op/s 2023-09-23T09:46:12.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2282302926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:46:12.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/401436963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:46:12.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:12 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/3222258012' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T09:46:12.416 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-09-23T09:46:12.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:46:12.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-09-23T09:46:12.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:12 smithi098 ceph-mon[102386]: pgmap v527: 97 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 83 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 28 op/s 2023-09-23T09:46:12.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2282302926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T09:46:12.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/401436963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T09:46:12.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:12 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/3222258012' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T09:46:12.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:46:12.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-09-23T09:46:12.844 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-23T09:46:12.844 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-09-23T09:46:12.844 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T09:46:12.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18004 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/56a1af9ee8194552dc0566d1e605f9f2316b0d88fe6e01d0cfe8fdcc7a331cad/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/56a1af9ee8194552dc0566d1e605f9f2316b0d88fe6e01d0cfe8fdcc7a331cad/diff,workdir=/var/lib/containers/storage/overlay/56a1af9ee8194552dc0566d1e605f9f2316b0d88fe6e01d0cfe8fdcc7a331cad/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/904e944e2158f2bc806e7273a0ed93b9fda6f6fcd2aa63c12e28d2a2bd154f4a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/904e944e2158f2bc806e7273a0ed93b9fda6f6fcd2aa63c12e28d2a2bd154f4a/diff,workdir=/var/lib/containers/storage/overlay/904e944e2158f2bc806e7273a0ed93b9fda6f6fcd2aa63c12e28d2a2bd154f4a/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/d7a3b0d4f99422a1b59e67a3ccb116b63c08a47796f12803f1b9be80b8bfaba3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/d7a3b0d4f99422a1b59e67a3ccb116b63c08a47796f12803f1b9be80b8bfaba3/diff,workdir=/var/lib/containers/storage/overlay/d7a3b0d4f99422a1b59e67a3ccb116b63c08a47796f12803f1b9be80b8bfaba3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4bcdc6afa6c690b34aa8e9c2b597ac36f17fd40911c81a3d2b2827c4af4c6f7a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/4bcdc6afa6c690b34aa8e9c2b597ac36f17fd40911c81a3d2b2827c4af4c6f7a/diff,workdir=/var/lib/containers/storage/overlay/4bcdc6afa6c690b34aa8e9c2b597ac36f17fd40911c81a3d2b2827c4af4c6f7a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/71cdcb5bf1428bde6089ff6fb980d42a09f933ea69c3abd23d06bd7f3044d102/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/71cdcb5bf1428bde6089ff6fb980d42a09f933ea69c3abd23d06bd7f3044d102/diff,workdir=/var/lib/containers/storage/overlay/71cdcb5bf1428bde6089ff6fb980d42a09f933ea69c3abd23d06bd7f3044d102/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/80c945cafab6d6a4c447122eea23f23ff4f76c20871182911a88f989ef5f68e7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/80c945cafab6d6a4c447122eea23f23ff4f76c20871182911a88f989ef5f68e7/diff,workdir=/var/lib/containers/storage/overlay/80c945cafab6d6a4c447122eea23f23ff4f76c20871182911a88f989ef5f68e7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/510466046ed14777c91473cc4739a63b7a256511bc987f896c85ec9418ed18c2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/510466046ed14777c91473cc4739a63b7a256511bc987f896c85ec9418ed18c2/diff,workdir=/var/lib/containers/storage/overlay/510466046ed14777c91473cc4739a63b7a256511bc987f896c85ec9418ed18c2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5471c2a267725983a148511b01cb18164330b5d075790b166e9253d401868daa/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/5471c2a267725983a148511b01cb18164330b5d075790b166e9253d401868daa/diff,workdir=/var/lib/containers/storage/overlay/5471c2a267725983a148511b01cb18164330b5d075790b166e9253d401868daa/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3066ccfd65994694a0e6be489396a2d8d9f53b28f9aec14795a025eb179c3da4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/3066ccfd65994694a0e6be489396a2d8d9f53b28f9aec14795a025eb179c3da4/diff,workdir=/var/lib/containers/storage/overlay/3066ccfd65994694a0e6be489396a2d8d9f53b28f9aec14795a025eb179c3da4/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8f58dc97a56c64ed7dd5b5a9775d41fe32fcfc8a0e6ae7e4914dfa1be98fb5f2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LE4ZLHD2QIWSEKI4V7WV63VFSB:/var/lib/containers/storage/overlay/l/CQE75OGTTKQ6PE2NDPJM2R5ZIM,upperdir=/var/lib/containers/storage/overlay/8f58dc97a56c64ed7dd5b5a9775d41fe32fcfc8a0e6ae7e4914dfa1be98fb5f2/diff,workdir=/var/lib/containers/storage/overlay/8f58dc97a56c64ed7dd5b5a9775d41fe32fcfc8a0e6ae7e4914dfa1be98fb5f2/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-09-23T09:46:12.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:46:12.869 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-09-23T09:46:12.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T09:46:12.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-09-23T09:46:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2023-09-23T09:46:13.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:46:13.034 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-23T09:46:13.046 INFO:tasks.cephadm:Teardown begin 2023-09-23T09:46:13.046 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T09:46:13.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:46:13.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.172+0000 7ff9409ce700 1 -- 172.21.15.45:0/3965917532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c0ffed0 msgr2=0x7ff93c1002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.172+0000 7ff9409ce700 1 --2- 172.21.15.45:0/3965917532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c0ffed0 0x7ff93c1002b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009990 tx=0x7ff92c009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.172+0000 7ff9409ce700 1 -- 172.21.15.45:0/3965917532 shutdown_connections 2023-09-23T09:46:13.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.172+0000 7ff9409ce700 1 --2- 172.21.15.45:0/3965917532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff93c104ff0 0x7ff93c10cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.172+0000 7ff9409ce700 1 --2- 172.21.15.45:0/3965917532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c0ffed0 0x7ff93c1002b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.172+0000 7ff9409ce700 1 --2- 172.21.15.45:0/3965917532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff93c100880 0x7ff93c1048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.172+0000 7ff9409ce700 1 -- 172.21.15.45:0/3965917532 >> 172.21.15.45:0/3965917532 conn(0x7ff93c0fb730 msgr2=0x7ff93c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:13.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.173+0000 7ff9409ce700 1 -- 172.21.15.45:0/3965917532 shutdown_connections 2023-09-23T09:46:13.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.173+0000 7ff9409ce700 1 -- 172.21.15.45:0/3965917532 wait complete. 2023-09-23T09:46:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.173+0000 7ff9409ce700 1 Processor -- start 2023-09-23T09:46:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.173+0000 7ff9409ce700 1 -- start start 2023-09-23T09:46:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.173+0000 7ff9409ce700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff93c0ffed0 0x7ff93c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.173+0000 7ff9409ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff93c104ff0 0x7ff93c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9409ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c19a700 0x7ff93c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9409ce700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c19b180 con 0x7ff93c19a700 2023-09-23T09:46:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9409ce700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c19b2f0 con 0x7ff93c0ffed0 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9409ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c19b4b0 con 0x7ff93c104ff0 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c19a700 0x7ff93c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c19a700 0x7ff93c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37424/0 (socket says 172.21.15.45:37424) 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff93c0ffed0 0x7ff93c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff939d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff93c104ff0 0x7ff93c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 -- 172.21.15.45:0/545745301 learned_addr learned my addr 172.21.15.45:0/545745301 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 -- 172.21.15.45:0/545745301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff93c104ff0 msgr2=0x7ff93c1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff93c104ff0 0x7ff93c1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 -- 172.21.15.45:0/545745301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff93c0ffed0 msgr2=0x7ff93c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff93c0ffed0 0x7ff93c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 -- 172.21.15.45:0/545745301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0095f0 con 0x7ff93c19a700 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff939d9b700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff93c104ff0 0x7ff93c1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff93ad9d700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c19a700 0x7ff93c19ab80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff93400e580 tx=0x7ff93400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9337fe700 1 -- 172.21.15.45:0/545745301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934004db0 con 0x7ff93c19a700 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9337fe700 1 -- 172.21.15.45:0/545745301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9340090d0 con 0x7ff93c19a700 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c067fe0 con 0x7ff93c19a700 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9337fe700 1 -- 172.21.15.45:0/545745301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9340045f0 con 0x7ff93c19a700 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.174+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c0684a0 con 0x7ff93c19a700 2023-09-23T09:46:13.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.175+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c067c10 con 0x7ff93c19a700 2023-09-23T09:46:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.176+0000 7ff9337fe700 1 -- 172.21.15.45:0/545745301 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7ff934009240 con 0x7ff93c19a700 2023-09-23T09:46:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.177+0000 7ff9337fe700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9280a1750 0x7ff9280a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.177+0000 7ff9337fe700 1 -- 172.21.15.45:0/545745301 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff934016070 con 0x7ff93c19a700 2023-09-23T09:46:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.177+0000 7ff93a59c700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9280a1750 0x7ff9280a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.177+0000 7ff93a59c700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9280a1750 0x7ff9280a3c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00b2e0 tx=0x7ff92c005960 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.181+0000 7ff9337fe700 1 -- 172.21.15.45:0/545745301 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff9340983b0 con 0x7ff93c19a700 2023-09-23T09:46:13.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.428+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff93c1970b0 con 0x7ff93c19a700 2023-09-23T09:46:13.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.429+0000 7ff9337fe700 1 -- 172.21.15.45:0/545745301 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+11312 (secure 0 0 0) 0x7ff93409b9d0 con 0x7ff93c19a700 2023-09-23T09:46:13.430 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:46:13.430 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:45:32.396059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25489":{"gid":25489,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2571161456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2571161456},{"type":"v1","addr":"172.21.15.123:6837","nonce":2571161456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:46:13.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9280a1750 msgr2=0x7ff9280a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9280a1750 0x7ff9280a3c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00b2e0 tx=0x7ff92c005960 comp rx=0 tx=0).stop 2023-09-23T09:46:13.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c19a700 msgr2=0x7ff93c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c19a700 0x7ff93c19ab80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff93400e580 tx=0x7ff93400e890 comp rx=0 tx=0).stop 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 shutdown_connections 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7ff9280a1750 0x7ff9280a3c10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93c19a700 0x7ff93c19ab80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff93c0ffed0 0x7ff93c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 --2- 172.21.15.45:0/545745301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff93c104ff0 0x7ff93c1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 >> 172.21.15.45:0/545745301 conn(0x7ff93c0fb730 msgr2=0x7ff93c1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 shutdown_connections 2023-09-23T09:46:13.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.431+0000 7ff9409ce700 1 -- 172.21.15.45:0/545745301 wait complete. 2023-09-23T09:46:13.434 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 61 2023-09-23T09:46:13.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T09:46:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 -- 172.21.15.45:0/733503777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5340fff00 msgr2=0x7fa5341002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 --2- 172.21.15.45:0/733503777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5340fff00 0x7fa5341002e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa524009990 tx=0x7fa524009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 -- 172.21.15.45:0/733503777 shutdown_connections 2023-09-23T09:46:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 --2- 172.21.15.45:0/733503777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa534105020 0x7fa53410cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 --2- 172.21.15.45:0/733503777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5340fff00 0x7fa5341002e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 --2- 172.21.15.45:0/733503777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5341008b0 0x7fa5341048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 -- 172.21.15.45:0/733503777 >> 172.21.15.45:0/733503777 conn(0x7fa5340fb780 msgr2=0x7fa5340fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 -- 172.21.15.45:0/733503777 shutdown_connections 2023-09-23T09:46:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.570+0000 7fa538bd4700 1 -- 172.21.15.45:0/733503777 wait complete. 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.571+0000 7fa538bd4700 1 Processor -- start 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.571+0000 7fa538bd4700 1 -- start start 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa538bd4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5340fff00 0x7fa534195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa538bd4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa534105020 0x7fa5341962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa538bd4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53419a710 0x7fa53419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa538bd4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53419b190 con 0x7fa534105020 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa538bd4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53419b300 con 0x7fa5340fff00 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa538bd4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53419b4c0 con 0x7fa53419a710 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa534105020 0x7fa5341962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa53259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5340fff00 0x7fa534195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa53259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5340fff00 0x7fa534195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.45:58784/0 (socket says 172.21.15.45:58784) 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa534105020 0x7fa5341962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37436/0 (socket says 172.21.15.45:37436) 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa53259c700 1 -- 172.21.15.45:0/2138364331 learned_addr learned my addr 172.21.15.45:0/2138364331 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa532d9d700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53419a710 0x7fa53419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 -- 172.21.15.45:0/2138364331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53419a710 msgr2=0x7fa53419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53419a710 0x7fa53419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 -- 172.21.15.45:0/2138364331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5340fff00 msgr2=0x7fa534195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5340fff00 0x7fa534195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 -- 172.21.15.45:0/2138364331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240095f0 con 0x7fa534105020 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa53259c700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5340fff00 0x7fa534195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa532d9d700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53419a710 0x7fa53419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T09:46:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.572+0000 7fa531d9b700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa534105020 0x7fa5341962b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9c0 tx=0x7fa52800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.573+0000 7fa5237fe700 1 -- 172.21.15.45:0/2138364331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52800c9c0 con 0x7fa534105020 2023-09-23T09:46:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.573+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5341a2a20 con 0x7fa534105020 2023-09-23T09:46:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.573+0000 7fa5237fe700 1 -- 172.21.15.45:0/2138364331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa528010430 con 0x7fa534105020 2023-09-23T09:46:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.573+0000 7fa5237fe700 1 -- 172.21.15.45:0/2138364331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528018680 con 0x7fa534105020 2023-09-23T09:46:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.573+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5341a2f50 con 0x7fa534105020 2023-09-23T09:46:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.574+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa534067380 con 0x7fa534105020 2023-09-23T09:46:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.575+0000 7fa5237fe700 1 -- 172.21.15.45:0/2138364331 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fa52800cb20 con 0x7fa534105020 2023-09-23T09:46:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.575+0000 7fa5237fe700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa51c0a1750 0x7fa51c0a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.576+0000 7fa53259c700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa51c0a1750 0x7fa51c0a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.576+0000 7fa5237fe700 1 -- 172.21.15.45:0/2138364331 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa528014070 con 0x7fa534105020 2023-09-23T09:46:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.576+0000 7fa53259c700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa51c0a1750 0x7fa51c0a3c10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa524006010 tx=0x7fa52400b630 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.579+0000 7fa5237fe700 1 -- 172.21.15.45:0/2138364331 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa528096980 con 0x7fa534105020 2023-09-23T09:46:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.825+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa534100b00 con 0x7fa534105020 2023-09-23T09:46:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.825+0000 7fa5237fe700 1 -- 172.21.15.45:0/2138364331 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+11312 (secure 0 0 0) 0x7fa534100b00 con 0x7fa534105020 2023-09-23T09:46:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:46:13.827 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/427947541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":427947541},{"type":"v1","addr":"172.21.15.45:6835","nonce":427947541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1158485570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1158485570},{"type":"v1","addr":"172.21.15.45:6837","nonce":1158485570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/581280967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":581280967},{"type":"v1","addr":"172.21.15.45:6839","nonce":581280967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/4027335319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":4027335319},{"type":"v1","addr":"172.21.15.45:6841","nonce":4027335319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/782227022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":782227022},{"type":"v1","addr":"172.21.15.123:6833","nonce":782227022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3376968618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3376968618},{"type":"v1","addr":"172.21.15.98:6833","nonce":3376968618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T09:27:50.073600+0000","modified":"2023-09-23T09:45:32.396059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24530,"mds_1":24500,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/3506316120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3506316120},{"type":"v1","addr":"172.21.15.98:6835","nonce":3506316120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3236511886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3236511886},{"type":"v1","addr":"172.21.15.123:6835","nonce":3236511886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/4054887711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4054887711},{"type":"v1","addr":"172.21.15.123:6839","nonce":4054887711}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/767375327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":767375327},{"type":"v1","addr":"172.21.15.98:6837","nonce":767375327}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2348516337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2348516337},{"type":"v1","addr":"172.21.15.98:6839","nonce":2348516337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25489":{"gid":25489,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/2571161456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2571161456},{"type":"v1","addr":"172.21.15.123:6837","nonce":2571161456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T09:46:13.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.827+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa51c0a1750 msgr2=0x7fa51c0a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.827+0000 7fa538bd4700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa51c0a1750 0x7fa51c0a3c10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa524006010 tx=0x7fa52400b630 comp rx=0 tx=0).stop 2023-09-23T09:46:13.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.827+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa534105020 msgr2=0x7fa5341962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.827+0000 7fa538bd4700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa534105020 0x7fa5341962b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9c0 tx=0x7fa52800ecd0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 shutdown_connections 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53419a710 0x7fa53419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5340fff00 0x7fa534195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa51c0a1750 0x7fa51c0a3c10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 --2- 172.21.15.45:0/2138364331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa534105020 0x7fa5341962b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 >> 172.21.15.45:0/2138364331 conn(0x7fa5340fb780 msgr2=0x7fa534103930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 shutdown_connections 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.828+0000 7fa538bd4700 1 -- 172.21.15.45:0/2138364331 wait complete. 2023-09-23T09:46:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 61 2023-09-23T09:46:13.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T09:46:13.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 -- 172.21.15.45:0/536400971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 msgr2=0x7fccfc1050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 --2- 172.21.15.45:0/536400971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc1050d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fccec0099c0 tx=0x7fccec009cd0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 -- 172.21.15.45:0/536400971 shutdown_connections 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 --2- 172.21.15.45:0/536400971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccfc1010d0 0x7fccfc10cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 --2- 172.21.15.45:0/536400971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc1050d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 --2- 172.21.15.45:0/536400971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccfc100710 0x7fccfc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 -- 172.21.15.45:0/536400971 >> 172.21.15.45:0/536400971 conn(0x7fccfc076110 msgr2=0x7fccfc076520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 -- 172.21.15.45:0/536400971 shutdown_connections 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.967+0000 7fcd016f6700 1 -- 172.21.15.45:0/536400971 wait complete. 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 Processor -- start 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 -- start start 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccfc100710 0x7fccfc198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccfc1010d0 0x7fccfc199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc19dfd0 con 0x7fccfc104cf0 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc19e140 con 0x7fccfc1010d0 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fcd016f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc19e300 con 0x7fccfc100710 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37452/0 (socket says 172.21.15.45:37452) 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccfc100710 0x7fccfc198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 -- 172.21.15.45:0/17764915 learned_addr learned my addr 172.21.15.45:0/17764915 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccf3fff700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccfc1010d0 0x7fccfc199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 -- 172.21.15.45:0/17764915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccfc100710 msgr2=0x7fccfc198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccfc100710 0x7fccfc198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 -- 172.21.15.45:0/17764915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccfc1010d0 msgr2=0x7fccfc199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccfc1010d0 0x7fccfc199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 -- 172.21.15.45:0/17764915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0095f0 con 0x7fccfc104cf0 2023-09-23T09:46:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.968+0000 7fccfb7fe700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc19d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ef30 tx=0x7fccf400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.969+0000 7fccf8ff9700 1 -- 172.21.15.45:0/17764915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4018070 con 0x7fccfc104cf0 2023-09-23T09:46:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.969+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc19e580 con 0x7fccfc104cf0 2023-09-23T09:46:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.969+0000 7fccf8ff9700 1 -- 172.21.15.45:0/17764915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf400f040 con 0x7fccfc104cf0 2023-09-23T09:46:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.969+0000 7fccf8ff9700 1 -- 172.21.15.45:0/17764915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400cd00 con 0x7fccfc104cf0 2023-09-23T09:46:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.969+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc1a5e20 con 0x7fccfc104cf0 2023-09-23T09:46:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.970+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccfc068060 con 0x7fccfc104cf0 2023-09-23T09:46:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.971+0000 7fccf8ff9700 1 -- 172.21.15.45:0/17764915 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fccf4003a00 con 0x7fccfc104cf0 2023-09-23T09:46:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.971+0000 7fccf8ff9700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcce80a1750 0x7fcce80a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.972+0000 7fccfaffd700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcce80a1750 0x7fcce80a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.972+0000 7fccf8ff9700 1 -- 172.21.15.45:0/17764915 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fccf40cce40 con 0x7fccfc104cf0 2023-09-23T09:46:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.972+0000 7fccfaffd700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcce80a1750 0x7fcce80a3c10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccec009990 tx=0x7fccec005fb0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:13.976+0000 7fccf8ff9700 1 -- 172.21.15.45:0/17764915 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fccf4096110 con 0x7fccfc104cf0 2023-09-23T09:46:14.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.182+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccfc10fe30 con 0x7fccfc104cf0 2023-09-23T09:46:14.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.183+0000 7fccf8ff9700 1 -- 172.21.15.45:0/17764915 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+23078 (secure 0 0 0) 0x7fccf4099730 con 0x7fccfc104cf0 2023-09-23T09:46:14.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-09-23T09:46:14.185 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":108,"fsid":"654e5bfa-59f2-11ee-8db4-212e2dc638e7","created":"2023-09-23T09:20:53.563973+0000","modified":"2023-09-23T09:46:04.972961+0000","last_up_change":"2023-09-23T09:27:08.761451+0000","last_in_change":"2023-09-23T09:26:51.290313+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T09:24:00.782778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T09:27:48.176304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T09:27:49.453516+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8136dd58-335a-4231-97c9-99ae3b5e6f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6803","nonce":1500595998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6805","nonce":1500595998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6809","nonce":1500595998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1500595998},{"type":"v1","addr":"172.21.15.45:6807","nonce":1500595998}]},"public_addr":"172.21.15.45:6803/1500595998","cluster_addr":"172.21.15.45:6805/1500595998","heartbeat_back_addr":"172.21.15.45:6809/1500595998","heartbeat_front_addr":"172.21.15.45:6807/1500595998","state":["exists","up"]},{"osd":1,"uuid":"934fb488-aa5c-4a75-acf1-4f1a42c9a428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6801","nonce":2863958609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6803","nonce":2863958609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6807","nonce":2863958609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2863958609},{"type":"v1","addr":"172.21.15.98:6805","nonce":2863958609}]},"public_addr":"172.21.15.98:6801/2863958609","cluster_addr":"172.21.15.98:6803/2863958609","heartbeat_back_addr":"172.21.15.98:6807/2863958609","heartbeat_front_addr":"172.21.15.98:6805/2863958609","state":["exists","up"]},{"osd":2,"uuid":"752d7ddf-4714-4f48-b2ce-5ad5e04e6426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6801","nonce":976297925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6803","nonce":976297925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6807","nonce":976297925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":976297925},{"type":"v1","addr":"172.21.15.123:6805","nonce":976297925}]},"public_addr":"172.21.15.123:6801/976297925","cluster_addr":"172.21.15.123:6803/976297925","heartbeat_back_addr":"172.21.15.123:6807/976297925","heartbeat_front_addr":"172.21.15.123:6805/976297925","state":["exists","up"]},{"osd":3,"uuid":"20e42b6b-8c74-4414-a64d-e76bb61de1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6811","nonce":3800235078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6813","nonce":3800235078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6817","nonce":3800235078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3800235078},{"type":"v1","addr":"172.21.15.45:6815","nonce":3800235078}]},"public_addr":"172.21.15.45:6811/3800235078","cluster_addr":"172.21.15.45:6813/3800235078","heartbeat_back_addr":"172.21.15.45:6817/3800235078","heartbeat_front_addr":"172.21.15.45:6815/3800235078","state":["exists","up"]},{"osd":4,"uuid":"ecfe4f49-dd41-46e6-b948-c396c14fe25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6809","nonce":4058003666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6811","nonce":4058003666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6815","nonce":4058003666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":4058003666},{"type":"v1","addr":"172.21.15.98:6813","nonce":4058003666}]},"public_addr":"172.21.15.98:6809/4058003666","cluster_addr":"172.21.15.98:6811/4058003666","heartbeat_back_addr":"172.21.15.98:6815/4058003666","heartbeat_front_addr":"172.21.15.98:6813/4058003666","state":["exists","up"]},{"osd":5,"uuid":"1bc31bcd-a03e-4e5d-a660-82eb8c8dee2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6809","nonce":3175137003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6811","nonce":3175137003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6815","nonce":3175137003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3175137003},{"type":"v1","addr":"172.21.15.123:6813","nonce":3175137003}]},"public_addr":"172.21.15.123:6809/3175137003","cluster_addr":"172.21.15.123:6811/3175137003","heartbeat_back_addr":"172.21.15.123:6815/3175137003","heartbeat_front_addr":"172.21.15.123:6813/3175137003","state":["exists","up"]},{"osd":6,"uuid":"ce5cc090-fa1f-4813-8950-970817a59b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6819","nonce":668839971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6821","nonce":668839971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6825","nonce":668839971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":668839971},{"type":"v1","addr":"172.21.15.45:6823","nonce":668839971}]},"public_addr":"172.21.15.45:6819/668839971","cluster_addr":"172.21.15.45:6821/668839971","heartbeat_back_addr":"172.21.15.45:6825/668839971","heartbeat_front_addr":"172.21.15.45:6823/668839971","state":["exists","up"]},{"osd":7,"uuid":"f045102a-f2d2-4e94-a7ac-ff19e15c4126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6817","nonce":3609791505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6819","nonce":3609791505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6823","nonce":3609791505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3609791505},{"type":"v1","addr":"172.21.15.98:6821","nonce":3609791505}]},"public_addr":"172.21.15.98:6817/3609791505","cluster_addr":"172.21.15.98:6819/3609791505","heartbeat_back_addr":"172.21.15.98:6823/3609791505","heartbeat_front_addr":"172.21.15.98:6821/3609791505","state":["exists","up"]},{"osd":8,"uuid":"7fc7b9f6-15f6-4025-8728-353e53602172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6817","nonce":101690512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6819","nonce":101690512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6823","nonce":101690512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":101690512},{"type":"v1","addr":"172.21.15.123:6821","nonce":101690512}]},"public_addr":"172.21.15.123:6817/101690512","cluster_addr":"172.21.15.123:6819/101690512","heartbeat_back_addr":"172.21.15.123:6823/101690512","heartbeat_front_addr":"172.21.15.123:6821/101690512","state":["exists","up"]},{"osd":9,"uuid":"3029adaf-0e5e-4ab8-8b6d-d5376029092c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6827","nonce":2888091166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6829","nonce":2888091166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6833","nonce":2888091166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2888091166},{"type":"v1","addr":"172.21.15.45:6831","nonce":2888091166}]},"public_addr":"172.21.15.45:6827/2888091166","cluster_addr":"172.21.15.45:6829/2888091166","heartbeat_back_addr":"172.21.15.45:6833/2888091166","heartbeat_front_addr":"172.21.15.45:6831/2888091166","state":["exists","up"]},{"osd":10,"uuid":"30007304-e7de-46d9-8b01-c03dfc7c22a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6825","nonce":2174091835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6827","nonce":2174091835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6831","nonce":2174091835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2174091835},{"type":"v1","addr":"172.21.15.98:6829","nonce":2174091835}]},"public_addr":"172.21.15.98:6825/2174091835","cluster_addr":"172.21.15.98:6827/2174091835","heartbeat_back_addr":"172.21.15.98:6831/2174091835","heartbeat_front_addr":"172.21.15.98:6829/2174091835","state":["exists","up"]},{"osd":11,"uuid":"fbebaf92-0e82-4758-b581-d15dc738abec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6825","nonce":523003313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6827","nonce":523003313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6831","nonce":523003313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":523003313},{"type":"v1","addr":"172.21.15.123:6829","nonce":523003313}]},"public_addr":"172.21.15.123:6825/523003313","cluster_addr":"172.21.15.123:6827/523003313","heartbeat_back_addr":"172.21.15.123:6831/523003313","heartbeat_front_addr":"172.21.15.123:6829/523003313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:17.209398+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:38.040926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:23:58.538313+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:19.352922+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:24:40.467711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:01.412371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:22.084826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:25:43.243189+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:04.479092+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:25.191883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:26:46.318230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T09:27:07.167954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/230875070":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/2302872888":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3011609987":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/258118400":"2023-09-24T09:21:28.141138+0000","172.21.15.45:0/3656954721":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/3269510756":"2023-09-24T09:29:25.699095+0000","172.21.15.45:6801/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/4072483061":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6800/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:6801/1791488577":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/690218910":"2023-09-24T09:21:54.248141+0000","172.21.15.45:0/2324483632":"2023-09-24T09:29:25.699095+0000","172.21.15.45:6800/944196506":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/65945615":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/3832887419":"2023-09-24T09:21:28.141138+0000","172.21.15.45:6801/944196506":"2023-09-24T09:29:25.699095+0000","172.21.15.45:0/1970790569":"2023-09-24T09:29:25.699095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T09:46:14.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcce80a1750 msgr2=0x7fcce80a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcce80a1750 0x7fcce80a3c10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccec009990 tx=0x7fccec005fb0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 msgr2=0x7fccfc19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc19d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ef30 tx=0x7fccf400c3a0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 shutdown_connections 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fcce80a1750 0x7fcce80a3c10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccfc104cf0 0x7fccfc19d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccfc100710 0x7fccfc198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 --2- 172.21.15.45:0/17764915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccfc1010d0 0x7fccfc199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 >> 172.21.15.45:0/17764915 conn(0x7fccfc076110 msgr2=0x7fccfc0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 shutdown_connections 2023-09-23T09:46:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.186+0000 7fcd016f6700 1 -- 172.21.15.45:0/17764915 wait complete. 2023-09-23T09:46:14.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:14 smithi045 ceph-mon[98627]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 33 op/s 2023-09-23T09:46:14.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:14 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/545745301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:46:14.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:14 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2138364331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:46:14.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T09:46:14.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-mon[102386]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 33 op/s 2023-09-23T09:46:14.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/545745301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:46:14.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2138364331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:46:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 -- 172.21.15.45:0/2140270723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc0ff490 msgr2=0x7f2cfc0ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 --2- 172.21.15.45:0/2140270723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc0ff490 0x7f2cfc0ff870 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009990 tx=0x7f2cec009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 -- 172.21.15.45:0/2140270723 shutdown_connections 2023-09-23T09:46:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 --2- 172.21.15.45:0/2140270723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cfc106190 0x7f2cfc10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 --2- 172.21.15.45:0/2140270723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc0ff490 0x7f2cfc0ff870 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 --2- 172.21.15.45:0/2140270723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cfc0ffe40 0x7f2cfc105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 -- 172.21.15.45:0/2140270723 >> 172.21.15.45:0/2140270723 conn(0x7f2cfc077290 msgr2=0x7f2cfc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 -- 172.21.15.45:0/2140270723 shutdown_connections 2023-09-23T09:46:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 -- 172.21.15.45:0/2140270723 wait complete. 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.326+0000 7f2d0213e700 1 Processor -- start 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2d0213e700 1 -- start start 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2d0213e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cfc0ff490 0x7f2cfc197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2d0213e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc106190 0x7f2cfc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2d0213e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cfc19b9f0 0x7f2cfc19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2d0213e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc19c470 con 0x7f2cfc106190 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2d0213e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc19c5e0 con 0x7f2cfc19b9f0 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2d0213e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc19c7a0 con 0x7f2cfc0ff490 2023-09-23T09:46:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc106190 0x7f2cfc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc106190 0x7f2cfc197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37464/0 (socket says 172.21.15.45:37464) 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 -- 172.21.15.45:0/1558645930 learned_addr learned my addr 172.21.15.45:0/1558645930 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfb7fe700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cfc0ff490 0x7f2cfc197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfbfff700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cfc19b9f0 0x7f2cfc19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 -- 172.21.15.45:0/1558645930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cfc0ff490 msgr2=0x7f2cfc197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cfc0ff490 0x7f2cfc197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 -- 172.21.15.45:0/1558645930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cfc19b9f0 msgr2=0x7f2cfc19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cfc19b9f0 0x7f2cfc19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 -- 172.21.15.45:0/1558645930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4009c40 con 0x7f2cfc106190 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cfaffd700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc106190 0x7f2cfc197590 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000e970 tx=0x7f2cf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.327+0000 7f2cf8ff9700 1 -- 172.21.15.45:0/1558645930 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0009790 con 0x7f2cfc106190 2023-09-23T09:46:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.328+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec0095f0 con 0x7f2cfc106190 2023-09-23T09:46:14.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.328+0000 7f2cf8ff9700 1 -- 172.21.15.45:0/1558645930 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf000c7c0 con 0x7f2cfc106190 2023-09-23T09:46:14.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.328+0000 7f2cf8ff9700 1 -- 172.21.15.45:0/1558645930 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0005460 con 0x7f2cfc106190 2023-09-23T09:46:14.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.328+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc1a4050 con 0x7f2cfc106190 2023-09-23T09:46:14.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.328+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc1093d0 con 0x7f2cfc106190 2023-09-23T09:46:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.330+0000 7f2cf8ff9700 1 -- 172.21.15.45:0/1558645930 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f2cf000c930 con 0x7f2cfc106190 2023-09-23T09:46:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.330+0000 7f2cf8ff9700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2ce40a1710 0x7f2ce40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.330+0000 7f2cf8ff9700 1 -- 172.21.15.45:0/1558645930 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2cf0014070 con 0x7f2cfc106190 2023-09-23T09:46:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.330+0000 7f2cfb7fe700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2ce40a1710 0x7f2ce40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.331+0000 7f2cfb7fe700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2ce40a1710 0x7f2ce40a3bd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00b2e0 tx=0x7f2cec005fd0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:14.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.334+0000 7f2cf8ff9700 1 -- 172.21.15.45:0/1558645930 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2cf00968d0 con 0x7f2cfc106190 2023-09-23T09:46:14.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-mon[103577]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 33 op/s 2023-09-23T09:46:14.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/545745301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:46:14.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2138364331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T09:46:14.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.586+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2cfc0ffe40 con 0x7f2cfc106190 2023-09-23T09:46:14.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.588+0000 7f2cf8ff9700 1 -- 172.21.15.45:0/1558645930 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2cf0099ef0 con 0x7f2cfc106190 2023-09-23T09:46:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.591+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2ce40a1710 msgr2=0x7f2ce40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.591+0000 7f2d0213e700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2ce40a1710 0x7f2ce40a3bd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00b2e0 tx=0x7f2cec005fd0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.591+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc106190 msgr2=0x7f2cfc197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.591+0000 7f2d0213e700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc106190 0x7f2cfc197590 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000e970 tx=0x7f2cf000ec80 comp rx=0 tx=0).stop 2023-09-23T09:46:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 shutdown_connections 2023-09-23T09:46:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cfc19b9f0 0x7f2cfc19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cfc0ff490 0x7f2cfc197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f2ce40a1710 0x7f2ce40a3bd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 --2- 172.21.15.45:0/1558645930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cfc106190 0x7f2cfc197590 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 >> 172.21.15.45:0/1558645930 conn(0x7f2cfc077290 msgr2=0x7f2cfc105860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 shutdown_connections 2023-09-23T09:46:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.592+0000 7f2d0213e700 1 -- 172.21.15.45:0/1558645930 wait complete. 2023-09-23T09:46:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T09:46:14.606 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T09:46:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 -- 172.21.15.45:0/3547985089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d0102850 msgr2=0x7fa7d010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/3547985089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d0102850 0x7fa7d010ad30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009990 tx=0x7fa7c0009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 -- 172.21.15.45:0/3547985089 shutdown_connections 2023-09-23T09:46:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/3547985089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7d010b350 0x7fa7d010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/3547985089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d0101f30 0x7fa7d0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/3547985089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d0102850 0x7fa7d010ad30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 -- 172.21.15.45:0/3547985089 >> 172.21.15.45:0/3547985089 conn(0x7fa7d00fb770 msgr2=0x7fa7d00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 -- 172.21.15.45:0/3547985089 shutdown_connections 2023-09-23T09:46:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 -- 172.21.15.45:0/3547985089 wait complete. 2023-09-23T09:46:14.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.736+0000 7fa7d72f7700 1 Processor -- start 2023-09-23T09:46:14.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d72f7700 1 -- start start 2023-09-23T09:46:14.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d72f7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d0101f30 0x7fa7d0196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d72f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7d0102850 0x7fa7d0197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d72f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d010b350 0x7fa7d019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d72f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d019bec0 con 0x7fa7d010b350 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d72f7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d019c030 con 0x7fa7d0101f30 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d72f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d019c1f0 con 0x7fa7d0102850 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d010b350 0x7fa7d019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d010b350 0x7fa7d019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37480/0 (socket says 172.21.15.45:37480) 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 -- 172.21.15.45:0/107809425 learned_addr learned my addr 172.21.15.45:0/107809425 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d4892700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7d0102850 0x7fa7d0197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5093700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d0101f30 0x7fa7d0196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 -- 172.21.15.45:0/107809425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7d0102850 msgr2=0x7fa7d0197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7d0102850 0x7fa7d0197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 -- 172.21.15.45:0/107809425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d0101f30 msgr2=0x7fa7d0196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d0101f30 0x7fa7d0196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 -- 172.21.15.45:0/107809425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00095f0 con 0x7fa7d010b350 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5093700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d0101f30 0x7fa7d0196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7d5894700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d010b350 0x7fa7d019b880 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8007ee0 tx=0x7fa7c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.737+0000 7fa7c67fc700 1 -- 172.21.15.45:0/107809425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c800ddd0 con 0x7fa7d010b350 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.738+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d019c4d0 con 0x7fa7d010b350 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.738+0000 7fa7c67fc700 1 -- 172.21.15.45:0/107809425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7c800f040 con 0x7fa7d010b350 2023-09-23T09:46:14.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.738+0000 7fa7c67fc700 1 -- 172.21.15.45:0/107809425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80048e0 con 0x7fa7d010b350 2023-09-23T09:46:14.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.738+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d01a3d10 con 0x7fa7d010b350 2023-09-23T09:46:14.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.738+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d0106420 con 0x7fa7d010b350 2023-09-23T09:46:14.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.740+0000 7fa7c67fc700 1 -- 172.21.15.45:0/107809425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fa7c8003a00 con 0x7fa7d010b350 2023-09-23T09:46:14.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.740+0000 7fa7c67fc700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7bc0a1710 0x7fa7bc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:14.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.740+0000 7fa7d5093700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7bc0a1710 0x7fa7bc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:14.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.740+0000 7fa7c67fc700 1 -- 172.21.15.45:0/107809425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8843+0+0 (secure 0 0 0) 0x7fa7c80cd530 con 0x7fa7d010b350 2023-09-23T09:46:14.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.741+0000 7fa7d5093700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7bc0a1710 0x7fa7bc0a3bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00bc10 tx=0x7fa7cc00b5f0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:14.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:14.744+0000 7fa7c67fc700 1 -- 172.21.15.45:0/107809425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa7c80966a0 con 0x7fa7d010b350 2023-09-23T09:46:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.001+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa7d0110cb0 con 0x7fa7d010b350 2023-09-23T09:46:15.040 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:14 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-a[98603]: 2023-09-23T09:46:14.587+0000 7fd8ce055700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T09:46:15.041 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b[137507]: ignoring --setuser ceph since I am not root 2023-09-23T09:46:15.041 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b[137507]: ignoring --setgroup ceph since I am not root 2023-09-23T09:46:15.041 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-mds[137531]: -- 172.21.15.98:0/250624080 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fff39831e0 con 0x55fff3958800 2023-09-23T09:46:15.041 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b[137507]: starting mds.b at 2023-09-23T09:46:15.041 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-e[136820]: ignoring --setuser ceph since I am not root 2023-09-23T09:46:15.042 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-e[136820]: ignoring --setgroup ceph since I am not root 2023-09-23T09:46:15.042 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-e[136820]: starting mds.e at 2023-09-23T09:46:15.042 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-h[137166]: ignoring --setuser ceph since I am not root 2023-09-23T09:46:15.042 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-h[137166]: ignoring --setgroup ceph since I am not root 2023-09-23T09:46:15.042 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-mds[137190]: -- 172.21.15.98:0/679851083 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561205ce11e0 con 0x561205cb7000 2023-09-23T09:46:15.042 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:46:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-h[137166]: starting mds.h at 2023-09-23T09:46:15.043 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[143197]: ignoring --setuser ceph since I am not root 2023-09-23T09:46:15.043 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[143197]: ignoring --setgroup ceph since I am not root 2023-09-23T09:46:15.043 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-mds[143222]: -- 172.21.15.123:0/2860785852 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569d57811e0 con 0x5569d5756c00 2023-09-23T09:46:15.043 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[143197]: starting mds.c at 2023-09-23T09:46:15.043 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-i[139874]: ignoring --setuser ceph since I am not root 2023-09-23T09:46:15.043 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-i[139874]: ignoring --setgroup ceph since I am not root 2023-09-23T09:46:15.043 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-mds[139899]: -- 172.21.15.123:0/17576604 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f100f9b1e0 con 0x55f100f70800 2023-09-23T09:46:15.044 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-i[139874]: starting mds.i at 2023-09-23T09:46:15.044 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-f[139183]: ignoring --setuser ceph since I am not root 2023-09-23T09:46:15.044 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-f[139183]: ignoring --setgroup ceph since I am not root 2023-09-23T09:46:15.044 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-mds[139207]: -- 172.21.15.123:0/2709541305 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b3ab84f1e0 con 0x55b3ab824800 2023-09-23T09:46:15.044 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:46:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-f[139183]: starting mds.f at 2023-09-23T09:46:15.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/17764915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:46:15.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1558645930' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T09:46:15.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-23T09:46:15.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T09:46:15.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1558645930' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-23T09:46:15.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: osdmap e109: 12 total, 12 up, 12 in 2023-09-23T09:46:15.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-23T09:46:15.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:15 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/107809425' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:46:15.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/17764915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:46:15.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1558645930' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T09:46:15.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-23T09:46:15.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T09:46:15.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1558645930' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-23T09:46:15.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: osdmap e109: 12 total, 12 up, 12 in 2023-09-23T09:46:15.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-23T09:46:15.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:15 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/107809425' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:46:15.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/17764915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T09:46:15.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1558645930' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T09:46:15.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-23T09:46:15.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T09:46:15.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1558645930' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-23T09:46:15.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: osdmap e109: 12 total, 12 up, 12 in 2023-09-23T09:46:15.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-23T09:46:15.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:15 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/107809425' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T09:46:15.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.590+0000 7fa7c67fc700 1 -- 172.21.15.45:0/107809425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa7c8099cc0 con 0x7fa7d010b350 2023-09-23T09:46:15.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7bc0a1710 msgr2=0x7fa7bc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7bc0a1710 0x7fa7bc0a3bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00bc10 tx=0x7fa7cc00b5f0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d010b350 msgr2=0x7fa7d019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d010b350 0x7fa7d019b880 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8007ee0 tx=0x7fa7c800d3b0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 shutdown_connections 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fa7bc0a1710 0x7fa7bc0a3bd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d010b350 0x7fa7d019b880 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d0101f30 0x7fa7d0196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 --2- 172.21.15.45:0/107809425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7d0102850 0x7fa7d0197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 >> 172.21.15.45:0/107809425 conn(0x7fa7d00fb770 msgr2=0x7fa7d00fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 shutdown_connections 2023-09-23T09:46:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.594+0000 7fa7d72f7700 1 -- 172.21.15.45:0/107809425 wait complete. 2023-09-23T09:46:15.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T09:46:15.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1145553428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0101f30 msgr2=0x7fcaa0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:15.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1145553428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0101f30 0x7fcaa0102310 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fca90009990 tx=0x7fca90009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1145553428 shutdown_connections 2023-09-23T09:46:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1145553428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa010b350 0x7fcaa010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1145553428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0101f30 0x7fcaa0102310 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1145553428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0102850 0x7fcaa010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1145553428 >> 172.21.15.45:0/1145553428 conn(0x7fcaa00fb770 msgr2=0x7fcaa00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1145553428 shutdown_connections 2023-09-23T09:46:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1145553428 wait complete. 2023-09-23T09:46:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.750+0000 7fcaa5f70700 1 Processor -- start 2023-09-23T09:46:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fcaa5f70700 1 -- start start 2023-09-23T09:46:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fcaa5f70700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0101f30 0x7fcaa0196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fcaa5f70700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa0102850 0x7fcaa01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fcaa5f70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa010b350 0x7fcaa019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fcaa5f70700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa019be80 con 0x7fcaa010b350 2023-09-23T09:46:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fcaa5f70700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa019bff0 con 0x7fcaa0102850 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fcaa5f70700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa019c1b0 con 0x7fcaa0101f30 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa010b350 0x7fcaa019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa010b350 0x7fcaa019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37492/0 (socket says 172.21.15.45:37492) 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 -- 172.21.15.45:0/1645675868 learned_addr learned my addr 172.21.15.45:0/1645675868 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9f7fe700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0101f30 0x7fcaa0196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9effd700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa0102850 0x7fcaa01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 -- 172.21.15.45:0/1645675868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0101f30 msgr2=0x7fcaa0196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0101f30 0x7fcaa0196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 -- 172.21.15.45:0/1645675868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa0102850 msgr2=0x7fcaa01973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa0102850 0x7fcaa01973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 -- 172.21.15.45:0/1645675868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca900095f0 con 0x7fcaa010b350 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9effd700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa0102850 0x7fcaa01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9ffff700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa010b350 0x7fcaa019b840 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fca9800ef30 tx=0x7fca9800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.751+0000 7fca9cff9700 1 -- 172.21.15.45:0/1645675868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98018070 con 0x7fcaa010b350 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.752+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa019c430 con 0x7fcaa010b350 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.752+0000 7fca9cff9700 1 -- 172.21.15.45:0/1645675868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca9800f040 con 0x7fcaa010b350 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.752+0000 7fca9cff9700 1 -- 172.21.15.45:0/1645675868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800cdb0 con 0x7fcaa010b350 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.752+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa01a3cd0 con 0x7fcaa010b350 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.752+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa01063e0 con 0x7fcaa010b350 2023-09-23T09:46:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.754+0000 7fca9cff9700 1 -- 172.21.15.45:0/1645675868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7fca98003a00 con 0x7fcaa010b350 2023-09-23T09:46:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.754+0000 7fca9cff9700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fca880a1710 0x7fca880a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.754+0000 7fca9cff9700 1 -- 172.21.15.45:0/1645675868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8843+0+0 (secure 0 0 0) 0x7fca980cda00 con 0x7fcaa010b350 2023-09-23T09:46:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.754+0000 7fca9f7fe700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fca880a1710 0x7fca880a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.755+0000 7fca9f7fe700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fca880a1710 0x7fca880a3bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b2e0 tx=0x7fca90005fd0 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.758+0000 7fca9cff9700 1 -- 172.21.15.45:0/1645675868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fca98096c00 con 0x7fcaa010b350 2023-09-23T09:46:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:15.962+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcaa0067ba0 con 0x7fcaa010b350 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 65 KiB/s wr, 31 op/s 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: Cluster is now healthy 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/107809425' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6836/1797080184,v1:172.21.15.98:6839/1797080184] up:boot 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6835/354596143,v1:172.21.15.98:6838/354596143] up:boot 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6836/305405229,v1:172.21.15.123:6838/305405229] up:boot 2023-09-23T09:46:16.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6834/3694602202,v1:172.21.15.123:6837/3694602202] up:boot 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.123:6835/2184643421,v1:172.21.15.123:6839/2184643421] up:boot 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: mds.? [v2:172.21.15.98:6834/4156801122,v1:172.21.15.98:6837/4156801122] up:boot 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: fsmap 12 up:standby 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: osdmap e110: 12 total, 12 up, 12 in 2023-09-23T09:46:16.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:16 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1645675868' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T09:46:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 65 KiB/s wr, 31 op/s 2023-09-23T09:46:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-23T09:46:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-23T09:46:16.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: Cluster is now healthy 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/107809425' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6836/1797080184,v1:172.21.15.98:6839/1797080184] up:boot 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6835/354596143,v1:172.21.15.98:6838/354596143] up:boot 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6836/305405229,v1:172.21.15.123:6838/305405229] up:boot 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6834/3694602202,v1:172.21.15.123:6837/3694602202] up:boot 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.123:6835/2184643421,v1:172.21.15.123:6839/2184643421] up:boot 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: mds.? [v2:172.21.15.98:6834/4156801122,v1:172.21.15.98:6837/4156801122] up:boot 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: fsmap 12 up:standby 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:46:16.339 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:46:16.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:46:16.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:46:16.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: osdmap e110: 12 total, 12 up, 12 in 2023-09-23T09:46:16.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:16 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1645675868' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T09:46:16.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 65 KiB/s wr, 31 op/s 2023-09-23T09:46:16.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-23T09:46:16.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-23T09:46:16.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: Cluster is now healthy 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/107809425' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6836/1797080184,v1:172.21.15.98:6839/1797080184] up:boot 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6835/354596143,v1:172.21.15.98:6838/354596143] up:boot 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6836/305405229,v1:172.21.15.123:6838/305405229] up:boot 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6834/3694602202,v1:172.21.15.123:6837/3694602202] up:boot 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.123:6835/2184643421,v1:172.21.15.123:6839/2184643421] up:boot 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: mds.? [v2:172.21.15.98:6834/4156801122,v1:172.21.15.98:6837/4156801122] up:boot 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: fsmap 12 up:standby 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T09:46:16.544 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T09:46:16.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T09:46:16.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T09:46:16.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='mgr.15405 172.21.15.45:0/1290693084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T09:46:16.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: osdmap e110: 12 total, 12 up, 12 in 2023-09-23T09:46:16.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:16 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1645675868' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T09:46:16.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.908+0000 7fca9cff9700 1 -- 172.21.15.45:0/1645675868 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7fca9809a220 con 0x7fcaa010b350 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fca880a1710 msgr2=0x7fca880a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fca880a1710 0x7fca880a3bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b2e0 tx=0x7fca90005fd0 comp rx=0 tx=0).stop 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa010b350 msgr2=0x7fcaa019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa010b350 0x7fcaa019b840 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fca9800ef30 tx=0x7fca9800c3a0 comp rx=0 tx=0).stop 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 shutdown_connections 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7fca880a1710 0x7fca880a3bd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa010b350 0x7fcaa019b840 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0101f30 0x7fcaa0196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 --2- 172.21.15.45:0/1645675868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa0102850 0x7fcaa01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.912+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 >> 172.21.15.45:0/1645675868 conn(0x7fcaa00fb770 msgr2=0x7fcaa00fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:16.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.913+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 shutdown_connections 2023-09-23T09:46:16.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:16.913+0000 7fcaa5f70700 1 -- 172.21.15.45:0/1645675868 wait complete. 2023-09-23T09:46:16.916 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-09-23T09:46:16.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 -- 172.21.15.45:0/734349903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c101000 msgr2=0x7f4b6c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 --2- 172.21.15.45:0/734349903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c101000 0x7f4b6c1013e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009990 tx=0x7f4b60009ca0 comp rx=0 tx=0).stop 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 -- 172.21.15.45:0/734349903 shutdown_connections 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 --2- 172.21.15.45:0/734349903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c106120 0x7f4b6c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 --2- 172.21.15.45:0/734349903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c101000 0x7f4b6c1013e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 --2- 172.21.15.45:0/734349903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b6c1019b0 0x7f4b6c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 -- 172.21.15.45:0/734349903 >> 172.21.15.45:0/734349903 conn(0x7f4b6c0fc880 msgr2=0x7f4b6c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 -- 172.21.15.45:0/734349903 shutdown_connections 2023-09-23T09:46:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.067+0000 7f4b731ed700 1 -- 172.21.15.45:0/734349903 wait complete. 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 Processor -- start 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 -- start start 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b6c101000 0x7f4b6c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106120 0x7f4b6c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c073ba0 0x7f4b6c1a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c1a56b0 con 0x7f4b6c106120 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c1a5820 con 0x7f4b6c073ba0 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b731ed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c1a59e0 con 0x7f4b6c101000 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106120 0x7f4b6c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106120 0x7f4b6c073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37502/0 (socket says 172.21.15.45:37502) 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b6bfff700 1 -- 172.21.15.45:0/2753460131 learned_addr learned my addr 172.21.15.45:0/2753460131 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b7178a700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c073ba0 0x7f4b6c1a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.068+0000 7f4b70f89700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b6c101000 0x7f4b6c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b6bfff700 1 -- 172.21.15.45:0/2753460131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b6c101000 msgr2=0x7f4b6c073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b6bfff700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b6c101000 0x7f4b6c073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b6bfff700 1 -- 172.21.15.45:0/2753460131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c073ba0 msgr2=0x7f4b6c1a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b6bfff700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c073ba0 0x7f4b6c1a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b6bfff700 1 -- 172.21.15.45:0/2753460131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600095f0 con 0x7f4b6c106120 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b6bfff700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106120 0x7f4b6c073660 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c9e0 tx=0x7f4b5800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b69ffb700 1 -- 172.21.15.45:0/2753460131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58007aa0 con 0x7f4b6c106120 2023-09-23T09:46:17.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b69ffb700 1 -- 172.21.15.45:0/2753460131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5800ce80 con 0x7f4b6c106120 2023-09-23T09:46:17.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c1a5cc0 con 0x7f4b6c106120 2023-09-23T09:46:17.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b69ffb700 1 -- 172.21.15.45:0/2753460131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58018660 con 0x7f4b6c106120 2023-09-23T09:46:17.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.069+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c1a61d0 con 0x7f4b6c106120 2023-09-23T09:46:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.070+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c067ba0 con 0x7f4b6c106120 2023-09-23T09:46:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.071+0000 7f4b69ffb700 1 -- 172.21.15.45:0/2753460131 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136323+0+0 (secure 0 0 0) 0x7f4b5800f450 con 0x7f4b6c106120 2023-09-23T09:46:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.072+0000 7f4b69ffb700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4b5c0a1710 0x7f4b5c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T09:46:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.072+0000 7f4b70f89700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4b5c0a1710 0x7f4b5c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T09:46:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.072+0000 7f4b69ffb700 1 -- 172.21.15.45:0/2753460131 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8245+0+0 (secure 0 0 0) 0x7f4b580cd1e0 con 0x7f4b6c106120 2023-09-23T09:46:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.072+0000 7f4b70f89700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4b5c0a1710 0x7f4b5c0a3bd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000b5c0 tx=0x7f4b60005960 comp rx=0 tx=0).ready entity=mgr.15405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T09:46:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.076+0000 7f4b69ffb700 1 -- 172.21.15.45:0/2753460131 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4b58096540 con 0x7f4b6c106120 2023-09-23T09:46:17.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:17.282+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b6c074650 con 0x7f4b6c106120 2023-09-23T09:46:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.048+0000 7f4b69ffb700 1 -- 172.21.15.45:0/2753460131 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4b5800fe60 con 0x7f4b6c106120 2023-09-23T09:46:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4b5c0a1710 msgr2=0x7f4b5c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4b5c0a1710 0x7f4b5c0a3bd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000b5c0 tx=0x7f4b60005960 comp rx=0 tx=0).stop 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106120 msgr2=0x7f4b6c073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106120 0x7f4b6c073660 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c9e0 tx=0x7f4b5800ccf0 comp rx=0 tx=0).stop 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 shutdown_connections 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c073ba0 0x7f4b6c1a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b6c101000 0x7f4b6c073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:6800/441481757,v1:172.21.15.45:6801/441481757] conn(0x7f4b5c0a1710 0x7f4b5c0a3bd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 --2- 172.21.15.45:0/2753460131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106120 0x7f4b6c073660 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 >> 172.21.15.45:0/2753460131 conn(0x7f4b6c0fc880 msgr2=0x7f4b6c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 shutdown_connections 2023-09-23T09:46:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-09-23T09:46:18.056+0000 7f4b731ed700 1 -- 172.21.15.45:0/2753460131 wait complete. 2023-09-23T09:46:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-09-23T09:46:18.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:46:18.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:17 smithi045 ceph-mon[98627]: pgmap v532: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 KiB/s wr, 21 op/s 2023-09-23T09:46:18.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:17 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/1645675868' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T09:46:18.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:17 smithi045 ceph-mon[98627]: osdmap e111: 12 total, 12 up, 12 in 2023-09-23T09:46:18.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:17 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2753460131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T09:46:18.237 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:46:18.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:17 smithi098 ceph-mon[102386]: pgmap v532: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 KiB/s wr, 21 op/s 2023-09-23T09:46:18.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:17 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/1645675868' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T09:46:18.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:17 smithi098 ceph-mon[102386]: osdmap e111: 12 total, 12 up, 12 in 2023-09-23T09:46:18.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:17 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2753460131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T09:46:18.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:17 smithi123 ceph-mon[103577]: pgmap v532: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 KiB/s wr, 21 op/s 2023-09-23T09:46:18.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:17 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/1645675868' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T09:46:18.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:17 smithi123 ceph-mon[103577]: osdmap e111: 12 total, 12 up, 12 in 2023-09-23T09:46:18.340 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:17 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2753460131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T09:46:18.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:46:18.468 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-23T09:46:18.468 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-23T09:46:18.566 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-23T09:46:18.641 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-23T09:46:18.731 INFO:tasks.cephadm:Stopping all daemons... 2023-09-23T09:46:18.731 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-23T09:46:18.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a 2023-09-23T09:46:19.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:18 smithi045 systemd[1]: Stopping Ceph mon.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:19.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:19 smithi045 ceph-mon[98627]: from='client.? 172.21.15.45:0/2753460131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T09:46:19.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:19 smithi045 ceph-mon[98627]: osdmap e112: 12 total, 12 up, 12 in 2023-09-23T09:46:19.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:19 smithi123 ceph-mon[103577]: from='client.? 172.21.15.45:0/2753460131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T09:46:19.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:19 smithi123 ceph-mon[103577]: osdmap e112: 12 total, 12 up, 12 in 2023-09-23T09:46:19.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:19 smithi098 ceph-mon[102386]: from='client.? 172.21.15.45:0/2753460131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T09:46:19.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:19 smithi098 ceph-mon[102386]: osdmap e112: 12 total, 12 up, 12 in 2023-09-23T09:46:19.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-a[98603]: 2023-09-23T09:46:19.359+0000 7fd8d3860700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:19.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-a[98603]: 2023-09-23T09:46:19.359+0000 7fd8d3860700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-23T09:46:19.942 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 23 09:46:19 smithi045 bash[179315]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-a 2023-09-23T09:46:19.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.a.service' 2023-09-23T09:46:20.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:20.007 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-23T09:46:20.007 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-23T09:46:20.008 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.b 2023-09-23T09:46:20.294 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:20 smithi098 systemd[1]: Stopping Ceph mon.b for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:20.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-b[102362]: 2023-09-23T09:46:20.625+0000 7f36fb463700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:20.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-b[102362]: 2023-09-23T09:46:20.625+0000 7f36fb463700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-23T09:46:21.293 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 23 09:46:20 smithi098 bash[144400]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-b 2023-09-23T09:46:21.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.b.service' 2023-09-23T09:46:21.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:21.544 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-23T09:46:21.544 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-23T09:46:21.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.c 2023-09-23T09:46:21.838 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:21 smithi123 systemd[1]: Stopping Ceph mon.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:22.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-c[103553]: 2023-09-23T09:46:21.956+0000 7fdd8eed0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:22.338 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-c[103553]: 2023-09-23T09:46:21.956+0000 7fdd8eed0700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-23T09:46:22.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.c.service' 2023-09-23T09:46:22.789 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:22 smithi123 bash[147489]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mon-c 2023-09-23T09:46:22.789 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:22 smithi123 systemd[1]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mon.c.service: Succeeded. 2023-09-23T09:46:22.789 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 23 09:46:22 smithi123 systemd[1]: Stopped Ceph mon.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7. 2023-09-23T09:46:22.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:22.840 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-23T09:46:22.840 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-23T09:46:22.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.x 2023-09-23T09:46:23.329 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:46:22 smithi045 systemd[1]: Stopping Ceph mgr.x for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:24.080 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:46:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x[98868]: 2023-09-23T09:46:23.721+0000 7f888e45b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:25.857 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 23 09:46:25 smithi045 bash[179460]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-x 2023-09-23T09:46:25.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.x.service' 2023-09-23T09:46:25.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:25.903 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-23T09:46:25.904 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-23T09:46:25.904 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.y 2023-09-23T09:46:26.293 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:46:25 smithi098 systemd[1]: Stopping Ceph mgr.y for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:26.616 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:46:26 smithi098 bash[144552]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mgr-y 2023-09-23T09:46:26.798 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 23 09:46:26 smithi098 systemd[1]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-23T09:46:26.803 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.y.service' 2023-09-23T09:46:26.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:26.840 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-23T09:46:26.840 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-23T09:46:26.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.z 2023-09-23T09:46:27.278 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 23 09:46:26 smithi123 systemd[1]: Stopping Ceph mgr.z for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:27.736 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mgr.z.service' 2023-09-23T09:46:27.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:27.773 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-23T09:46:27.773 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-23T09:46:27.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.0 2023-09-23T09:46:28.079 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:46:27 smithi045 systemd[1]: Stopping Ceph osd.0 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:28.580 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:46:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-0[118000]: 2023-09-23T09:46:28.128+0000 7fdd7f853700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:28.580 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:46:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-0[118000]: 2023-09-23T09:46:28.128+0000 7fdd7f853700 -1 osd.0 112 *** Got signal Terminated *** 2023-09-23T09:46:28.580 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:46:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-0[118000]: 2023-09-23T09:46:28.128+0000 7fdd7f853700 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:46:34.080 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 23 09:46:33 smithi045 bash[179609]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-0 2023-09-23T09:46:35.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.0.service' 2023-09-23T09:46:35.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:35.347 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-23T09:46:35.347 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-23T09:46:35.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.1 2023-09-23T09:46:35.793 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:46:35 smithi098 systemd[1]: Stopping Ceph osd.1 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:36.543 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:46:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-1[107309]: 2023-09-23T09:46:36.120+0000 7fcb2fa6a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:36.543 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:46:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-1[107309]: 2023-09-23T09:46:36.120+0000 7fcb2fa6a700 -1 osd.1 112 *** Got signal Terminated *** 2023-09-23T09:46:36.543 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:46:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-1[107309]: 2023-09-23T09:46:36.120+0000 7fcb2fa6a700 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:46:42.043 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 23 09:46:41 smithi098 bash[144709]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-1 2023-09-23T09:46:43.636 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.1.service' 2023-09-23T09:46:43.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:43.677 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-23T09:46:43.677 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-23T09:46:43.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.2 2023-09-23T09:46:44.043 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:46:43 smithi123 systemd[1]: Stopping Ceph osd.2 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:44.338 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:46:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-2[109353]: 2023-09-23T09:46:44.041+0000 7f75ac5dd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:44.338 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:46:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-2[109353]: 2023-09-23T09:46:44.041+0000 7f75ac5dd700 -1 osd.2 112 *** Got signal Terminated *** 2023-09-23T09:46:44.338 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:46:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-2[109353]: 2023-09-23T09:46:44.041+0000 7f75ac5dd700 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:46:50.020 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 23 09:46:49 smithi123 bash[147790]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-2 2023-09-23T09:46:51.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.2.service' 2023-09-23T09:46:51.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:51.723 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-23T09:46:51.724 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-23T09:46:51.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.3 2023-09-23T09:46:52.080 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:46:51 smithi045 systemd[1]: Stopping Ceph osd.3 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:52.580 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:46:52 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-3[125835]: 2023-09-23T09:46:52.083+0000 7ff176b41700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:46:52.580 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:46:52 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-3[125835]: 2023-09-23T09:46:52.083+0000 7ff176b41700 -1 osd.3 112 *** Got signal Terminated *** 2023-09-23T09:46:52.580 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:46:52 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-3[125835]: 2023-09-23T09:46:52.083+0000 7ff176b41700 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:46:55.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:46:54 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:46:54.652+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:46:55.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:46:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:46:54.744+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:46:56.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:46:55 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:46:55.618+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:46:56.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:46:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:46:55.758+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:46:57.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:46:56 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:46:56.614+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:46:57.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:46:56 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:46:56.749+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:46:57.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:46:56 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:46:56.731+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:46:57.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:46:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:46:56.765+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:46:57.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:46:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:46:57.190+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:46:58.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:46:57 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:46:57.658+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:46:58.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:46:57 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:46:57.759+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:46:58.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:46:57 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:46:57.730+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:46:58.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:46:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:46:57.745+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:46:58.579 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 23 09:46:58 smithi045 bash[179886]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-3 2023-09-23T09:46:58.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:46:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:46:58.240+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:46:59.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:46:58 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:46:58.706+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:46:59.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:46:58 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:46:58.797+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:46:59.090 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:46:58 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:46:58.720+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:46:59.090 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:46:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:46:58.747+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:46:59.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.3.service' 2023-09-23T09:46:59.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:46:59.485 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-23T09:46:59.485 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-23T09:46:59.485 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.4 2023-09-23T09:46:59.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:46:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:46:59.210+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:46:59.793 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:46:59 smithi098 systemd[1]: Stopping Ceph osd.4 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:46:59.793 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:46:59 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:46:59.681+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:00.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:46:59 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:46:59.805+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:00.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:46:59 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:46:59.768+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:00.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:46:59 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:46:59.785+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:00.293 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:46:59 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4[115193]: 2023-09-23T09:46:59.830+0000 7f2b916b1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:00.293 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:46:59 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4[115193]: 2023-09-23T09:46:59.830+0000 7f2b916b1700 -1 osd.4 112 *** Got signal Terminated *** 2023-09-23T09:47:00.293 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:46:59 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4[115193]: 2023-09-23T09:46:59.830+0000 7f2b916b1700 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:00.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:00 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:00.237+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:01.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:00 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:00.687+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:01.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:00 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:00.851+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:01.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:00 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:00.747+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:01.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:00 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:00.738+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:01.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:01 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:01.276+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:02.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:01 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:01.703+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:02.046 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:01 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:01.763+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:02.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:01 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:01.865+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:02.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:01 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:01.738+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:02.338 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:02 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:02.238+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:02.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:02 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:02.045+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:03.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:02 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:02.750+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:03.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:02 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:02.750+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:03.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:02 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:02.861+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:03.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:02 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:02.746+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:03.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:02 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:02.793+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:03.088 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:03 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:03.028+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:03.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:03 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:03.248+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:03.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:03 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:03.248+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:03.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:03 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:03.492+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:04.050 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:03 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:03.761+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:04.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:03 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:03.870+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:04.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:03 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:03.766+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:04.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:03 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:03.794+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:04.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:03 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:03.794+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:04.338 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:04.205+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:04.338 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:04.205+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:04.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:04.048+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:04.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:04 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:04.492+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:04.793 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:47:04 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4[115193]: 2023-09-23T09:47:04.727+0000 7f2b8aca1700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.195154+0000 front 2023-09-23T09:46:39.195179+0000 (oldest deadline 2023-09-23T09:47:04.495355+0000) 2023-09-23T09:47:05.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:04 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:04.903+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:05.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:04 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:04.903+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:05.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:04 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:04.811+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:05.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:04 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:04.811+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:05.085 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:04 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:04.794+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:05.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:04 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:04.798+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:05.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:04 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:04.798+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:05.338 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:05 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:05.212+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:05.338 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:05 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:05.212+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:05.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:05 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:05.082+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:05.695 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:05 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:05.505+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:06.043 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 23 09:47:05 smithi098 bash[144982]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-4 2023-09-23T09:47:06.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:05 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:05.838+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:06.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:05 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:05.838+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:06.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:05 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:05.894+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:06.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:05 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:05.894+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:06.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:05 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:05.796+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:06.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:05 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:05.796+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:06.088 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:06 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:06.048+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:06.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:05 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:05.819+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:06.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:06 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:06.261+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:06.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:06 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:06.261+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:06.838 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:06 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:06.549+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:06.838 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:06 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:06.836+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:06.875 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.4.service' 2023-09-23T09:47:06.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:47:06.915 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-23T09:47:06.915 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-23T09:47:06.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.5 2023-09-23T09:47:07.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:06.883+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:07.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:06.883+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:07.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:06.825+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:07.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:06.825+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:07.262 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:06 smithi123 systemd[1]: Stopping Ceph osd.5 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:47:07.262 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:06 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:06.864+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:07.262 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:06 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:06.864+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:07.262 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:07.089+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:07.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:06 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:06.836+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:07.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:07.259+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:07.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:07.259+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:07.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:07.261+0000 7fcd1036c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:07.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:07.261+0000 7fcd1036c700 -1 osd.5 112 *** Got signal Terminated *** 2023-09-23T09:47:07.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:07.261+0000 7fcd1036c700 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:07.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:07 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:07.534+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:08.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:07 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:07.853+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:08.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:07 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:07.853+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:08.301 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:08 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:08.045+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:08.301 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:07.908+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:08.301 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:07 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:07.908+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:08.329 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:07 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:07.859+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:08.329 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:07 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:07.859+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:08.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:07 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:07.837+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:08.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:07 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:07.837+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:08.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:08 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:08.298+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:08.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:08 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:08.298+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:08.827 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:08 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:08.572+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:09.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:08.908+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:09.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:08.908+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:09.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:08.811+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:09.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:08 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:08.811+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:09.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:08 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:08.826+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:09.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:08 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:08.826+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:09.304 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:09 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:09.025+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:09.304 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:08 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:08.862+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:09.304 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:08 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:08.862+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:09.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:09 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:09.302+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:09.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:09 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:09.302+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:09.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:09 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:09.531+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:10.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:09 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:09.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:10.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:09 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:09.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:10.305 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:10 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:10.054+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:10.305 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:09 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:09.907+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:10.305 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:09 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:09.907+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:10.329 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:09 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:09.899+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:10.329 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:09 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:09.899+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:10.329 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:09 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:09.838+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:10.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:09 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:09.838+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:10.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:10 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:10.303+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:10.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:10 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:10.303+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:11.043 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:10 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:10.547+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:11.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:10 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:10.789+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:11.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:10 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:10.789+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:11.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:10 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:10.790+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:11.329 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:10 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:10.926+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:11.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:10 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:10.926+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:11.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:10 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:10.849+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:11.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:10 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:10.849+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:11.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:10 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:10.849+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:11.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:11 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:11.020+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:11.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:10 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:10.906+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:11.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:10 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:10.906+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:11.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:11 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:11.531+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:11.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:11 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:11.531+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:11.838 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:11 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:11.347+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:31.307366+0000 front 2023-09-23T09:46:31.307354+0000 (oldest deadline 2023-09-23T09:46:56.607316+0000) 2023-09-23T09:47:11.838 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:11 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5[117407]: 2023-09-23T09:47:11.347+0000 7fcd0c962700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:36.607778+0000 front 2023-09-23T09:46:36.607801+0000 (oldest deadline 2023-09-23T09:47:02.507840+0000) 2023-09-23T09:47:12.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:11 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:11.894+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:12.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:11 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:11.894+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:12.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:11 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:11.814+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:12.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:11 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:11.814+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:12.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:11 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:11.814+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:12.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:11 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:11.815+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:12.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:11 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:11.815+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:12.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:11 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:11.815+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:12.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:12 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:12.058+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:12.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:11 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:11.882+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:12.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:11 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:11.882+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:12.788 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:12 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:12.513+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:12.788 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:12 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:12.513+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:13.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:12 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:12.787+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:13.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:12 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:12.787+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:13.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:12 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:12.787+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:13.219 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:12 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:12.885+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:13.220 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:12 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:12.885+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:13.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:13 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:13.046+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:13.329 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:12 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:12.888+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:13.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:12 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:12.888+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:13.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:12 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:12.842+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:13.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:12 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:12.842+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:13.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:12 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:12.842+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:13.588 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 23 09:47:13 smithi123 bash[148054]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-5 2023-09-23T09:47:13.768 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:13 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:13.467+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:13.768 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:13 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:13.467+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:14.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:13 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:13.767+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:14.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:13 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:13.767+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:14.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:13 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:13.767+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:14.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:13 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:13.918+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:14.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:13 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:13.918+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:14.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:13 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:13.815+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:14.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:13 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:13.815+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:14.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:13 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:13.815+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:14.226 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:14.073+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:14.226 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:13 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:13.882+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:14.226 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:13 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:13.882+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:14.227 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:13 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:13.882+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:14.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.5.service' 2023-09-23T09:47:14.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:47:14.651 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-23T09:47:14.651 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-23T09:47:14.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.6 2023-09-23T09:47:14.757 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:14.493+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:14.757 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:14.493+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:14.957 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:14 smithi045 systemd[1]: Stopping Ceph osd.6 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:47:14.957 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:14 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:14.768+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:14.957 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:14 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:14.768+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:14.957 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:14 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:14.768+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:15.031 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:15.030+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:15.031 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:14.838+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:15.031 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:14.838+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:15.031 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:14 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:14.838+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:15.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:14.756+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:15.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:14.756+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:15.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:14 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:14.756+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:15.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:14 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:14.956+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:15.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:14 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:14.956+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:15.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:14 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:14.956+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.683985+0000 front 2023-09-23T09:46:48.683746+0000 (oldest deadline 2023-09-23T09:47:13.983844+0000) 2023-09-23T09:47:15.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:15.012+0000 7f6bb9323700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:15.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:15.012+0000 7f6bb9323700 -1 osd.6 112 *** Got signal Terminated *** 2023-09-23T09:47:15.330 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:15.012+0000 7f6bb9323700 -1 osd.6 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:15.788 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:15 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:15.532+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:15.788 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:15 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:15.532+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:16.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:15 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:15.787+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:16.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:15 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:15.787+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:16.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:15 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:15.787+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:16.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:15.980+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:16.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:15.980+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:16.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:15.980+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.683985+0000 front 2023-09-23T09:46:48.683746+0000 (oldest deadline 2023-09-23T09:47:13.983844+0000) 2023-09-23T09:47:16.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:15.773+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:16.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:15.773+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:16.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:15 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:15.773+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:16.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:15.030+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:16.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:15.983+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:16.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:15.983+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:16.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:15.838+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:16.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:15.838+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:16.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:15 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:15.838+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:16.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:16 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:16.483+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:16.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:16 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:16.483+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:17.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:16.974+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:17.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:16.974+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:17.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:16.974+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.683985+0000 front 2023-09-23T09:46:48.683746+0000 (oldest deadline 2023-09-23T09:47:13.983844+0000) 2023-09-23T09:47:17.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:16.815+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:17.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:16.815+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:17.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:16.815+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:17.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:16 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:16.792+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:17.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:16 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:16.792+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:17.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:16 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:16.792+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:17.088 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:17.017+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:17.088 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:17.017+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:17.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:16 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:16.817+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:17.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:16 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:16.817+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:17.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:16 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:16.817+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:17.787 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:17.466+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:17.787 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:17.466+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:18.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:17.786+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:18.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:17.786+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:18.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:17 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:17.786+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:18.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:18.000+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:18.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:18.000+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:18.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:18.000+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.683985+0000 front 2023-09-23T09:46:48.683746+0000 (oldest deadline 2023-09-23T09:47:13.983844+0000) 2023-09-23T09:47:18.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:17.792+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:18.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:17.792+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:18.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:17 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:17.792+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:18.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:17.839+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:18.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:17.839+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:18.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:17 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:17.839+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:18.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:18.065+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:18.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:18.065+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:18.742 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:18.436+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:18.742 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:18.436+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:19.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:18.741+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:19.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:18.741+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:19.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:18 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:18.741+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:19.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:19.031+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:19.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:19.031+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:19.080 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:19.031+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.683985+0000 front 2023-09-23T09:46:48.683746+0000 (oldest deadline 2023-09-23T09:47:13.983844+0000) 2023-09-23T09:47:19.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:18.791+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:19.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:18.791+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:19.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:18 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:18.791+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:19.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:19.096+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:19.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:19.096+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:19.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:18.885+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:19.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:18.885+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:19.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:18 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:18.885+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:19.737 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:19.471+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:19.737 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:19.471+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:19.737 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:19.471+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:20.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:19.736+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:20.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:19.736+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:20.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:19 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:19.736+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:20.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:19.984+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.580240+0000 front 2023-09-23T09:46:30.580194+0000 (oldest deadline 2023-09-23T09:46:56.480415+0000) 2023-09-23T09:47:20.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:19.984+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.882499+0000 front 2023-09-23T09:46:39.882756+0000 (oldest deadline 2023-09-23T09:47:03.982582+0000) 2023-09-23T09:47:20.079 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6[134367]: 2023-09-23T09:47:19.984+0000 7f6bb1911700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.683985+0000 front 2023-09-23T09:46:48.683746+0000 (oldest deadline 2023-09-23T09:47:13.983844+0000) 2023-09-23T09:47:20.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:19.741+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:20.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:19.741+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:20.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:19 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:19.741+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:20.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:20.089+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:20.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:20.089+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:20.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:19.917+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:20.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:19.917+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:20.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:19 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:19.917+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:20.757 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:20.462+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:20.757 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:20.462+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:20.757 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:20.462+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:20.994 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 23 09:47:20 smithi045 bash[180147]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-6 2023-09-23T09:47:20.995 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:20.722+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:20.995 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:20.722+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:20.995 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:20 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:20.722+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:21.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:20.755+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:21.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:20.755+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:21.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:20.756+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:21.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:20 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:20.756+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:21.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:21.072+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:21.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:21.072+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:21.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:21.072+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:21.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:20.920+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:21.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:20.920+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:21.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:20 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:20.920+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:21.787 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:21.428+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:21.787 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:21.428+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:21.787 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:21.428+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:21.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:21.718+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:21.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:21.718+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:21.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:21 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:21.718+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:22.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:21.786+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:22.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:21.786+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:22.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:21.786+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:22.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:21.786+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:22.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.6.service' 2023-09-23T09:47:22.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:47:22.312 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-23T09:47:22.312 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-23T09:47:22.312 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.7 2023-09-23T09:47:22.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:22.103+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:22.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:22.103+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:22.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:22.103+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:22.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:21.921+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:22.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:21.921+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:22.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:21 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:21.921+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:22.669 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:22 smithi098 systemd[1]: Stopping Ceph osd.7 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:47:22.669 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:22.462+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:22.669 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:22.462+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:22.669 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:22.462+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:22.669 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:22.668+0000 7fece11b0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:22.669 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:22.668+0000 7fece11b0700 -1 osd.7 112 *** Got signal Terminated *** 2023-09-23T09:47:23.043 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:22.668+0000 7fece11b0700 -1 osd.7 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:23.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:22.789+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:23.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:22.789+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:23.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:22.789+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:23.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:22 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:22.789+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:23.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:22.682+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:23.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:22.682+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:23.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:22.682+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:23.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:22 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:22.682+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:23.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:23 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:23.135+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:23.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:23 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:23.135+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:23.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:23 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:23.135+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:23.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:22.951+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:23.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:22.951+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:23.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:22 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:22.951+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:23.770 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:23.458+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:23.771 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:23.458+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:23.771 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:23.458+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:24.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:23.770+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:24.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:23.770+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:24.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:23.770+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:24.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:23.770+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:24.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:23.726+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:24.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:23.726+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:24.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:23.726+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:24.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:23 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:23.726+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:24.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:24 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:24.098+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:24.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:24 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:24.098+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:24.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:24 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:24.098+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:24.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:23 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:23.922+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:24.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:23 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:23.922+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:24.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:23 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:23.922+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:24.791 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:24.504+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:24.791 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:24.504+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:24.791 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:24.504+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:25.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:24.790+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:25.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:24.790+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:25.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:24.790+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:25.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:24 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:24.790+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:25.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:24.699+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:25.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:24.699+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:25.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:24.699+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:25.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:24 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:24.699+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:25.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:25 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:25.060+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:25.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:25 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:25.060+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:25.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:25 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:25.060+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:25.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:24 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:24.938+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:25.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:24 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:24.938+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:25.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:24 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:24.938+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:25.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:25 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:25.469+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:25.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:25 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:25.469+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:25.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:25 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:25.469+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:26.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:25.701+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:26.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:25.701+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:26.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:25.701+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:26.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:25 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:25.701+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:26.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:25 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:25.838+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:26.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:25 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:25.838+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:26.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:25 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:25.838+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:26.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:25 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:25.838+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:26.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:26 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:26.066+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:26.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:26 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:26.066+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:26.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:26 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:26.066+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:26.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:25 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:25.905+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:26.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:25 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:25.905+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:26.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:25 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:25.905+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:26.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:26 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:26.481+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:26.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:26 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:26.481+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:26.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:26 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:26.481+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:27.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:26.731+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:27.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:26.731+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:27.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:26.731+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:27.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:26 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:26.731+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:27.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:26 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:26.887+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:27.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:26 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:26.887+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:27.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:26 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:26.887+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:27.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:26 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:26.887+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:27.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:27 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:27.072+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:27.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:27 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:27.072+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:27.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:27 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:27.072+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:27.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:26 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:26.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:27.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:26 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:26.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:27.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:26 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:26.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:27.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:27.511+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:37.060831+0000 front 2023-09-23T09:46:37.060943+0000 (oldest deadline 2023-09-23T09:47:02.960981+0000) 2023-09-23T09:47:27.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:27.511+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.562798+0000 front 2023-09-23T09:46:47.562824+0000 (oldest deadline 2023-09-23T09:47:11.062662+0000) 2023-09-23T09:47:27.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:27.511+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.765201+0000 front 2023-09-23T09:46:56.765171+0000 (oldest deadline 2023-09-23T09:47:19.065199+0000) 2023-09-23T09:47:27.793 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7[123147]: 2023-09-23T09:47:27.511+0000 7fecd979e700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:01.167054+0000 front 2023-09-23T09:47:01.167079+0000 (oldest deadline 2023-09-23T09:47:27.067190+0000) 2023-09-23T09:47:28.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:27 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:27.690+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:28.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:27 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:27.690+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:28.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:27 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:27.690+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:28.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:27 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:27.690+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:28.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:27.859+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:28.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:27.859+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:28.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:27.859+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:28.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:27.859+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:28.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:28 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:28.095+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:28.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:28 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:28.095+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:28.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:28 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:28.095+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:28.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:28 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:28.095+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:28.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:27 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:27.972+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:28.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:27 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:27.972+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:28.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:27 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:27.972+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:28.831 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 23 09:47:28 smithi098 bash[145247]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-7 2023-09-23T09:47:29.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:28.645+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:29.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:28.645+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:29.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:28.645+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:29.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:28 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:28.645+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:29.264 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:28 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:28.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:29.264 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:28 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:28.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:29.264 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:28 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:28.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:29.264 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:28 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:28.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:29.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:29.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:29.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:29.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:29.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:29.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:29.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:29.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:29.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.018+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:29.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.018+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:29.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.018+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:29.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.018+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:29.901 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.7.service' 2023-09-23T09:47:29.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:47:29.941 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-23T09:47:29.941 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-23T09:47:29.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.8 2023-09-23T09:47:30.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:29 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:29.596+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:30.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:29 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:29.596+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:30.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:29 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:29.596+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:30.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:29 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:29.596+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:30.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:29 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:29.878+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:30.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:29 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:29.878+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:30.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:29 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:29.878+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:30.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:29 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:29.878+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:30.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:29 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:29.878+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:30.294 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.991+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:30.294 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.991+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:30.294 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.991+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:30.294 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:29 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:29.991+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:30.294 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:29 smithi123 systemd[1]: Stopping Ceph osd.8 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:47:30.294 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:30.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:30.294 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:30.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:30.295 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:30.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:30.295 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:30.104+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:30.295 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:30.292+0000 7fbe3cabf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:30.295 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:30.292+0000 7fbe3cabf700 -1 osd.8 112 *** Got signal Terminated *** 2023-09-23T09:47:30.588 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:30.292+0000 7fbe3cabf700 -1 osd.8 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:30.829 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:30 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:30.549+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:30.829 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:30 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:30.549+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:30.829 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:30 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:30.549+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:30.829 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:30 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:30.549+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:31.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:30 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:30.861+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:31.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:30 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:30.861+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:31.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:30 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:30.861+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:31.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:30 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:30.861+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:31.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:30 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:30.861+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:31.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:31 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:31.068+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:31.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:31 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:31.068+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:31.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:31 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:31.068+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:31.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:31 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:31.068+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:31.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:30.988+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:31.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:30.988+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:31.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:30.988+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:31.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:30 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:30.988+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:31.829 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:31 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:31.507+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:31.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:31 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:31.507+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:31.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:31 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:31.507+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:31.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:31 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:31.507+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:32.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:31 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:31.825+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:32.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:31 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:31.825+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:32.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:31 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:31.825+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:32.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:31 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:31.825+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:32.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:31 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:31.825+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:32.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:32.063+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:32.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:32.064+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:32.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:32.064+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:32.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:32.064+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:32.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:32.009+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:32.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:32.009+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:32.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:32.009+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:32.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:32 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:32.009+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:32.829 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:32 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:32.536+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:32.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:32 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:32.536+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:32.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:32 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:32.536+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:32.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:32 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:32.536+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:33.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:32 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:32.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:33.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:32 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:32.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:33.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:32 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:32.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:33.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:32 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:32.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:33.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:32 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:32.830+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:33.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:33.109+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:33.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:33.109+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:33.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:33.109+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:33.338 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:33.109+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:33.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:33.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:33.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:33.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:33.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:33.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:33.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:33 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:33.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:33.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:33.556+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:33.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:33.556+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:33.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:33.556+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:33.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:33.556+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:33.830 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:33 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:33.556+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:34.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:33 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:33.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:34.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:33 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:33.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:34.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:33 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:33.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:34.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:33 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:33.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:34.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:33 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:33.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:34.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:33 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:33.834+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:34.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:34.016+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:34.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:34.016+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:34.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:34.016+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:34.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:34.016+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:34.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:34.159+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:34.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:34.159+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:34.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:34.159+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:34.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:34 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:34.159+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:35.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:34 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:34.605+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:35.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:34 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:34.605+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:35.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:34 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:34.605+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:35.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:34 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:34.605+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:35.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:34 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:34.605+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:35.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:34 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:34.884+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:35.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:34 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:34.884+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:35.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:34 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:34.884+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:35.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:34 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:34.884+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:35.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:34 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:34.884+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:35.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:34 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:34.884+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:35.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:35.047+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:35.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:35.047+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:35.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:35.047+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:35.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:35.047+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:35.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:35.193+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.168216+0000 front 2023-09-23T09:46:40.168142+0000 (oldest deadline 2023-09-23T09:47:01.868086+0000) 2023-09-23T09:47:35.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:35.193+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.570164+0000 front 2023-09-23T09:46:47.570142+0000 (oldest deadline 2023-09-23T09:47:14.570871+0000) 2023-09-23T09:47:35.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:35.193+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:56.872132+0000 front 2023-09-23T09:46:56.872098+0000 (oldest deadline 2023-09-23T09:47:20.972026+0000) 2023-09-23T09:47:35.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:35.193+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.273077+0000 front 2023-09-23T09:47:03.273144+0000 (oldest deadline 2023-09-23T09:47:27.373038+0000) 2023-09-23T09:47:35.339 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8[125312]: 2023-09-23T09:47:35.193+0000 7fbe380b3700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.374607+0000 front 2023-09-23T09:47:11.374631+0000 (oldest deadline 2023-09-23T09:47:34.874780+0000) 2023-09-23T09:47:36.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:35 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:35.613+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:36.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:35 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:35.613+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:36.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:35 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:35.613+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:36.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:35 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:35.613+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:36.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:35 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:35.613+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:36.280 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 23 09:47:36 smithi123 bash[148318]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-8 2023-09-23T09:47:36.280 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:36 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:36.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:36.280 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:36 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:36.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:36.280 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:36 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:36.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:36.280 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:36 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:36.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:36.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:35 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:35.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:36.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:35 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:35.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:36.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:35 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:35.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:36.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:35 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:35.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:36.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:35 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:35.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:36.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:35 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:35.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:37.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:36 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:36.642+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:37.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:36 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:36.642+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:37.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:36 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:36.642+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:37.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:36 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:36.642+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:37.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:36 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:36.642+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:37.252 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:37 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:37.061+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:37.252 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:37 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:37.061+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:37.252 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:37 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:37.061+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:37.252 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:37 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:37.061+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:37.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:36.934+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:37.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:36.934+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:37.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:36.934+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:37.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:36.934+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:37.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:36.934+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:37.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:36 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:36.934+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:37.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.8.service' 2023-09-23T09:47:37.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:47:37.788 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-23T09:47:37.788 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-23T09:47:37.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.9 2023-09-23T09:47:38.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:37 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:37.649+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:38.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:37 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:37.649+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:38.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:37 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:37.649+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:38.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:37 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:37.649+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:38.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:37 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:37.649+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:38.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:37 smithi045 systemd[1]: Stopping Ceph osd.9 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:47:38.153 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.150+0000 7fe6a9b62700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:38.153 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.150+0000 7fe6a9b62700 -1 osd.9 112 *** Got signal Terminated *** 2023-09-23T09:47:38.153 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.150+0000 7fe6a9b62700 -1 osd.9 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:38.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:37 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:37.910+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:38.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:37 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:37.910+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:38.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:37 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:37.910+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:38.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:37 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:37.910+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:38.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:37 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:37.910+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:38.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:37 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:37.910+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:38.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:38 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:38.068+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:38.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:38 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:38.068+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:38.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:38 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:38.068+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:38.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:38 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:38.068+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:38.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:38 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:38.068+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:39.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.619+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:39.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.619+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:39.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.619+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:39.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.619+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:39.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:38 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:38.619+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:39.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:38.956+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:39.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:38.956+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:39.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:38.956+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:39.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:38.956+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:39.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:38.956+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:39.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:38 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:38.956+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:39.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:39 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:39.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:39.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:39 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:39.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:39.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:39 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:39.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:39.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:39 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:39.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:39.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:39 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:39.031+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:40.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:39 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:39.658+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:40.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:39 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:39.658+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:40.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:39 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:39.658+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:40.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:39 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:39.658+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:40.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:39 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:39.658+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:40.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:39 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:39.942+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:40.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:39 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:39.942+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:40.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:39 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:39.942+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:40.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:39 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:39.942+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:40.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:39 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:39.942+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:40.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:39 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:39.942+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:40.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:40 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:40.036+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:40.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:40 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:40.036+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:40.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:40 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:40.036+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:40.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:40 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:40.036+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:40.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:40 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:40.036+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:41.079 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:40 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:40.618+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:41.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:40 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:40.618+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:41.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:40 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:40.618+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:41.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:40 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:40.618+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:41.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:40 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:40.618+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:41.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:40.936+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:41.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:40.936+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:41.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:40.936+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:41.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:40.936+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:41.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:40.936+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:41.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:40 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:40.936+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:41.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:41 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:41.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:41.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:41 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:41.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:41.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:41 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:41.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:41.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:41 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:41.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:41.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:41 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:41.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:42.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:41 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:41.661+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:42.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:41 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:41.661+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:42.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:41 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:41.661+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:42.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:41 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:41.661+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:42.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:41 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:41.661+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:42.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:41 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:41.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:42.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:41 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:41.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:42.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:41 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:41.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:42.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:41 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:41.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:42.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:41 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:41.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:42.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:41 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:41.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:42.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:41 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:41.913+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:42.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:42.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:42.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:42.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:42.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.005+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:43.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:42 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:42.626+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:30.393171+0000 front 2023-09-23T09:46:30.392961+0000 (oldest deadline 2023-09-23T09:46:56.293092+0000) 2023-09-23T09:47:43.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:42 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:42.626+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:39.194378+0000 front 2023-09-23T09:46:39.194439+0000 (oldest deadline 2023-09-23T09:47:03.894511+0000) 2023-09-23T09:47:43.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:42 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:42.627+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:47.995795+0000 front 2023-09-23T09:46:47.995759+0000 (oldest deadline 2023-09-23T09:47:10.295876+0000) 2023-09-23T09:47:43.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:42 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:42.627+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.597092+0000 front 2023-09-23T09:46:55.597074+0000 (oldest deadline 2023-09-23T09:47:21.997600+0000) 2023-09-23T09:47:43.080 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:42 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9[143134]: 2023-09-23T09:47:42.627+0000 7fe6a6158700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:07.000951+0000 front 2023-09-23T09:47:07.000922+0000 (oldest deadline 2023-09-23T09:47:32.900988+0000) 2023-09-23T09:47:43.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:42.864+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:43.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:42.864+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:43.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:42.864+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:43.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:42.864+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:43.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:42.864+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:43.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:42.864+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:43.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:42 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:42.864+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:43.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.957+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:43.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.957+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:43.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.957+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:43.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.957+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:43.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:42 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:42.957+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:44.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:43.832+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:44.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:43.832+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:44.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:43.832+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:44.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:43.832+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:44.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:43.832+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:44.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:43.832+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:44.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:43 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:43.832+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:44.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 23 09:47:44 smithi045 bash[180422]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-9 2023-09-23T09:47:44.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:43.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:44.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:43.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:44.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:43.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:44.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:43.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:44.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:43.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:44.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:43.945+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:45.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:44 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:44.823+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:45.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:44 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:44.823+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:45.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:44 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:44.823+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:45.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:44 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:44.823+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:45.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:44 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:44.823+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:45.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:44 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:44.823+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:45.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:44 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:44.823+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:45.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:44.977+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:45.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:44.977+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:45.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:44.977+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:45.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:44.977+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:45.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:44.977+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:45.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:44 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:44.977+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:45.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.9.service' 2023-09-23T09:47:45.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:47:45.725 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-23T09:47:45.725 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-23T09:47:45.725 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.10 2023-09-23T09:47:46.043 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 systemd[1]: Stopping Ceph osd.10 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:47:46.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:45.873+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:46.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:45.873+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:46.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:45.873+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:46.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:45.873+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:46.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:45.873+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:46.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:45.873+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:46.044 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:45 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:45.873+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:46.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:46 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:46.013+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:46.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:46 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:46.013+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:46.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:46 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:46.013+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:46.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:46 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:46.013+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:46.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:46 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:46.013+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:46.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:46 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:46.013+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:46.543 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.078+0000 7f5fe2f31700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:46.543 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.078+0000 7f5fe2f31700 -1 osd.10 112 *** Got signal Terminated *** 2023-09-23T09:47:46.544 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.078+0000 7f5fe2f31700 -1 osd.10 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:47.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.919+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:47.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.919+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:47.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.919+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:47.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.919+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:47.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.919+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:47.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.919+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:47.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:46 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:46.919+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:47.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:47.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:47.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:47.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:47.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:47.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:47.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:47.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:47.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:47.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:47.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:47.046+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:48.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:47.925+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:48.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:47.925+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:48.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:47.925+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:48.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:47.925+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:48.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:47.925+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:48.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:47.925+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:48.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:47 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:47.925+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:48.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:48 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:48.071+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:48.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:48 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:48.071+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:48.338 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:48 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:48.071+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:48.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:48 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:48.071+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:48.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:48 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:48.071+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:48.339 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:48 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:48.071+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:49.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:48 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:48.885+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:49.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:48 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:48.885+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:49.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:48 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:48.885+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:49.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:48 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:48.885+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:49.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:48 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:48.885+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:49.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:48 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:48.885+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:49.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:48 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:48.885+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:49.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:49 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:49.089+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:49.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:49 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:49.089+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:49.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:49 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:49.089+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:49.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:49 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:49.089+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:49.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:49 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:49.089+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:49.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:49 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:49.089+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:50.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:49 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:49.866+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:50.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:49 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:49.866+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:50.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:49 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:49.866+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:50.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:49 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:49.866+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:50.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:49 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:49.866+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:50.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:49 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:49.866+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:50.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:49 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:49.866+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:50.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:50 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:50.129+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:50.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:50 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:50.129+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:50.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:50 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:50.129+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:50.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:50 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:50.129+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:50.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:50 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:50.129+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:50.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:50 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:50.129+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:51.293 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:50 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:50.840+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:32.661960+0000 front 2023-09-23T09:46:32.662088+0000 (oldest deadline 2023-09-23T09:46:53.762140+0000) 2023-09-23T09:47:51.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:50 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:50.840+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:38.463362+0000 front 2023-09-23T09:46:38.463407+0000 (oldest deadline 2023-09-23T09:47:02.464333+0000) 2023-09-23T09:47:51.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:50 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:50.840+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:44.165471+0000 front 2023-09-23T09:46:44.165631+0000 (oldest deadline 2023-09-23T09:47:10.065411+0000) 2023-09-23T09:47:51.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:50 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:50.840+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-09-23T09:46:55.366439+0000 front 2023-09-23T09:46:55.366513+0000 (oldest deadline 2023-09-23T09:47:20.066306+0000) 2023-09-23T09:47:51.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:50 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:50.840+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:03.567381+0000 front 2023-09-23T09:47:03.567355+0000 (oldest deadline 2023-09-23T09:47:29.467429+0000) 2023-09-23T09:47:51.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:50 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:50.840+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:09.468151+0000 front 2023-09-23T09:47:09.467983+0000 (oldest deadline 2023-09-23T09:47:33.567967+0000) 2023-09-23T09:47:51.294 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:50 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10[131170]: 2023-09-23T09:47:50.840+0000 7f5fdcd22700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:15.869119+0000 front 2023-09-23T09:47:15.869226+0000 (oldest deadline 2023-09-23T09:47:41.768888+0000) 2023-09-23T09:47:51.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:51 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:51.179+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:51.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:51 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:51.179+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:51.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:51 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:51.179+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:51.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:51 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:51.179+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:51.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:51 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:51.179+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:51.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:51 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:51.179+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:51.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:51 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:51.179+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:51.964 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 23 09:47:51 smithi098 bash[145525]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-10 2023-09-23T09:47:52.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:52 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:52.176+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:52.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:52 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:52.176+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:52.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:52 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:52.176+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:52.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:52 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:52.176+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:52.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:52 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:52.176+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:52.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:52 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:52.176+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:52.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:52 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:52.176+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:53.119 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.10.service' 2023-09-23T09:47:53.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:47:53.160 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-23T09:47:53.160 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-23T09:47:53.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.11 2023-09-23T09:47:53.533 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.182+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:53.533 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.182+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:53.533 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.182+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:53.533 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.182+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:53.534 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.182+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:53.534 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.182+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:53.534 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.182+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:53.534 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 systemd[1]: Stopping Ceph osd.11 for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:47:53.534 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.530+0000 7f7905c8d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:47:53.534 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.530+0000 7f7905c8d700 -1 osd.11 112 *** Got signal Terminated *** 2023-09-23T09:47:53.534 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:53 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:53.530+0000 7f7905c8d700 -1 osd.11 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T09:47:54.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:54.185+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:54.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:54.185+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:54.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:54.185+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:54.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:54.185+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:54.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:54.185+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:54.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:54.185+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:54.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:54 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:54.185+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:55.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:55.168+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:55.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:55.168+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:55.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:55.168+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:55.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:55.168+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:55.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:55.168+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:55.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:55.168+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:55.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:55 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:55.168+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:56.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:56.122+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:56.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:56.122+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:56.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:56.122+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:56.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:56.122+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:56.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:56.122+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:56.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:56.122+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:56.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:56 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:56.122+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:57.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:57.163+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:57.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:57.163+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:57.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:57.163+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:57.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:57.163+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:57.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:57.163+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:57.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:57.163+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:57.590 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:57 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:57.163+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:58.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:58.181+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-09-23T09:46:29.756566+0000 front 2023-09-23T09:46:29.756700+0000 (oldest deadline 2023-09-23T09:46:53.856757+0000) 2023-09-23T09:47:58.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:58.181+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-09-23T09:46:40.759368+0000 front 2023-09-23T09:46:40.759268+0000 (oldest deadline 2023-09-23T09:47:06.059331+0000) 2023-09-23T09:47:58.588 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:58.181+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-23T09:46:48.960159+0000 front 2023-09-23T09:46:48.960194+0000 (oldest deadline 2023-09-23T09:47:13.660373+0000) 2023-09-23T09:47:58.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:58.181+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-09-23T09:47:04.162391+0000 front 2023-09-23T09:47:04.162564+0000 (oldest deadline 2023-09-23T09:47:28.262351+0000) 2023-09-23T09:47:58.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:58.181+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-23T09:47:11.763265+0000 front 2023-09-23T09:47:11.763301+0000 (oldest deadline 2023-09-23T09:47:37.063454+0000) 2023-09-23T09:47:58.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:58.181+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-09-23T09:47:17.064200+0000 front 2023-09-23T09:47:17.064147+0000 (oldest deadline 2023-09-23T09:47:42.964032+0000) 2023-09-23T09:47:58.589 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:58 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11[133296]: 2023-09-23T09:47:58.181+0000 7f790027f700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-09-23T09:47:27.565679+0000 front 2023-09-23T09:47:27.565633+0000 (oldest deadline 2023-09-23T09:47:50.465548+0000) 2023-09-23T09:47:59.838 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 23 09:47:59 smithi123 bash[148592]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-osd-11 2023-09-23T09:48:00.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@osd.11.service' 2023-09-23T09:48:00.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:00.953 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-23T09:48:00.953 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-23T09:48:00.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.a 2023-09-23T09:48:01.328 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:48:00 smithi045 systemd[1]: Stopping Ceph mds.a for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:01.579 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:48:01 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-a[151750]: 2023-09-23T09:48:01.326+0000 7ff58585d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:01.579 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:48:01 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-a[151750]: 2023-09-23T09:48:01.326+0000 7ff58585d700 -1 mds.a *** got signal Terminated *** 2023-09-23T09:48:05.830 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 23 09:48:05 smithi045 bash[180700]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-a 2023-09-23T09:48:05.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.a.service' 2023-09-23T09:48:05.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:05.982 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-23T09:48:05.983 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-23T09:48:05.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.d 2023-09-23T09:48:06.329 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:48:06 smithi045 systemd[1]: Stopping Ceph mds.d for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:06.829 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:48:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-d[150755]: 2023-09-23T09:48:06.372+0000 7fac0e626700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:06.830 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:48:06 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-d[150755]: 2023-09-23T09:48:06.372+0000 7fac0e626700 -1 mds.d *** got signal Terminated *** 2023-09-23T09:48:10.829 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 23 09:48:10 smithi045 bash[180846]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-d 2023-09-23T09:48:10.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.d.service' 2023-09-23T09:48:10.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:10.871 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-23T09:48:10.871 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-23T09:48:10.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.g 2023-09-23T09:48:11.247 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:48:10 smithi045 systemd[1]: Stopping Ceph mds.g for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:11.579 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:48:11 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-g[151368]: 2023-09-23T09:48:11.245+0000 7f4ec4420700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:11.579 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:48:11 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-g[151368]: 2023-09-23T09:48:11.245+0000 7f4ec4420700 -1 mds.g *** got signal Terminated *** 2023-09-23T09:48:15.770 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 23 09:48:15 smithi045 bash[180988]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-g 2023-09-23T09:48:15.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.g.service' 2023-09-23T09:48:15.810 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:15.810 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-23T09:48:15.810 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-23T09:48:15.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.j 2023-09-23T09:48:16.079 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:48:15 smithi045 systemd[1]: Stopping Ceph mds.j for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:16.579 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:48:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-j[150362]: 2023-09-23T09:48:16.185+0000 7f96a5d6e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:16.580 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:48:16 smithi045 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-j[150362]: 2023-09-23T09:48:16.185+0000 7f96a5d6e700 -1 mds.j *** got signal Terminated *** 2023-09-23T09:48:20.643 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 23 09:48:20 smithi045 bash[181138]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-j 2023-09-23T09:48:20.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.j.service' 2023-09-23T09:48:20.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:20.847 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-23T09:48:20.847 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-23T09:48:20.847 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.b 2023-09-23T09:48:21.196 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:48:20 smithi098 systemd[1]: Stopping Ceph mds.b for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:21.543 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:48:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b[137507]: 2023-09-23T09:48:21.193+0000 7f0045bb2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:21.543 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:48:21 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b[137507]: 2023-09-23T09:48:21.193+0000 7f0045bb2700 -1 mds.b *** got signal Terminated *** 2023-09-23T09:48:23.061 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 23 09:48:23 smithi098 bash[145783]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-b 2023-09-23T09:48:23.248 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.b.service' 2023-09-23T09:48:23.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:23.283 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-23T09:48:23.284 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-23T09:48:23.284 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.e 2023-09-23T09:48:23.543 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:48:23 smithi098 systemd[1]: Stopping Ceph mds.e for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:24.043 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:48:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-e[136820]: 2023-09-23T09:48:23.650+0000 7f62d112f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:24.043 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:48:23 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-e[136820]: 2023-09-23T09:48:23.650+0000 7f62d112f700 -1 mds.e *** got signal Terminated *** 2023-09-23T09:48:27.247 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 23 09:48:26 smithi098 bash[145918]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-e 2023-09-23T09:48:27.253 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.e.service' 2023-09-23T09:48:27.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:27.287 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-23T09:48:27.287 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-23T09:48:27.287 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.h 2023-09-23T09:48:27.543 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:48:27 smithi098 systemd[1]: Stopping Ceph mds.h for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:28.043 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:48:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-h[137166]: 2023-09-23T09:48:27.657+0000 7f7b5766d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:28.043 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:48:27 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-h[137166]: 2023-09-23T09:48:27.657+0000 7f7b5766d700 -1 mds.h *** got signal Terminated *** 2023-09-23T09:48:31.293 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 23 09:48:30 smithi098 bash[146064]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-h 2023-09-23T09:48:31.307 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.h.service' 2023-09-23T09:48:31.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:31.340 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-23T09:48:31.340 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-23T09:48:31.340 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.k 2023-09-23T09:48:31.718 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:48:31 smithi098 systemd[1]: Stopping Ceph mds.k for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:32.043 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:48:31 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-k[136479]: 2023-09-23T09:48:31.717+0000 7f77191ef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:32.043 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:48:31 smithi098 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-k[136479]: 2023-09-23T09:48:31.717+0000 7f77191ef700 -1 mds.k *** got signal Terminated *** 2023-09-23T09:48:34.581 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 23 09:48:34 smithi098 bash[146205]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-k 2023-09-23T09:48:34.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.k.service' 2023-09-23T09:48:34.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:34.815 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-23T09:48:34.815 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-23T09:48:34.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.c 2023-09-23T09:48:35.175 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:48:34 smithi123 systemd[1]: Stopping Ceph mds.c for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:35.588 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:48:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[143197]: 2023-09-23T09:48:35.173+0000 7fa793e59700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:35.588 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:48:35 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c[143197]: 2023-09-23T09:48:35.173+0000 7fa793e59700 -1 mds.c *** got signal Terminated *** 2023-09-23T09:48:39.088 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 23 09:48:39 smithi123 bash[148877]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-c 2023-09-23T09:48:39.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.c.service' 2023-09-23T09:48:39.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:39.277 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-23T09:48:39.277 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-23T09:48:39.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.f 2023-09-23T09:48:39.588 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:48:39 smithi123 systemd[1]: Stopping Ceph mds.f for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:40.088 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:48:39 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-f[139183]: 2023-09-23T09:48:39.646+0000 7f11535a6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:40.088 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:48:39 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-f[139183]: 2023-09-23T09:48:39.646+0000 7f11535a6700 -1 mds.f *** got signal Terminated *** 2023-09-23T09:48:43.308 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 23 09:48:42 smithi123 bash[149025]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-f 2023-09-23T09:48:43.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.f.service' 2023-09-23T09:48:43.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:43.348 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-23T09:48:43.348 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-23T09:48:43.348 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.i 2023-09-23T09:48:43.588 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:48:43 smithi123 systemd[1]: Stopping Ceph mds.i for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:44.087 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:48:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-i[139874]: 2023-09-23T09:48:43.709+0000 7f9cc56d8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:44.088 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:48:43 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-i[139874]: 2023-09-23T09:48:43.709+0000 7f9cc56d8700 -1 mds.i *** got signal Terminated *** 2023-09-23T09:48:47.258 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 23 09:48:46 smithi123 bash[149173]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-i 2023-09-23T09:48:47.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.i.service' 2023-09-23T09:48:47.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:47.296 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-23T09:48:47.297 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-23T09:48:47.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.l 2023-09-23T09:48:47.588 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:48:47 smithi123 systemd[1]: Stopping Ceph mds.l for 654e5bfa-59f2-11ee-8db4-212e2dc638e7... 2023-09-23T09:48:48.088 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:48:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-l[138845]: 2023-09-23T09:48:47.670+0000 7f3e8ae42700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T09:48:48.088 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:48:47 smithi123 ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-l[138845]: 2023-09-23T09:48:47.670+0000 7f3e8ae42700 -1 mds.l *** got signal Terminated *** 2023-09-23T09:48:52.047 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 23 09:48:51 smithi123 bash[149321]: ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7-mds-l 2023-09-23T09:48:52.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-654e5bfa-59f2-11ee-8db4-212e2dc638e7@mds.l.service' 2023-09-23T09:48:52.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T09:48:52.287 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-23T09:48:52.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 --force --keep-logs 2023-09-23T09:48:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:48:54.290 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 --force --keep-logs 2023-09-23T09:48:54.592 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:48:56.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 --force --keep-logs 2023-09-23T09:48:56.571 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:48:58.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:48:58.271 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:48:58.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T09:48:58.336 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-23T09:48:58.338 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi045/crash 2023-09-23T09:48:58.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash -- . 2023-09-23T09:48:58.387 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-23T09:48:58.387 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2023-09-23T09:48:58.389 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi098/crash 2023-09-23T09:48:58.390 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash -- . 2023-09-23T09:48:58.453 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-23T09:48:58.453 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2023-09-23T09:48:58.455 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi123/crash 2023-09-23T09:48:58.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash -- . 2023-09-23T09:48:58.505 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-23T09:48:58.506 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2023-09-23T09:48:58.506 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-23T09:48:58.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-23T09:48:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:grep: /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph.log: No such file or directory 2023-09-23T09:48:58.545 INFO:tasks.cephadm:Compressing logs... 2023-09-23T09:48:58.545 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T09:48:58.588 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T09:48:58.590 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T09:48:58.625 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-23T09:48:58.625 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-23T09:48:58.632 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-23T09:48:58.632 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-23T09:48:58.633 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.10.log 2023-09-23T09:48:58.633 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.h.log 2023-09-23T09:48:58.634 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-23T09:48:58.634 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-23T09:48:58.635 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.e.log 2023-09-23T09:48:58.635 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.10.log: /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.4.log 2023-09-23T09:48:58.635 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.1.log 2023-09-23T09:48:58.635 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.k.log 2023-09-23T09:48:58.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log 2023-09-23T09:48:58.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.7.log 2023-09-23T09:48:58.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.b.log 2023-09-23T09:48:58.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.b.log 2023-09-23T09:48:58.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.y.log 2023-09-23T09:48:58.639 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.y.log: 94.3% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.k.log.gz 2023-09-23T09:48:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.0.log 2023-09-23T09:48:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.g.log 2023-09-23T09:48:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.a.log 2023-09-23T09:48:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.6.log 2023-09-23T09:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.3.log 2023-09-23T09:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.d.log 2023-09-23T09:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.x.log 2023-09-23T09:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.9.log 2023-09-23T09:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.d.log: /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log 2023-09-23T09:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.j.log 2023-09-23T09:48:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.a.log 2023-09-23T09:48:58.646 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.5.log 2023-09-23T09:48:58.646 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.2.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.11.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.z.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.8.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.f.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.c.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.i.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.c.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log 2023-09-23T09:48:58.647 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.i.log: /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.l.log 2023-09-23T09:48:58.648 INFO:teuthology.orchestra.run.smithi098.stderr: 92.0% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.y.log.gz 2023-09-23T09:48:58.648 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.j.log: /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.a.log: 94.3% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.g.log.gz 2023-09-23T09:48:58.648 INFO:teuthology.orchestra.run.smithi045.stderr: 94.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.a.log.gz 2023-09-23T09:48:58.649 INFO:teuthology.orchestra.run.smithi045.stderr: 94.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.j.log.gz 2023-09-23T09:48:58.649 INFO:teuthology.orchestra.run.smithi045.stderr: 94.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.d.log.gz 2023-09-23T09:48:58.649 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log: /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.l.log: 94.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.l.log.gz 2023-09-23T09:48:58.649 INFO:teuthology.orchestra.run.smithi123.stderr: 92.1% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.z.log.gz 2023-09-23T09:48:58.679 INFO:teuthology.orchestra.run.smithi098.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-23T09:48:58.824 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-23T09:48:59.028 INFO:teuthology.orchestra.run.smithi098.stderr: 91.0% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log.gz 2023-09-23T09:48:59.323 INFO:teuthology.orchestra.run.smithi045.stderr: 90.0% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mgr.x.log.gz 2023-09-23T09:48:59.549 INFO:teuthology.orchestra.run.smithi123.stderr: 93.7% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.f.log.gz 2023-09-23T09:48:59.823 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log.gz 2023-09-23T09:48:59.997 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-volume.log.gz 2023-09-23T09:49:00.153 INFO:teuthology.orchestra.run.smithi123.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-23T09:49:00.325 INFO:teuthology.orchestra.run.smithi098.stderr: 93.6% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.e.log.gz 2023-09-23T09:49:00.644 INFO:teuthology.orchestra.run.smithi098.stderr: 91.9% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.b.log.gz 2023-09-23T09:49:00.772 INFO:teuthology.orchestra.run.smithi045.stderr: 90.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.a.log.gz 2023-09-23T09:49:00.817 INFO:teuthology.orchestra.run.smithi123.stderr: 93.9% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.c.log.gz 2023-09-23T09:49:00.950 INFO:teuthology.orchestra.run.smithi123.stderr: 91.9% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mon.c.log.gz 2023-09-23T09:50:15.597 INFO:teuthology.orchestra.run.smithi045.stderr: 91.6% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.9.log.gz 2023-09-23T09:50:31.006 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.3.log.gz 2023-09-23T09:50:31.471 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.6.log.gz 2023-09-23T09:50:39.752 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.0.log.gz 2023-09-23T09:50:39.766 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-09-23T09:50:39.766 INFO:teuthology.orchestra.run.smithi045.stderr:real 1m41.158s 2023-09-23T09:50:39.766 INFO:teuthology.orchestra.run.smithi045.stderr:user 4m9.816s 2023-09-23T09:50:39.766 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m8.145s 2023-09-23T09:51:30.055 INFO:teuthology.orchestra.run.smithi123.stderr: 91.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.8.log.gz 2023-09-23T09:51:45.084 INFO:teuthology.orchestra.run.smithi123.stderr: 91.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.2.log.gz 2023-09-23T09:51:48.285 INFO:teuthology.orchestra.run.smithi123.stderr: 91.3% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.5.log.gz 2023-09-23T09:52:09.988 INFO:teuthology.orchestra.run.smithi098.stderr: 91.6% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.4.log.gz 2023-09-23T09:52:22.337 INFO:teuthology.orchestra.run.smithi123.stderr: 91.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.11.log.gz 2023-09-23T09:52:37.439 INFO:teuthology.orchestra.run.smithi098.stderr: 91.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.10.log.gz 2023-09-23T09:52:54.637 INFO:teuthology.orchestra.run.smithi123.stderr: 97.1% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.i.log.gz 2023-09-23T09:52:54.644 INFO:teuthology.orchestra.run.smithi123.stderr: 2023-09-23T09:52:54.644 INFO:teuthology.orchestra.run.smithi123.stderr:real 3m56.038s 2023-09-23T09:52:54.644 INFO:teuthology.orchestra.run.smithi123.stderr:user 5m35.645s 2023-09-23T09:52:54.644 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m14.416s 2023-09-23T09:52:58.517 INFO:teuthology.orchestra.run.smithi098.stderr: 91.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.1.log.gz 2023-09-23T09:53:50.271 INFO:teuthology.orchestra.run.smithi098.stderr: 97.6% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.b.log.gz 2023-09-23T09:53:56.618 INFO:teuthology.orchestra.run.smithi098.stderr: 91.4% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-osd.7.log.gz 2023-09-23T09:55:23.183 INFO:teuthology.orchestra.run.smithi098.stderr: 97.7% -- replaced with /var/log/ceph/654e5bfa-59f2-11ee-8db4-212e2dc638e7/ceph-mds.h.log.gz 2023-09-23T09:55:23.194 INFO:teuthology.orchestra.run.smithi098.stderr: 2023-09-23T09:55:23.194 INFO:teuthology.orchestra.run.smithi098.stderr:real 6m24.587s 2023-09-23T09:55:23.194 INFO:teuthology.orchestra.run.smithi098.stderr:user 7m18.170s 2023-09-23T09:55:23.194 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m20.018s 2023-09-23T09:55:23.194 INFO:tasks.cephadm:Archiving logs... 2023-09-23T09:55:23.194 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi045/log 2023-09-23T09:55:23.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T09:56:00.068 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi098/log 2023-09-23T09:56:00.069 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T09:56:52.965 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi123/log 2023-09-23T09:56:52.967 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T09:57:40.489 INFO:tasks.cephadm:Removing cluster... 2023-09-23T09:57:40.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 --force 2023-09-23T09:57:41.227 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:57:42.049 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 --force 2023-09-23T09:57:42.883 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:57:43.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 654e5bfa-59f2-11ee-8db4-212e2dc638e7 --force 2023-09-23T09:57:44.420 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 654e5bfa-59f2-11ee-8db4-212e2dc638e7 2023-09-23T09:57:45.330 INFO:tasks.cephadm:Removing cephadm ... 2023-09-23T09:57:45.330 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-23T09:57:45.352 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-23T09:57:45.392 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-23T09:57:45.442 INFO:tasks.cephadm:Teardown complete 2023-09-23T09:57:45.442 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-23T09:57:45.455 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-23T09:57:45.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T09:57:45.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T09:57:45.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T09:57:45.597 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-23T09:57:45.608 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-23T09:57:45.609 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T09:57:45.612 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T09:57:45.615 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T09:57:45.629 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-09-23T09:57:45.631 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-09-23T09:57:45.631 INFO:teuthology.orchestra.run.smithi098.stderr:bash: ntpq: command not found 2023-09-23T09:57:45.647 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T09:57:45.647 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-09-23T09:57:45.647 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 -88us[ -65us] +/- 80ms 2023-09-23T09:57:45.648 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -971us[ -948us] +/- 53ms 2023-09-23T09:57:45.648 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +1428us[+1428us] +/- 75ms 2023-09-23T09:57:45.648 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:57:45.655 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T09:57:45.655 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-09-23T09:57:45.655 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 -66us[ -66us] +/- 80ms 2023-09-23T09:57:45.656 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv02.front.sepia.ceph.com 3 6 377 23 -966us[ -964us] +/- 53ms 2023-09-23T09:57:45.656 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +1463us[+1463us] +/- 75ms 2023-09-23T09:57:45.656 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:57:45.661 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T09:57:45.661 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2023-09-23T09:57:45.661 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 -102us[ -102us] +/- 80ms 2023-09-23T09:57:45.661 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -1008us[-1002us] +/- 53ms 2023-09-23T09:57:45.661 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +1404us[+1409us] +/- 75ms 2023-09-23T09:57:45.661 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T09:57:45.663 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-23T09:57:45.674 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-23T09:57:45.675 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-23T09:57:45.702 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T09:57:45.839 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T09:57:46.038 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T09:57:46.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T09:57:46.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:57:46.236 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T09:57:46.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:57:46.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T09:57:46.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:57:46.305 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-23T09:57:46.316 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-23T09:57:46.360 INFO:teuthology.task.internal:Duration was 2951.503217 seconds 2023-09-23T09:57:46.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-23T09:57:46.411 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-23T09:57:46.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T09:57:46.414 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T09:57:46.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T09:57:46.481 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T09:57:46.490 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T09:57:46.524 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T09:57:46.873 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-23T09:57:46.873 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2023-09-23T09:57:46.874 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T09:57:46.901 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2023-09-23T09:57:46.901 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T09:57:46.935 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2023-09-23T09:57:46.936 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T09:57:46.965 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-23T09:57:46.965 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T09:57:46.968 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T09:57:46.979 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T09:57:47.069 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-23T09:57:47.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T09:57:47.072 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T09:57:47.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T09:57:47.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-23T09:57:47.201 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-23T09:57:47.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T09:57:47.261 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T09:57:47.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T09:57:47.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-23T09:57:47.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T09:57:47.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T09:57:47.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T09:57:47.379 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2023-09-23T09:57:47.393 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2023-09-23T09:57:47.404 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2023-09-23T09:57:47.441 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T09:57:47.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:57:47.458 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T09:57:47.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:57:47.476 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T09:57:47.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T09:57:47.498 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-23T09:57:47.511 INFO:teuthology.task.internal:Transferring archived files... 2023-09-23T09:57:47.512 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi045 2023-09-23T09:57:47.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T09:57:47.560 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi098 2023-09-23T09:57:47.560 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T09:57:47.604 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/remote/smithi123 2023-09-23T09:57:47.604 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T09:57:47.648 INFO:teuthology.task.internal:Removing archive directory... 2023-09-23T09:57:47.649 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T09:57:47.651 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T09:57:47.654 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T09:57:47.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-23T09:57:47.711 INFO:teuthology.task.internal:Not uploading archives. 2023-09-23T09:57:47.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-23T09:57:47.741 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-23T09:57:47.741 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T09:57:47.744 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T09:57:47.747 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T09:57:47.758 INFO:teuthology.orchestra.run.smithi045.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 09:57 /home/ubuntu/cephtest 2023-09-23T09:57:47.761 INFO:teuthology.orchestra.run.smithi098.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 09:57 /home/ubuntu/cephtest 2023-09-23T09:57:47.763 INFO:teuthology.orchestra.run.smithi123.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 09:57 /home/ubuntu/cephtest 2023-09-23T09:57:47.776 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-23T09:57:47.788 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-23T09:57:47.895 INFO:teuthology.nuke:Checking targets against current locks 2023-09-23T09:57:47.923 DEBUG:teuthology.nuke:shortname: smithi045 2023-09-23T09:57:47.923 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T09:57:47.948 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 09:03:12.678871', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T09:57:47.953 DEBUG:teuthology.nuke:shortname: smithi098 2023-09-23T09:57:47.953 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T09:57:47.978 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 09:03:12.676220', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T09:57:47.984 DEBUG:teuthology.nuke:shortname: smithi123 2023-09-23T09:57:47.984 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T09:57:48.012 INFO:teuthology.orchestra.console.smithi045:Power off 2023-09-23T09:57:48.013 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T09:57:48.026 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 09:03:12.677677', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T09:57:48.040 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2023-09-23T09:57:48.041 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:48.064 INFO:teuthology.orchestra.console.smithi098:Power off 2023-09-23T09:57:48.064 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T09:57:48.074 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-09-23T09:57:48.087 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2023-09-23T09:57:48.087 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:48.099 INFO:teuthology.orchestra.console.smithi123:Power off 2023-09-23T09:57:48.100 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T09:57:48.109 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-09-23T09:57:48.123 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-09-23T09:57:48.123 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:48.144 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-23T09:57:52.075 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:52.110 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:52.146 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:52.185 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-09-23T09:57:52.218 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-09-23T09:57:52.254 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-23T09:57:56.190 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:56.219 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:56.255 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T09:57:56.299 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2023-09-23T09:57:56.327 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-09-23T09:57:56.363 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-09-23T09:57:56.401 INFO:teuthology.orchestra.console.smithi045:Power off completed 2023-09-23T09:57:56.429 INFO:teuthology.orchestra.console.smithi098:Power off completed 2023-09-23T09:57:56.464 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-09-23T09:57:56.569 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} duration: 2951.5032172203064 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2023-09-23T09:57:56.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T09:57:56.656 INFO:teuthology.run:FAIL